Back in November of 2015 I taught myself CloudFormation on the Amtrak ride from DC to NY. As I was building out my ultimate VPC template, I began to discover the limitations of CloudFormation. There was no clean way to link the output of a stack to the inputs of another stack. All the examples I’d seen at work had all the settings defined as defaults. Meaning that sharing CFTs was a risky proposition and code-reuse was very limited.
Nor was there wasn’t a good way to handle the things that weren’t yet in CloudFormation (back then it was NAT Gateways).
Finally, json was the only supported template format, and it was impossible to include comments in the json template.
Over that Thanksgiving weekend I wrote deploy_stack.rb, where I decided that all the CloudFormation options and parameters should be kept in a Manifest file. The Manifest file would support the idea of doing parameter look-ups from other existing stacks. There was also the intention to support converting a YAML template into json, but that became un-necessary as AWS introduced native YAML templates at that year’s re:Invent. Finally I messed around with trying to include a pre-install and post-install script (like in rpm & dpkg).
deploy_stack.rb served me reliably for many years, but as I was building Antiope and using Makefiles for deployment, I realized it was time to dust off this code. Also I maintain stacks in my protected accounts from my security account, and I needed a better way to do that.
So I decided to re-implement deploy_stack.rb in Python, and release it on PyPi.
In addition to three python classes for Templates, Stacks and Manifests, it includes a handful of scripts to make CloudFormation deployment easier:
- cft-validate - Will validate a template with the AWS CloudFormation service
- cft-upload - Will upload a CFT to S3, which is required if the template is over a certain size
- cft-generate-manifest - Will take a local or s3-hosted template, and generate a manifest file
- cft-validate-manifest - Will perform all of the parameter substitutions and validate that dependencies exist
- cft-deploy - Will take the manifest (and optional command-line params) and create or update the stack (providing a tail -f like experience of the events)
- cft-delete - Will delete the specified stack (providing a tail -f like experience of the deletion events)
These can be run standalone, or as part of a Makefile (which is how they’re used by Antiope).
The source code and docs are on github. You can install it via pip3 install cftdeploy. It is a work in progress, so pull-requests are appreciated.
This was my first time building a Python module I intended to share publicly, my first time pushing something to PyPi, and my first time really using the PEP8 standard.