ertius.org

cloudformation is a pain

#aws #cloudformation #devops #infrastructure #sre-brain-damage

I've spent a bunch of time fiddling around trying to build a CloudFormation thingo to create a whole website - from some creds and a domain name, create a Route53 zone, an S3 bucket, a CloudFront distribution, and an SSL cert. It isn't that hard (domain/cert, s3/cloudfront) in the end, but it was super fiddly, especially since I have barely used CF in 10 years.

Some things that made it hard:

  1. out of date posts online, especially from before S3 locked everything down by default. Approximately all examples or discussions about ACLs from more before ~early 2023 don't work.
  2. Related to that, Cloudfront has a new way to auth to S3, which is not very well represented in online examples, but is simple enough once it's set up and is nice because it doesn't require exposing S3 to anything else at all. However, if you do that, then you can't have S3 provide index.html rewriting for paths like https://example.org/foo/. There is a workaround, though! You can make Cloudfront execute random JS on every request which does the redirect inside Cloudfront - AWS even documents this rather mad idea themselves (my use of it).
  3. Speed, it can take 10 minutes+ for a run to fail, with no logging in the meantime, and even in the case of things that appear to be data consistency errors that could have failed very fast. Googling what errors eventually appeared in the UI eventually found someone else having that problem in the past.
  4. Debugging ACLs is extremely annoying, since things just fail to work with no error. I feel like there's some massively annoying way to get errors logged to Yet Another S3 Bucket and then have some other thing analyse them, but I just want a log or access denied message ffs.

On the plus side:

  1. CF supports YAML now, which is - amazingly - less crappy than AWS's super verbose JSON.
  2. vscode has a very elaborate CF mode that inlines cfn-lint's endless complaints.
  3. splitting it up into two templates - one for the domain/ssl cert, which can't work automatically because of the need to delegate NS records from the parent zone, and one for the hosting, which can be automated - makes it much easier to test and debug.