Accidentally overwrote entire stack

I was working on my rather large stack, and needed to create a small test stack to provide a component needed to test my real stack. When doing so, I copy&pasted my serverless.yml, and forgot to change the very first line - the stack name.
My surprise when my entire stack was deleted without warning was … substantial. Lambda functions, API gateway, DynamoDB tables full of content, …

Is this intended? Shouldn’t it be possible to detect if a stack is ENTIRELY different than the existing stack and if so warn about completely overwriting it?

In my case nothing was lost from production since it was during development, but still a LOT of time was wasted for me.

Is this expected? Yes.

CloudFormation (the underlying AWS service that handles deployment) will happily add and remove any resources required.

I would strongly recommend two things:

  1. Automate deployments to production and do not allow anyone (including yourself) to by-pass that without jumping through hoops.
  2. Add a DeletionPolicy, UpdatePolicy and UpdateReplacePolicy to protect resources that cannot easily be re-created.

As a side note: “substantial” is very subjective. A small change to the stack like removing a DynamoDB table or Cognito User Pool might be only one resource but it’s substantial because you cannot replace the data in it.

Thanks. It makes sense I guess, and while you’re right that even a small change to delete a single resource could be catastrophic, that wouldn’t typically happen in the scenario where someone accidentally used the same stack name as something existing, and you’d think that it would at least be quite easy to detect that a deployed template is, say, more than 50% different from the existing stack and at least prompt a warning.

But anyway, this was nowhere near tearing down something from production, but a valid lesson learnt the hard way I guess. I’m aware of DeletionPolicies and have been using them for things like DynamoDB tables in the past, so maybe this was a good wake-up call to start using them again…

1 Like