Did anybody else hit their concurrency limits on Lambda? We had a couple of functions that went into a death loop but because the limit is global, they take down our entire service landscape.
I know similar failure patterns can arise in different architectures and we are mitigating this, but the concurrency being a shared limit feels to me like a fundamental problem of the lambda architecture.
Then again most typical Lambda users will probably never hit it, but we are not a typical lambda user.
If anybody else has good examples of solutions, I’d be very curious. I saw some blog posts with full or half baked lambda circuit breakers but none of them felt like they would really be suitable to use at scale.