AWS SQS event Trigger for Lambdas

Serverless Framework
,
1

Hey.

I wanted to ask in Serverless when it will be added the support for SQS trigger for lambdas as described in the article below:

As now its not efficient to use SNS and SQS together or the cron job approach where lambda is frequently checking for SQS records.

Thanks.

1 Like
2

I know its no where as clean, but may be able just generate a AWS::Lambda::EventSourceMapping resource. From their could populate the needed SQS information.

3

The work has already been completed. It will be in v1.28.

1 Like
4

Hope to see it ASAP. I was reading some posts that FIFO queues are not supported for this trigger? Can you tell me if you know about it?

5

FIFO is not supported since it couldnā€™t scale past a concurrency of 1. I ran some tests and wrote about the behavior here:

6

Fair enough. So for FIFO we are still left with cron lambdas. I read you tests I want to run it for more than 5000 queues same way and see if its still processing successfully without failures. Based on your example it need to scale without issues. Thanks for article.

1 Like
7

How important is processing order? If you can scale up your consumers, the order differences will probably be fairly minimal since they process so quickly.

Iā€™m using a FIFO queue in one of my apps now that I trigger with CloudWatch, and then trigger again at the end of every invocation until the queue is drained. I really hate the way it works, so Iā€™m thinking about using SQS triggers and keeping the concurrency low to minimize the impact on order. I need to run some tests, but if it works, it will be so much cleaner.

8

I cannot speak for other peopleā€™s use cases, but I have numerous ETL scripts where order is imperative and FIFO queues are required. That being said, I can imagine there are plenty of other people, also doing ETL work where FIFO queues are required. In these cases, since they are out-of-band ETL processes, the order is more important than the performance.

9

Yeah Jeremy theres a sub cases that Im facing too that the order is very important here and ruining the order under high loads of queues will give catastrophic results on the apps. So I donā€™t know if your approach will work for me as the apps must to work stable no mater the scale of the app is. Thats why I love Serverless architecture.

10

I get that processing order is sometime imperative. I think the use case for SQS triggers doesnā€™t really make a lot of sense in those cases. If ā€œspeedā€ isnā€™t an issue, then perhaps CloudWatch schedule or log triggers are the right way to go.

1 Like
11

Or Kinesis if you keep it to a single stream.

2 Likes
12

Looking at the documentation for AWS SQS, I saw that FIFO queues donā€™t support Lambda function triggers. Since this is an AWS limitation, I am now not going to expect SLS to solve it. It just wouldnā€™t make sense ā€¦ but at least there are couple of work arounds