Apologies if this is really basic, but I’d love to not architect this incorrectly from the start
When, in a serverless/microservices world, what’s the best way to interact with other functions in different cases?
Example: Function1.js looks up users in DynamoDB. Function2.js sends sales email to those users.
–I assume I should just require Function2.js into Function1.js and call the function directly, or should I make a full API Gateway call to Function2?
Example: GroupOfFunctionsA handles user interactions (like looking up users and sending emails to them) GroupOfFunctionsB handles cron jobs/events that schedule things like when you should invoke GroupOfFunctionsA
–Should I use Lambda.Invoke to launch GroupOfFunctionsA from within GroupOfFunctionsB or should this be an API call or require() situation?
Obviously interacting with external apps or organizations, that’s all API calls, but internally, is there something I should read, or someone who has advice?
As a general rule I don’t have one Lambda directly invoke another. In this case I think it’s important to consider that having one Lambda lookup users and another send the emails is both slower and more expensive than having everything inside a single Lambda. Unless there’s a strong reason to build it this way I’d build one Lambda that does both.
Currently there are two reasons I split Lambda’s:
To improve user response times.
To decouple parts of the code.
For example: I have a Lambda that handles an OAuth callback that meets both of those criteria. After exchanging the OAuth token it needs to make a couple of external API calls that would delay the API response but don’t actually change what’s returned. The callback Lambda stores the access token in DynamoDB then pushes an app installed message to an SNS topic.
I have three other Lambda’s that subscribe to the app installed topic and take action. Technically I could have used a single Lambda. I’ve taken this approach because
Each Lambda does one thing that’s full independent of the other Lambda’s (i.e. they can run in parallel)
It makes it easy add/remove functionality that happens on app installation without needing to modify existing code.
SNS will retry if the Lambda fails which provides a convenient way to handle API throttling by the third party without restarting the entire process.
Why shouldnt a lambda function call another lambda? We were thinking of splitting a lambda function because we are going to have to make several rest calls for each request we receive, some of which connect to slow rest servers sometimes taking 2 to 3sec handle a request, therefore we were thinking of splitting the functions into one that handles requests (main lambda), and another one that caches data from db (db-lambda) (as our db dataset is very small but business requires us to use a db). we thought that we could have the db-lambda serving more instances of the main lambda as requests would be handled very fast).
Would this approach be wrong as per your post?
Also why is it bad to connect directly to another lambda function, wont people that use micro-service pattern be doing this?
It’s not a hard rule - there are definitely times when it’s fine to call a function directly from another function (hey, I even did it in my course to keep things simple).
The main concern I have is with tight coupling of functions (leading to brittle code, or hard to debug/trace errors). This can totally be mitigated by design. Using Lambda DLQs is one way to deal with the downsides.
The use case you have described is one I would watch out for - What happens if one of those secondary requests fails/takes a long time? Does the initial request fail? What about all those other secondary requests that succeeded? Now you have to trace all the requests to work out where the real error is. With something in-between (e.g. SNS or SQS) you can build-in sensible retry logic, or at least report where the exact failure is cleanly.
This may not be a big issue in your system, but one I would watch out for (as systems have a tendency to grow bigger than initially expected, if they’re useful).
Buggy is on the money, always aim for event driven development with functions, though a good rule of thumb is, if it’s a request/response type user case then calling one function from another can be implemented, but as mentioned already, you’re responsible for retries, timeouts, failures and corolation tracking of events, etc, so try to change the use case if possible, makes it easier in the long run.
Thanks for your replies, i see what you mean with issues in debugging, as we would have to trace any issues between two lambda functions.
So are you saying that microservices pattern is not a good pattern to use within lambda, however at the moment if data is not found within db we return an error, we are looking at a similar approach if we migrate db access to a different lambda.
If we have two lambdas that execute a very different speeds;
Main lambda would have to execute auth validation, and several other requests to return valid data.
Db-lambda would just mostly fetch cached data, therefore it should be much quicker?
Would this be more error prone isnt this the same as making another async rest request? And why? I can see that this would be slower than keeping it in the same lambda, but wouldnt the db-lambda be able to serve more instances of the main lambda?
Why is it important to have complex retry policies? Couldn’t existing error handling logic be applied?
Apologies for all the questions, but both answers created more questions.
Hmm, no we’re not saying that; We’re just pointing out that doing microservices properly usually involves things like messaging layers.
In a complex application it can be hard to differentiate between a timeout and a failure, and what to do about it. Messaging layers is one of the common ways to address that. Of course you can write your own error handling logic to do this, but then that’s more work you have to do yourself rather than leveraging existing services.
Lambda is great for Microservices. The API Gateway pattern is a major exception to my rule that “if you have one Lambda calling another you’re probably doing it wrong”. It makes complete sense to build an API gateway as a single microservice handling common tasks like authentication then invoke Lambdas from another microservice to perform actions.
The problem is that a lot of people (especially starting out) equate a Lambda function to a regular function inside their code and start splitting up validation, database access, sending emails, etc as separate Lambda’s all invoked from a main Lambda.
Trying thinking of the code in your Lambda as stuff that needs to happen before you respond to the user and stuff that can happen after. Anything that needs to happen before should be part of a single Lambda. Anything that can happen after should generally be connected via SNS which gives your lose coupling, retries, etc.
An example:
A customer places an order (order service). You need to tell the warehouse (warehouse service) to pick the item and package it for shipping. If you’re invoking Lambda’s directly then either:
The order service can’t create an order unless it can communicate with the warehouse service, or
You now need to build a retry mechanism into the order service.
But there are more problems:
Your order service needs to understand which function to call and how to call it in order to initiate picking. What happens if this changes in the warehouse service?
What happens if the warehouse services needs to call other services? Could the order services now get delayed for seconds or minutes while accepting an order?
When you need to add/remove other functionality (perhaps sending a receipt from a receipt service) then you need to keep modifying the order service.
The alternative is to publish a “new order” message via SNS. Services that are interested in knowing about new orders can listen for this. If the warehouse service is temporarily down then orders don’t stop. When the receipt service is added the order service isn’t modified. This is loose coupling.
This doesn’t mean that one Lambda never needs to invoke another. But you should ask yourself
Does this functionality really belong in another Lambda?
Hey, i’m tasked with building a service which will pull from a database then for each record, call a lambda function. Each request takes seconds to complete, which is why we were going to run it in parallel it by having each record call a separate API request to the lambda. How do I invoke another lambda from within it? Is there a better way to run this? We wanted to run each request through lambda so we can get logs and see reporting and other metadata.
@neoadventist The first thing I would try is using SNS to handle the mismatch between the rate at which you can read from the database and the rate you can process the records. If it works then it’s much easier than writing code to manage parallel invocations and retries.
Hey, I think it’s worth mentioning AWS Step Functions as a solution to chaining lambda functions without hard coupling one to another. Step functions is great for acting as a state machine between events.
I actually have an oauth function too… but in there I need to do a couple of things synchronously really though so the info is available immediately(save user and 2 other entity types to dynamodb tables etc).
My ‘user’ microservice though - should I call it through the api gateway or just through lambda invocation?
does a directly lambda invocation bypass any authorizors btw ?
Let’s say I had a service to create orders. I need to validate that the product id’s are valid in the order, and that the customer has the appropriate settings and subscription plan to pay for this order.
What I would expect to do is to create an orders::createOrder lambda function, which would make parallel calls to products::listProducts and customers::getCustomer to retrieve the information needed for validation, before I am happy to create the order in the system. This needs to happen synchronously as it’s a request/response from a third party to create the order.
I would usually want the logical domains - customers, products, orders to be separate microservices, how do I correctly model this on serverless if I am not supposed to call Lambdas from lambdas?
