Convince me to use API-Gateway and not call Lambda direct

aws

#1

What is the advantage of putting API-Gateway in front of my Lambda calls? Some AWS people are pushing me to do that but I can’t get a clear explanation of why I would want to do that. My Lambdas already have individual roles and policies that totally lock them down. We are serverless so no need for the gateway to talk to EC2 or any other external servers. The front ends (web/IOS/Android) use AWS_IAM security,

One argument is rate limiting control of API calls for DOS protection. Surely there are more advantages, can someone please explain?


#2

Calling lambda’s directly is fine (and cheaper)

If you need to expose the lambda to a broader audience (web) then api gateway is it.

Some cool things about api gateway is custom authorizers, stage variables, and like you mentioned api rate limits.


#3

Hi @jonsmirl

The fundamental thinking behind the lambda is event driven design. When ever an event triggered by the user only a small amount of code (Lambda function) will execute. AWS provides various events to trigger lambda execution API Gateway and CLI calls is 2 of those.

CLI calls are good if the lambda is not going to be distributed to to mass audience and used by a Team in closed environment. The reason for this is to execute lambda from CLI you need to create an IAM user and give him sufficient permissions, which will become very difficult if functionality of lambda needs to be exposed to large group. For larger reach API Gateway with Custom Authorizer is very good option.


#4

My lambdas use AWS Cognito logins to get temp credentials. The role associated with the Cognito pool only allows execution of specific set of lambdas listed in the policy. Inside side each lambda the Cognito user ID that is passed in vai the environment is used to keep the users from messing with each other. This setup does not require external people to have a IAM user.

Flow is like this:
User authenticates via Google/Facebook/Cognito User pool.
Authentication provides a JWT token
Present the JWT token to the Cognito Federation pool
Receive temporary IAM credentials for the role associated with the Federation pool
Use those temp IAM credentials to call the lambda functions.
The policy in the role locks the functions down so that you can only execute them.
The Federation pool has also assigned the JWT user a Cognito ID.
That Cognito ID is provided inside the environment for the lambda function.
Now the lambda function can use it to keep the users separate.
You can’t mess with the Cognito ID since it has been encrypted into the temp credentials.


#5

Your implementing is perfectly all right as you (or your team) are using all the lambdas in your code / application and not exposing them to any thirdpary / team for using in their code.

API Gateway is mainly for exposing the functionality to different team / users to used this their code and getting standard REST api signature for the calls

Only issue with your implementation is you do not have control over no of execution of lambda which may cost you unexpected money. I personally paid the price on similar implementation during development when one of my developer created an infinite condition in the code and ended up additional cost of several hundred $


#6

I think this is totally dependent on your application, but one of the main reason for using an API over a direct invocation is adaptability to change. It is, in my opinion, not just for when you need to expose your services externally. What happens if 6 months from now your project lead or executives say they want you to use Docker containers (or worse, VMs) to run a particular service currently handled by a Lambda function? If you only have direct invocations, then you will find yourself having to cascade down all the callers and have them point to the new service–this may involve lots of code modifications.

Contrast that with having a pre-defined API which all your callers use. All you would have to do is modify the API to point to your new resource.

I think overall it’s just a matter of loose coupling between micro services–the same reason why you often want to use queues between them if you want more of a pull mechanism rather than push.


#7

Definitely depends on your domain, but as someone who mostly works with APIs and mobile apps to consume them, I wouldn’t use Lambda directly.

Having an intermediary like API Gateway between your app and lambdas is pretty valuable in itself. It decouples your mobile clients from needing to know about all the lambdas, and from them changing.

You can’t force iOS and Android app updates, so whatever coupling you put out in the wile, needs to supported for quite a while. Even with automatic updates, this is at least months until there’s an insignificant number of people who haven’t updated (depending on your domain).

So as your backend evolves, do you just deploy new Lambdas and leave the old ones running for backwards compatibility?

If you change things with your AWS roles and IAM, does that break all existing apps in the wild?

I quite like having a standard HTTP interface so I can manage what’s exposed to apps better.


#8

You still have to leave all of the old lambdas running with API gateway to support the different API versions in the gateway. API Gateway does not make that problem go away.


#9

Yeah, it wont solve the API versioning problem, but it does decouple consumers from knowing anything more than endpoints and payload contracts.

I change my lambdas and internal structure more often than I change the external API contract, so I like the flexibility it gives me.


#10

A new reason I’ve heard is because API Gateway some how deploys into Cloudfront. I believe the reasoning is that you can hide a multi-regional deployment behind Cloudfront and the app is then not aware it is multi-regional. With direct lambda calls the regions are visible. Is this something worth considering? We are a long ways away from needing multi-region support.

Related to this is the ability to use Clouldfront and API gateway together to get rid of the need for CORS.
https://apimeister.com/2017/05/09/hosting-a-cloudfront-site-with-s3-and-api-gateway.html


#11

Hey, sorry to post this so long after the thread, but I just wanted to follow-up to see how this is working out for you.

As far as I can tell for my use case (express app on Lambda), API Gateway is doing nothing for me. It may even be constraining me a bit due to some of its own inherent limitations/quirks.

So, I’d like to dump it and just allow direct access (via CloudFront) to my lambda functions.

So, how it working out for you?

Thanks,
Ryan


#12

The web app I work on has been changed to do direct lambda access. It uses AWS Amplify to do that. Check out the docs here, there is example for both lambda and API Gateway

https://aws.github.io/aws-amplify/media/api_guide

There is nothing illegal about doing direct lambda access from a web app, it is just slightly harder to do than API Gateway since lambda is restricted to only returning JSON. If you need something other format returned you have to deal with that in your web app. Most mobile apps only do direct lambda access and ignore API Gateway. That’s why I thought it was useless, I had previously worked on the mobile app.

I also ignore cloudfront for lambda access. The cloudfront cache is pointless, these are API calls. I do use cloudfront to serve up my web app.


#13

isnt AWS Amplify API is API Gateway and not direct lambda execution ?


#14

You can use it either way. You can set a direct target of aws lambda into the API object in Amplify.
I am using Amplify to directly call lamda.


#15

oh is that possible ? could you suggest how to do it.


#16

Amplify.configure({
Auth: {
identityPoolId: config.IdentityPoolId,
region: ‘us-east-1’,
userPoolId: config.UserPoolId,
userPoolWebClientId: config.UserPoolClientId,
},
API: {
endpoints: [
{
name: ‘api’,
endpoint: ‘https://lambda.us-east-1.amazonaws.com’,
service: ‘lambda’,
},
],
},
});


#17

Sorry to bring up this old post. I’ve some questions on your flow. Mine is as follows:

  1. Upon successfully authentication via Cognito, my user will get temp IAM credentials with “Auth_Role” role
  2. I’ve setup a policy for “Auth_Role” to allow [“lambda:InvokeFunction”]. So that only authenticated user can invoke the lambda function.

However, I’m a bit confused about the “Execution Role” in the Lambda setup. Am I correct to say that:

  1. My temp IAM credential only controls who can invoke the lambda function, and when my function runs, it will assume the “Execution Role”?
  2. If yes, is there a way where I could cascade down my “Auth_Role” down as my “Execution Role”. (assuming here that I might have multiple roles who can invoke my lambda function)

Also, you mentioned you used the Cognito ID in the lambda function to keep the users separate. It is more for your application logic purposes? Or is it used for managing access to AWS resources? Or something else I should be aware of?


#18

Everyone gets confused at this point when using AWS. First consider that anyone using your app can mess with the code and get access to the Cognito credentials (not too bad since they expire in a couple of hours). So you really want to lock down what the Cognito credentials can do, for example mine are locked down to the absolution minimum possible. Like no access to dynamodb but they can trigger the execution of specific lambda functions.

Once you use the locked down Cognito credentials to run a lambda function, you want that lambda function to be able to do something, right? That’s where execution credentials come in. I set specific credential onto each function that allow the minimum privs needed for that function to run. So lets let this function read from dynamodb.

Now you have to use some application logic. The caller’s CognitoID was securely passed into your lambda function in the environment. Get this ID from the environment and use it in your code to limit what the function can access in dynamodb.

It doesn’t work for the lambda function to run under then Cognito credentials, remember those credential don’t have access to dynamodb.

After a while you will learn that it is possible to give permissions to directly access dynamodb using you Cognito credentials. But I don’t recommend doing that. It is possible to get it to work and it might be a tad faster, but you will have to be very careful in considering how that permission can be abused by a hostile user. For example it is probably unwise to let anyone directly write to the database without a lambda in front validating the write data.


#19

Yep … coming from WCF background, I’m really getting confused.

So, am I right to say that the main design considerations are:

  1. Only allow authenticated user via Cognito Federation pool to be able to invoke Lambda function
  2. For each Lambda function, assign the minimum privilege role as the “execution role”. Example, if my function doesn’t interact with dynamodb, no need to give the function with a role that has access to dynamodb.
  3. Since Cognito Federation pool can only assign either “authenticated role” or “unauthenticated role”. I will use the user Cognito ID as a mean of authorization control in my function logic.

Totally off topic, but is there an “AWS-way” of doing authorization in the function? The convention method I can think of is:

  1. In the function, use the Cognito ID to get the User ID from the Cognito User Pool. (from my understanding, Cognito ID changes)
  2. Maintain a dynamodb table for user authorization rules. Cross check the user id against this table to see if the user is authorized for the further function’s logic

Thanks.


#20

It is possible to run all of your federated logins (google/facebook) in via Cognito User pools. If you do that you can get support for groups, admins, etc. But… it costs $$ to do that for large numbers of logins. For example a million users in Cognito User pools is about $50,000/yr or 100,000 is $3,300/yr. If you just stick with the federated pools it is free. If you federate via User Pools then you can assign as many different types of credentials as you want.

If you are going the way of custom authorizers you probably want to look at API gateway where they are much easier to use.

We use both techniques, normal users come in via federation pool and all receive the same credentials. Admins come in via user pool and get different credentials. We manually create the user pool accounts. The downside of this is that there is no simple way to promote a normal user to an admin since moving them between the pools changes their cognito ID.

BTW, if you are using unauthenticated cognito IDs be aware that you will generate many millions of them in your federated pool.