Directly proxying Lambda via Cloudfront without API Gateway

Serverless Framework
,
1

I’m trying to set up an origin in Cloudfront that directly points to lambda.us-east-1.amazonaws.com without going through API Gateway. This is because I can’t see that API Gateway is doing anything for me other than increasing my bills.

I’m stuck getting this error {
“message”: “The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.”
}

This is something to do with my cloudfront proxy. I can build a URL and call the function without Cloudfront (via lambda.us-east-1.amazonaws.com) and it works.

Is cloudfront doing something to my ‘host’ header or some other header that is needed to verify the signature?

1 Like
2

The secret is to set “Cache Based on Selected Request Headers” to ALL in the behavior.

In Yaml it looks like this:

    ForwardedValues:
      QueryString: 'false'
      Headers:
        - '*'

Don’t worry that this turns off caching for this lambda origin. It is unlikely that you would want IAM secured results to be cached. You can easily turn caching on for other origins such as S3.

3

I also wondered what was the use of API Gateway. One thing that comes to my mind is the usage of custom domains. If you want your content to be deliver through a custom domain of yours, you’ll need it.

1 Like
4

Cloudfront fully supports custom domains. In fact, that is how APIGateway gets it custom domain - via a hidden Cloudfront distribution. I have this working now with https, custom domain, and cloudfront. Note that S3 caching is disabled in this YAML so that I can test my code. Turn it on for production.

Specifying the CloudFront Distribution to serve your Web Application

WebAppCloudFrontDistribution:
  Type: AWS::CloudFront::Distribution
  Properties:
    DistributionConfig:
      PriceClass: PriceClass_100
      Origins:
        - DomainName: $<self:provider.environment.WEBBUCKET>.s3.amazonaws.com
          ## An identifier for the origin which must be unique within the distribution
          Id: WebApp
          ## In case you want to restrict the bucket access use S3OriginConfig and remove CustomOriginConfig
          S3OriginConfig:
             OriginAccessIdentity:
              Ref: AWS::NoValue
        - DomainName: lambda.us-east-1.amazonaws.com
          Id: api
          CustomOriginConfig:
            HTTPPort: '80'
            HTTPSPort: '443'
            OriginProtocolPolicy: https-only
      Enabled: 'true'
      ## Uncomment the following section in case you are using a custom domain
      Aliases:
        - $<self:provider.environment.WEBBUCKET>
      DefaultRootObject: index.html
      ## Since the Single Page App is taking care of the routing we need to make sure ever path is served with index.html
      ## The only exception are files that actually exist e.h. app.js, reset.css
      CustomErrorResponses:
        - ErrorCode: 404
          ResponseCode: 200
          ResponsePagePath: /index.html
      DefaultCacheBehavior:
        DefaultTTL: 0
        MaxTTL: 0
        MinTTL: 0
        Compress: true
        AllowedMethods:
          - GET
          - HEAD
          - OPTIONS
        ## The origin id defined above
        TargetOriginId: WebApp
        ## Defining if and how the QueryString and Cookies are forwarded to the origin which in this case is S3
        ForwardedValues:
          QueryString: 'false'
          Cookies:
            Forward: none
        ## The protocol that users can use to access the files in the origin. To allow HTTP use `allow-all`
        ViewerProtocolPolicy: redirect-to-https
      CacheBehaviors:
      - AllowedMethods:
        - HEAD
        - DELETE
        - POST
        - GET
        - OPTIONS
        - PUT
        - PATCH
        DefaultTTL: 0
        MaxTTL: 0
        MinTTL: 0
        TargetOriginId: api
        ForwardedValues:
          QueryString: 'false'
          Headers:
            - '*'
        ViewerProtocolPolicy: redirect-to-https
        PathPattern: /2015-03-31/*
        Compress: true
      ## The certificate to use when viewers use HTTPS to request objects.
      ViewerCertificate:
        AcmCertificateArn: arn:aws:acm:....
        SslSupportMethod: sni-only
      ## Uncomment the following section in case you want to enable logging for CloudFront requests
      #Logging:
      #  IncludeCookies: 'false'
      #  Bucket: logs.digispeaker.com.s3.amazonaws.com
      #  Prefix: digi
1 Like
5

That’s where you set the custom domain name.

And the certificate for it

6

This is the messy bit that picks off the lambda requests but it doesn’t have to be this way. If you want to pay for lambda@edge you can insert a little function that will rewrite the URLs. A function would let you rewrite something like ‘api/myfunc’ into ‘/2015-03-31/functions/myfunc/invocations’. I suspect APIGateway has a hidden @edge function doing just that. Both schemes work.

Note that there is no free tier for lambda@edge. You will likely run a $10/mth bill when using @edge to rewrite the URLs in a modestly active API.

7

Very interesting. I haven’t looked much at AWS configuration outside of what SLS shows in the documentation.

I assume that they had a good reason to use API Gateway instead of CloudFront directly, maybe for the sake of simplicity?

Regarding your caching, you could set something like this to disable it in development and enable it in the other environments, I do the same kind of stuff to define the RAM to be used by each lambda:

custom:
  webpackIncludeModules: true
  domains:
    development: 'development.simulator.studylink.fr'
    staging: 'staging.simulator.studylink.fr'
    production: 'simulator.studylink.fr'
  memorySizes: # TODO Check how much is actually needed
    development: 128
    staging: 128
    production: 128
  customDomain:
    domainName: ${self:custom.domains.${self:provider.stage}}
    basePath: 'feedback' # This will be prefixed to all routes
    stage: ${self:provider.stage}
    createRoute53Record: true

provider:
  name: aws
  runtime: nodejs6.10
  versionFunctions: false # See https://serverless.com/framework/docs/providers/aws/guide/functions#versioning-deployed-functions
  stage: ${opt:stage, 'development'}
  region: ${opt:region, 'eu-west-3'}
  memorySize: ${self:custom.memorySizes.${self:provider.stage}}

I define values for each stage, and the right value is picked up when I run the script.

8

API Gateway is simpler to use. API Gateway is also good for when you are selling API access. It is also not at all obvious how to directly proxy lambda with Cloudfront, it took me several months to discover how to do it. Plus if you can’t figure out this bit ‘PathPattern: /2015-03-31/*’ you will never get it to work.

With API Gateway in the way, my API calls turn around in about 150ms and they are pretty consistent. With direct Cloudfront to Lambda I run 70-130ms with far more variability. Occasionally a simple call will finish in 40ms. Note that lambda is still protected by https and AWS4 signatures.

I suspect over time API Gateway will stop being an actual machine somewhere and instead simply turn into a bunch of lambda@edge functions. As far as I can tell there isn’t anything in API Gateway that couldn’t be done via lambda@edge. This is probably why they hide their internal Cloudfront distribution from the developers.

9

I’m trying to achieve Cloudfront + Lambda without Api Gateway as well.
I’ve researched a lot and it seems currently you are the only one that managed it and spoke about it @jonsmirl. THANK YOU!

I’ve got few questions:

When you make an HTTP POST to xxx.cloudfront.com/2015-03-31/functions/myfunc/invocations is the call routed to HTTP POST lambda.us-east-1.amazonaws.com/2015-03-31/functions/myfunc/invocations ?

Reading your other posts on this forum I understood that you are using AWS Amplify + Cognito and when you are calling xxx.cloudfront.com/2015-03-31/functions/myfunc/invocations AWS Amplify adds a bunch of Authorization headers with the Cognito user data.

I’ve tried curl -X POST https://xxx.cloudfront.net/2015-03-31/functions/myFunc/invocations but without Authorization Header I’m getting {“message”:“Missing Authentication Token”}

Would it be possible to NOT USE Cognito and specify an Amazon IAM key and secret for the Cloudfront Distribution?

How could you handle HTTP methods other than POST? Using a Lambda@Edge to rewrite the request as a HTTP POST and passing the original http method as a parameter?

10

You need to sign the requests using AWS Signature 4.
https://www.npmjs.com/package/aws-signature-v4

11

Could you share some snippet code for the lambda@edge rewrite @jonsmirl?

I managed to add the authorization header using aws4 but the Lambda Custom Origin is returning the error:

504 ERROR
The request could not be satisfied.
CloudFront attempted to establish a connection with the origin, but either the attempt failed or the origin closed the connection. 
For more information on how to troubleshoot this error, please refer to the CloudFront documentation (https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/http-504-gateway-timeout.html). 


//lamba@edge requestRewriter.js
const aws4 = require('aws4');

exports.handler = function (event, context, callback) {

	const request = event.Records[0].cf.request;

	let opts = aws4.sign({
		service: 'lambda',
		method: 'POST',
		path: '/2015-03-31/functions/myFunc/invocations',
	}, {
		accessKeyId: 'MY_ACCESS_KEY',
		secretAccessKey: 'MY_SECRET_KEY',
	});

	request.method = opts.method;
	request.uri = opts.path;
	request.headers = Object.keys(opts.headers).reduce((headers, key) => {

		if (key !== 'Host') {
			headers[key.toLowerCase()] = [{
				key,
				value: opts.headers[key]
			}];
		}

		return headers;

	}, {
		host: request.headers.host
	});

	callback(null, request);

};
12

I don’t use lambda at edge. I sign the requests in the browser and send them directly to lambda.us-east-1.amazonaws.com. No cloudfront at all. In my measurement cloudfront made lambda calls slower. Cloudfront is great for caching your static stuff.

AWS Amply can generate the sigv4 for you.

Amplify.configure({
Auth: {
identityPoolId: config.IdentityPoolId,
region: ‘us-east-1’,
userPoolId: config.UserPoolId,
userPoolWebClientId: config.UserPoolClientId,
},
API: {
endpoints: [
{
name: ‘api’,
endpoint: ‘https://lambda.us-east-1.amazonaws.com’,
service: ‘lambda’,
},
],
},
});

13

Is it actually possible to use CloudFront with a Lambda as Custom Origin at all?

The problem is that the Host header forwarded from CloudFront to the Lambda cannot be changed. So even if you use a lambda@edge to add authorizations headers signed with aws4 signature the Host header will be stuck at ‘xxx.cloudfront.com’ or your custom domain name and will not match the Host used in the aws4 signature.

With Amplify or a HTTP POST you can call directly the HTTP POST lambda.us-east-1.amazonaws.com/2015-03-31/functions/myfunc/invocations but you cannot call a CloudFront distribution in front of the lambda for the same problem with the Host header I described above.

14

You need to make a distribution in cloudfront that points to lambda.us-east-1.amazonaws.com

Maybe this is more what you are looking for…

Another article on a similar technique

15

Yes I did use a CloudFront Distribution and the serverless-plugin-cloudfront-lambda-edge.
The problem is the authorization headers part though. You need to sign the request with aws4 but CloudFront won’t allow you to replace the Host header and so the aws4 signature will not match and return 

{“message”:“The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.”}

I also tried generasting the aws4 signature with the cloudfront Host header but then I would get: 
AccessDeniedException>
  Unable to determine service/operation name to be authorized

// serverless.yml
{
    "service": "service",
    "provider": {
        "name": "aws",
        "runtime": "nodejs8.10",
        "stage": "${opt:stage, 'development'}",
        "region": "us-east-1",
        "memorySize": "512",
        "timeout": "10"
    },
    "functions": {

        "index": {
            "handler": "handler.index",
            "memorySize": 256
        },
        "requestRewriter": {
            "handler": "requestRewriter.handler",
            "memorySize": 128,
            "timeout": 1,
            "lambdaAtEdge": {
                "distribution": "CloudfrontDistributionProduction",
                "eventType": "viewer-request"
            }
        }

    },
    "resources": {
        "Resources": {
            "CloudfrontDistributionProduction": {
                "Type": "AWS::CloudFront::Distribution",
                "Properties": {
                    "DistributionConfig": {
                        "Origins": [{
                            "DomainName": "lambda.us-east-1.amazonaws.com",
                            "Id": "api",
                            "CustomOriginConfig": {
                                "HTTPPort": "80",
                                "HTTPSPort": "443",
                                "OriginProtocolPolicy": "match-viewer"
                            }
                        }],
                        "Enabled": "true",
                        "Aliases": ["*.mydomain.com"],
                        "DefaultCacheBehavior": {
                            "AllowedMethods": ["HEAD", "DELETE", "POST", "GET", "OPTIONS", "PUT", "PATCH"],
                            "DefaultTTL": 0,
                            "MaxTTL": 0,
                            "MinTTL": 0,
                            "TargetOriginId": "api",
                            "ForwardedValues": {
                                "QueryString": false,
                                "Headers": ["*"],
                            },
                            "ViewerProtocolPolicy": "allow-all",
                            "Compress": true
                        },
                        "ViewerCertificate": {
                        	"AcmCertificateArn": "arn:aws:acm:us-east-xxx",
                        	"SslSupportMethod": "sni-only"
                        }
                    }
                }
            }
        }
    },
    "plugins": [
        "serverless-plugin-cloudfront-lambda-edge"
    ],
    "custom": {
        "webpackIncludeModules": {
            "forceExclude": ["aws-sdk"]
        }
    }

}
16

There is definitely a way to do this, I remember getting stuck at this exact issue about a year ago when I tired this. This is some trick with setting up the CF distribution that allows this to work. But, after I got it working I concluded that all Cloudfront was doing was making my API call take an extra 60ms to complete. Since I abandoned this method I’ve forgotten how I did it.

Maybe this was it?
https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/forward-custom-headers.html

17

If you read below there is a section that states:

You can't configure CloudFront to forward the following custom headers to your origin:
Host

https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/forward-custom-headers.html#forward-custom-headers-blacklist

And Lambda@Edge can only Read but not write the Host header:

Read-only Headers for CloudFront Viewer Request Events
Host

https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/lambda-requirements-limits.html#lambda-read-only-headers

So maybe you did place the CloudFront Distribution in front of a Lambda using the Custom Origin but then you still called the Lambda directly using AWS Amplify or a HTTP POST @jonsmirl

18

I did not need to use Edge at all. There is a way to configure the distribution behavior to do this.

19

I think you might have setup the CloudFront Distribution in front of the Lambda using the Custom Origin but then you might have called directly the Lambda anyway not finding out that the CloudFront Distribution wouldn’t allow accept the aws4 signing.

Could you recreate a basic github repository project demonstrating how to place a CloudFront Distribution in front of a Lambda and having them called not from Amplify/Cognito but by making an HTTP POST request to the CloudFront url xxx.cloudfront.com?

20

I looked in my git log, I think this was the configuration I used. But note that I concluded that doing this gained nothing but slowing down your lambda access.

## Specifying the CloudFront Distribution to server your Web Application
WebAppCloudFrontDistribution:
  Type: AWS::CloudFront::Distribution
  Properties:
    DistributionConfig:
      PriceClass: PriceClass_100
      Origins:
        - DomainName: $<self:provider.environment.WEBBUCKET>.s3.amazonaws.com
          ## An identifier for the origin which must be unique within the distribution
          Id: WebApp
          ## In case you want to restrict the bucket access use S3OriginConfig and remove CustomOriginConfig
          S3OriginConfig:
             OriginAccessIdentity:
              Ref: AWS::NoValue
        - DomainName: lambda.us-east-1.amazonaws.com
          Id: api
          CustomOriginConfig:
            HTTPPort: '80'
            HTTPSPort: '443'
            OriginProtocolPolicy: https-only
      Enabled: 'true'
      ## Uncomment the following section in case you are using a custom domain
      Aliases:
        - $<self:provider.environment.WEBBUCKET>
      DefaultRootObject: index.html
      ## Since the Single Page App is taking care of the routing we need to make sure ever path is served with index.html
      ## The only exception are files that actually exist e.h. app.js, reset.css
      CustomErrorResponses:
        - ErrorCode: 404
          ResponseCode: 200
          ResponsePagePath: /index.html
      DefaultCacheBehavior:
        DefaultTTL: 0
        MaxTTL: 0
        MinTTL: 0
        Compress: true
        AllowedMethods:
          - GET
          - HEAD
          - OPTIONS
        ## The origin id defined above
        TargetOriginId: WebApp
        ## Defining if and how the QueryString and Cookies are forwarded to the origin which in this case is S3
        ForwardedValues:
          QueryString: 'false'
          Cookies:
            Forward: none
        ## The protocol that users can use to access the files in the origin. To allow HTTP use `allow-all`
        ViewerProtocolPolicy: redirect-to-https
      CacheBehaviors:
      - AllowedMethods:
        - HEAD
        - DELETE
        - POST
        - GET
        - OPTIONS
        - PUT
        - PATCH
        DefaultTTL: 0
        MaxTTL: 0
        MinTTL: 0
        TargetOriginId: api
        ForwardedValues:
          QueryString: 'false'
          Headers:
            - '*'
        ViewerProtocolPolicy: redirect-to-https
        PathPattern: /2015-03-31/*
        Compress: true
      ## The certificate to use when viewers use HTTPS to request objects.
      ViewerCertificate:
        AcmCertificateArn: arn:aws:acm:us-east-1:298225384574:certificate/f55c98f2-13e7-482c-b9e3-05549617c345
        SslSupportMethod: sni-only
      ## Uncomment the following section in case you want to enable logging for CloudFront requests
      #Logging:
      #  IncludeCookies: 'false'
      #  Bucket: logs.digispeaker.com.s3.amazonaws.com
      #  Prefix: digi