Has anyone worked out the magic incantation to have an image returned from Lambda via the API Gateway?
I’ve added contentHandling: CONVERT_TO_BINARY to serverless.yml but doesn’t seem to be passing through.
As far as I know, APIGateway do not support binary transfers. Use S3 upload and setup event in lambda for that S3.
I did not implement it yet, so cant say much more, but working on it.
AWS just announced support for binary data in the API Gateway: https://aws.amazon.com/about-aws/whats-new/2016/11/binary-data-now-supported-by-api-gateway/
Trying to get it working via serverless, but it may be that the CloudFormation deploy needs to be changed to support.
Unfortunately, it appears that the only way to enable binary payload support is either through the Web Console described in the announcement, or through the Swagger-based API Gateway deployments: http://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-content-encodings-examples-image-lambda.html
Maybe Serverless should switch to Swagger definitions instead of listing all the resources in the CloudFormation? It looks like AWS API Gateway is putting the advanced options there.
Swagger doesn’t help when it comes to non API GW resources, so isn’t really a valid option to manage infrastructure.
My guess is that the Swagger definitions are managed by the API GW Team internally, so they’re free to make changes whenever they feel like it. CFN is probably another team internally, hence the lag… CFN will get there.
@rowanu
I don’t get your point. Why is Swagger not an option? The recently introduced serverless application model (SAM) is using that approach, as well.
(See “Example: AWS::Serverless::Api”)
@tobyhede
Leaving aside the fact that you cannot enable binary support through serverless, yet, it seems to me that binary support is not working with Lambda Proxy Integration (lambda-proxy is the default configuration in serverless). It works, however, with “lambda integrated” functions.
I started a thread in the AWS forum about this (bug?!): https://forums.aws.amazon.com/thread.jspa?threadID=243584&tstart=0
Note:
You need to configure a binary mime type (e.g. image/png) in APIG’s binary support section and request the resource with the accept header set to the configured binary mime type (e.g. image/png)
Swagger is fine for provisioning API GW resources, but not other AWS resources (e.g. Buckets, CW Events, IAM Roles, etc), so using Swagger is not an option for the entire service. Before the SAM update it was not possible to use Swagger in CFN, so it was an either/or choice between the two.
Now that CFN (via SAM) can provision Swagger-defined resources, there might be a way to combine the two, but that will definitely require some re-work in Serverless to make happen…
3 Likes
Hi @tobyhede - I stumbled across this forum post and am facing my own challenges getting binary support working (for file uploads) with a Lambda Integration function that I am deploying via serverless (version 1.2.1). Have you had success with getting binary support working for your use case?
1 Like
No, struggled with it for a while but no luck.
1 Like
I was also looking into it and will more over the week. We are also considering options how to extend the framework for adding binary support.
Before we move on I would love to understand your use-cases.
Hey,
Good to see I wasn’t the only person having problems with this… It did my head in for awhile.
I was looking at making a serverless image hosting service but was having a hard time getting binary data into my lambdas. I’ve abandoned the idea, for now, due to APIG (and lambda) payload size limitations but I might come back with signed uploads and then work via S3 events. That’s a different story.
I managed to get the binary data going in on a LAMBDA_PROXY but never tried going out. I suspect the method is similar though. Annoyingly, I couldn’t find any serverless config options to set the binary formats and has to do them manually.
Anyway… https://aws.amazon.com/blogs/compute/binary-support-for-api-integrations-with-amazon-api-gateway/ was a lot of help.
callback(null, postProcessResource(outputFile, (file) => new Buffer(fs.readFileSync(file)).toString('base64')));
postProcessResource() just unlinks the file.
Granted that is with a non-proxy example but if you have a look at index.js#L7-L15 and api.json#L71-L73 from bustlelabs/gziptest it is definitely possible.
Basically… base64. Binary in will give you base64, base64 out will give you binary.
This was a bit of a brain dump but I hope it helps.
@rob
thanks a lot for the link to https://github.com/bustlelabs/gziptest/ So one has to add “isBase64Encoded: true” to the response object in order to make it work with lambda proxy integrated functions.
Still I think one cannot enable binary media types with serverless - so this must be done manually.
My use case was actually uploading a binary file to API Gateway/Lambda (rather than returning binary data). I just cracked the code on making it work using the serverless framework). I’ll put a sample project on github and link to it here by the end of my afternoon that recreates the thumbnail service described here … https://aws.amazon.com/blogs/compute/binary-support-for-api-integrations-with-amazon-api-gateway/ … but uses the serverless framework. Hopefully it will save others some headaches (and remind me what I did to get it working when I forget a month from now and encounter the same problems).
2 Likes
@krisgholson I don’t know what you mean by your afternoon, but I’m refreshing the page every 5 min from now on 
I was working on an image serving example/tutorial for https://github.com/serverless/examples. Definitely will mentioned you and link yo yours as well 
@nikgraf
Stop refreshing your page … here it is! … at 4:37pm EST. That’s my afternoon. Hopefully it helps.
2 Likes
Nice work!
My use-case is quite similar, trying to return an image via lambda. Have it as Base64, but the data is coming out and not being able to be rendered as an image by the browser. Will have another play and see if I can get your magic to work.
1 Like
I spent two days wrestling handling posted files, and finally found the binary option.
Now, like others, I want to be able to enable it in my serverless config.
From what I can see in the API Docs http://docs.aws.amazon.com/apigateway/api-reference/resource/integration/ it’s a matter of passing contentHandling = “CONVERT_TO_BINARY” in the Integration.
Then the body of the event passed to your Lambda is the base64 encoded content of the request body.
Does it look right to anyone that handling this would be added about here -> https://github.com/serverless/serverless/blob/2cb4dc4dc5dab459bb63659bf5e5c2136440368b/lib/plugins/aws/deploy/compile/events/apiGateway/lib/method/integration.js#L30
Thanks @krisgholson this is great. I was hoping for a solution that would not need the user interface. @funkybob yep, exactly I was looking at contentHandling exactly like you did and tried to add it where you mentioned it. Unfortunately the deployment fails then, because that option yet doesn’t exist CloudFormation.
I will try to get a response from AWS if they plan to add this soon, but can’t promise anything.
I might be wrong here but I don’t think that you can configure the contentHandling at all for lambda functions invoked from APIGW or rather that it will have any effect doing so.
After this feature is configured, you can specify if you would like API Gateway to either pass the Integration Request and Response bodies through, convert them to text (Base64 encoding), or convert them to binary (Base64 decoding). These options are available for HTTP, AWS Service, and HTTP Proxy integrations. In the case of Lambda Function and Lambda Function Proxy Integrations, which currently only support JSON, the request body is always converted to JSON.
See Binary Support for API Integrations with Amazon API Gateway | AWS Compute Blog
Only the binary media types need to be defined - which I guess is also not possible using CloudFormation.
You can see this also if you export the swagger file.
If you follow for example GitHub - krisgholson/serverless-thumbnail: Recreate the thumbnail service described here .. https://aws.amazon.com/blogs/compute/binary-support-for-api-integrations-with-amazon-api-gateway/ .. using the serverless.com framework (and document some gotchas). and export the swagger file from APIGW you’ll see “contentHandling” set to “CONVERT_TO_TEXT”.
That’s why I suggested serverless to follow SAM and allow referencing a swagger file.