Good to see I wasn't the only person having problems with this... It did my head in for awhile.
I was looking at making a serverless image hosting service but was having a hard time getting binary data into my lambdas. I've abandoned the idea, for now, due to APIG (and lambda) payload size limitations but I might come back with signed uploads and then work via S3 events. That's a different story.
I managed to get the binary data going in on a
LAMBDA_PROXY but never tried going out. I suspect the method is similar though. Annoyingly, I couldn't find any serverless config options to set the binary formats and has to do them manually.
Anyway... https://aws.amazon.com/blogs/compute/binary-support-for-api-integrations-with-amazon-api-gateway/ was a lot of help.
callback(null, postProcessResource(outputFile, (file) => new Buffer(fs.readFileSync(file)).toString('base64')));
postProcessResource() just unlinks the file.
Granted that is with a non-proxy example but if you have a look at index.js#L7-L15 and api.json#L71-L73 from bustlelabs/gziptest it is definitely possible.
Basically... base64. Binary in will give you base64, base64 out will give you binary.
This was a bit of a brain dump but I hope it helps.