We’ve been building some complex Python code on the Serverless framework w/AWS Lambda. Recently we spent a month resolving a Lambda malfunction with Amazon support. There is a very unexpected behavior. Because the “logger” object is global to the interpreter in Python, log handlers can hang around and multiply for multiple executions of a Lambda because Amazon reuses the same environment and leaves all global variables in place – even if you try to NULL out the object you’ll find that it stays. We stored some log metadata in our handler and found that it persisted from one execution to the next, so that we had data from triggers (message UUIDs) hanging around, and also a single log message would show up multiple times.
Here is what solved it:
import logging from logstash_formatter import LogstashFormatterV1 logger = logging.getLogger(function_name) logger.handlers =  # <== SOLUTION HERE handler = logging.StreamHandler(sys.stdout) log_fmt = "" # omitted for simplicity formatter = LogstashFormatterV1(log_fmt) handler.setFormatter(formatter) logger.addHandler(handler)
This is going to be common for anyone doing log shipping to ElasticSearch with CloudWatch Logs, etc.
I’d like to see this make it into documentation so others don’t suffer as we did. I’m not sure where to direct a pull request or what would be appropriate.