@dfernper I think this should be working by default: https://forums.aws.amazon.com/thread.jspa?threadID=216000
Basically if the Mysql Node driver does proper connection pooling, stores the pool in the same process it should be fine. Have you tried accessing the connection pool and logging it on every request to make sure if it already has been running?
One issue of course is that if you burst functions like crazy you might run out of connections, but other than that it should be fine.