This is only an issue with serverless-offline and is caused by the way it does cache invalidation.
If you want a quick fix you can do sls offline start --skipCacheInvalidation which will start (and kill) a node process for every request. This slows things down but fixes the issue of database connections staying open. See this PR for details: https://github.com/dherault/serverless-offline/pull/368