It is possible for Lambda functions to only be a couple of kb but you need to restrict yourself to just code you write plus the AWS-SDK (as a dev dependency). Once you start to use third party NPM’s then you get all of the baggage that comes with them.
Serverless helps you with some of this. It will automatically exclude dev dependencies from the upload package. You can further control what is being uploaded using the exclude/include options in your serverless.yml file. This allows you to exclude additional config files, source files, test files, etc.
If that still isn’t enough then consider something like webpack to reduce the size further through tree shaking but understand that while it might decrease the size of each function it may increase the size of the upload package.