Using python modules inside common functions

Serverless Framework
1

I have a simple serverless function that does a data cleanup, my current folder structure is

project_path
 |_ lib
      |_ common_function.py
|_serverless.yml
|_ handler.py

serverless.yml

frameworkVersion: '2'

provider:
  name: aws
  runtime: python3.8
  region: ap-southeast-2
  
functions:
  populateData:
    handler: handler.populate
    timeout: 900
    package:
      include:
        - lib/**

plugins:
  - serverless-python-requirements
  - serverless-offline

custom:
  pythonRequirements:
    dockerizePip: true
    zip: true
    #Get rid of unnecessary package files
    slim: true
    #But keep necessary binary files to avoid "ELF load command address/offset not properly aligned"
    strip: false

package:
  individually: true

common_function.py imports pandas and sqlalchemy

import pandas as pd
from sqlalchemy import create_engine

In my handler.py I am importing the common function as

from lib import common_function as cmf

this passes deployments but when I invoke the function I get the following error

{
    "errorMessage": "Unable to import module 'handler': No module named 'pandas'",
    "errorType": "Runtime.ImportModuleError",
    "stackTrace": []
}

not sure what is causing the error. I tried using layers but then the deployment fails because the size is too high

2

In most cases this error in Python generally raised:

  • You haven’t installed Pandas explicitly with pip install pandas.
  • You may have different Python versions on your computer and Pandas is not installed for the particular version you’re using.

You can run the following command in your Linux/MacOS/Windows terminal.

pip install pandas

To be sure you are not having multiple Python versions that are confusing, you should run following commands:

python3 -m pip install pandas
python3 -c 'import pandas'