Zip file in AWS Layer stays unzipped

I’m using Serverless Framework to deploy my AWS resources and I’m creating a layer from the following commands:

mkdir -p layers/numpan
pip3 install numpy pandas --target python/lib/python3.8/site-packages/
zip -r numpan.zip python/; mv numpan.zip layers/numpan/; rm -rf python/

These commands are run from CodePipeline using a custom buildspec.yml. The serverless.yml file declares the layer as follows:

layers:
  numpan:
    path: layers/numpan
    name: python-numpypandas
    description: "Python 3.8 packages numpy and pandas"
    compatibleRuntimes:
      - python3.8

The lambda function I deploy (see further down after horizontal line for the whole code) has a import pandas line at the beginning, which results in

{
  "errorMessage": "Unable to import module 'lambda_function': No module named 'pandas'",
  "errorType": "Runtime.ImportModuleError",
  "stackTrace": []
}

Commenting out that import, using a print(glob.glob("/opt/*")), and redeploying the lambda, I get

START RequestId: cab5e643-9003-4691-aa13-146de2b045ac Version: $LATEST
['/opt/numpan.zip']
END RequestId: cab5e643-9003-4691-aa13-146de2b045ac

The numpan.zip created at the steps above did not get unzipped… I can see however that the structure within that zip is valid, as the following suggests:

with zipfile.ZipFile("/opt/numpan.zip", 'r') as zip_ref:
     print(zip_ref.namelist())

results in (here an excerpt only):

'python/lib/python3.8/site-packages/pandas/tests/tseries/holiday/test_calendar.py', 'python/lib/python3.8/site-packages/pandas/tests/tseries/holiday/__init__.py', ...

If I run the commands above,

mkdir -p layers/numpan
pip3 install numpy pandas --target python/lib/python3.8/site-packages/
zip -r numpan.zip python/; mv numpan.zip layers/numpan/; rm -rf python/

in, e.g., a Cloud9 instance, push the numpan.zip into a manually created layer, link that new layer to the same lambda function as above, I do get

START RequestId: cab5e643-9003-4691-aa13-146de2b045ac Version: $LATEST
['/opt/python/lib/python3.8/site-packages']
END RequestId: cab5e643-9003-4691-aa13-146de2b045ac

so numpan.zip is unzipped here and the lambda can import pandas. Any idea what’s happening?


Lambda function:

import json
import glob
#import pandas 
import zipfile

def lambda_handler(event, context):

    # print(glob.glob('/opt/*'))
    #print(glob.glob('/opt/python/lib/python3.8/site-packages/*'))
    with zipfile.ZipFile("/opt/numpan.zip", 'r') as zip_ref:
         print(zip_ref.namelist())

It is not fetching proper folder that’s why it is showing this error. Just make a folder and inside that folder you can put pandas zip folder. After that you can add into layer.

Tried a few different locations from which to zip and to provide to the Serverless file. What worked and solved my issue in the end was to replace in the above the path entry with a package entry:

layers:
  numpan:
    name: python-numpypandas
    description: "Python 3.8 packages numpy and pandas"
    compatibleRuntimes:
      - python3.8
    package:
      artifact: numpan.zip

where the zip file is obtained from the following commands:

pip3 install numpy pandas --target python/lib/python3.8/site-packages/
zip -r numpan.zip python/