Why I can't get a list of files in bucket folder?

Serverless Framework
,
1

My lambda can download or upload files/folders to a bucket, no problems here.

Then I just wanted to add another step, to check if files are there already and avoid uploading again.

def list_s3_files(folder: str) -> List[str]:
    """List files in specific S3 URL"""
    files_in_s3 = [f.key for f in s3bucket.objects.filter(Prefix=folder).all()]
    return files_in_s3

But when I run this step in my lambda, I got this error:

sls invoke -s test -f pe-convert -p input.json -l
{
    "errorMessage": "An error occurred (AccessDenied) when calling the ListObjects operation: Access Denied",
    "errorType": "ClientError",
    "stackTrace": [
        "  File \"/var/task/pe_sls.py\", line 113, in pe_convert\n    res = list_s3_files(output_s3_dir)\n",
        "  File \"/var/task/pe_sls.py\", line 26, in list_s3_files\n    files_in_s3 = [f.key for f in s3bucket.objects.filter(Prefix=folder).all()]\n",
        "  File \"/var/task/pe_sls.py\", line 26, in <listcomp>\n    files_in_s3 = [f.key for f in s3bucket.objects.filter(Prefix=folder).all()]\n",
        "  File \"/var/runtime/boto3/resources/collection.py\", line 83, in __iter__\n    for page in self.pages():\n",
        "  File \"/var/runtime/boto3/resources/collection.py\", line 166, in pages\n    for page in pages:\n",
        "  File \"/var/runtime/botocore/paginate.py\", line 255, in __iter__\n    response = self._make_request(current_kwargs)\n",
        "  File \"/var/runtime/botocore/paginate.py\", line 332, in _make_request\n    return self._method(**current_kwargs)\n",
        "  File \"/var/runtime/botocore/client.py\", line 386, in _api_call\n    return self._make_api_call(operation_name, kwargs)\n",
        "  File \"/var/runtime/botocore/client.py\", line 705, in _make_api_call\n    raise error_class(parsed_response, operation_name)\n"
    ]
}

The details from my serverless.yml:

provider:
  name: aws
  ecr:
    images:
      pe-img-docker:
        path: ./
  region: eu-west-2
  lambdaHashingVersion: 20201221
  iamRoleStatements:
    - Effect: Allow
      Action:
        - s3:ListBucket
      Resource: ${self:custom.s3_bucket.${sls:stage}.full}
    - Effect: Allow
      Action:
        - s3:GetObject
        - s3:PutObject
      Resource: ${self:custom.s3_bucket.${sls:stage}.full}/*

And not even setting - s3:* everywhere has done any effect. I also waited a while (I read somewhere policies take minutes to be deployed to the buckets) for no avail.

2

The problem is not here, it was in my bucket definition. My s3_bucket.full var had ‘/*’ already appending.