S3 createObject event doesn't fire lambda function from custom bucket

Serverless Framework
1

I’ve created a custom s3 bucket, lambda function with iamRoleStatements, and a lambda permission according to the documentation:

functions:
  transformOnBucketUpload:
    handler: dist/transform.onBucketUpload
    description: waits for s3 bucket uploads of design files and writes applications to an output folder
    reservedConcurrency: 100
    events:
    - s3:
      bucket:
        Ref: DiyBucket
      event: s3:ObjectCreated:*
      rules:
      - prefix: input/
      - suffix: .sketch
    environment:
      sloppy: true
      outputBucketArn:
        Fn::GetAtt: [DiyBucket, Arn]
    iamRoleStatements:
    - Effect: Allow
      Action:
        - s3:*
      Resource:
      - Fn::GetAtt: [DiyBucket, Arn]
      - Fn::Join: ['', [Fn::GetAtt: [DiyBucket, Arn], '/*']]
          
resources:
   
  Resources:

    # lambda permission for the function to be invoked by the s3 bucket
    ResizeLambdaPermissionPhotosS3:
      Type: AWS::Lambda::Permission
      Properties:
        FunctionName:
          Fn::GetAtt:
            - TransformOnBucketUploadLambdaFunction
            - Arn
        Principal: s3.amazonaws.com
        Action: lambda:InvokeFunction
        SourceAccount:
          Ref: AWS::AccountId
        SourceArn:
          Fn::GetAtt: [DiyBucket, Arn]

    # the s3 bucket
    DiyBucket:
      Type: AWS::S3::Bucket

Yet when I deploy:

  • I see no errors on deployment
  • In AWS console, the lambda function has no triggers/events other than the cloudwatch logs defined
  • In AWS console, there are no logs when I upload a file to the bucket

How can I debug this when I receive absolutely no errors or information?

Here you can see that the trigger is not shown in the AWS console:

25%20PM
25%20PM2722×676 41.8 KB

When I inspect the final cloudformation file being uploaded to AWS, I was shocked to see that there is absolutely no bucket notification configuration or event stream mapping being set. There is nothing that connects the bucket with the lambda function!

Based on this thread it seems like this is a serverless framework bug, and therefore I tried to manually setup the NotificationConfiguration. Unfortunately this results in a cryptic error:

An error occurred: DiyBucket - Unable to validate the following destination configurations (Service: Amazon S3; Status Code: 400; Error Code: InvalidArgument; Request ID: 6516C03FC6506837; S3 Extended Request ID: RD1gHlYOOA+DRL7J8vg1MIf6xnPkDQ28p+lAkZOzgJijzalp/z6i1u1CaXUslLIzrmZ6Y4glYFE=).

Based on Resolve the "Unable to validate the following destination configurations" error with Lambda event notifications in CloudFormation it appears that there is an implicit circular dependency. While I make the lambda permission more lax so the circular dependency is gone, I still need to access the bucket arn in my function’s environment. Meanwhile the bucket depends on the lambda invoke permission, which itself depends on the function: again reintroducing the circular dependency. How to tackle this?

I.e. this makes it deploy:

  functions:
  transformOnBucketUpload:
    handler: dist/transform.onBucketUpload
    description: waits for s3 bucket uploads of design files and writes applications to an output folder
    reservedConcurrency: 100
    # iamRoleStatements:
    #   - Effect: Allow
    #     Action:
    #       - s3:*
    #     Resource:
    #       - Fn::GetAtt: [DiyBucket, Arn]
    #       - Fn::Join: ['', [Fn::GetAtt: [DiyBucket, Arn], '/*']]
    # environment:
    #   sloppy: true
    #   outputBucketArn:
    #     Fn::GetAtt: [DiyBucket, Arn]
          
resources:
   
  Resources:

    # lambda permission for the function to be invoked by the s3 bucket
    ResizeLambdaPermissionPhotosS3:
      Type: AWS::Lambda::Permission
      Properties:
        FunctionName:
          Fn::GetAtt: [TransformOnBucketUploadLambdaFunction, Arn]
        Principal: s3.amazonaws.com
        Action: lambda:InvokeFunction
        SourceAccount:
          Ref: AWS::AccountId
        # Whilst it would make sense to define an explicit source arn here
        # This results in a cryptic error related to circular dependencies.
        # The bucket depends on this permission, and this permission depends
        # on the bucket. See also:
        # https://aws.amazon.com/premiumsupport/knowledge-center/unable-validate-circular-dependency-cloudformation/
        # https://aws.amazon.com/premiumsupport/knowledge-center/unable-validate-destination-s3/
        # SourceArn:
        #   Fn::GetAtt: [DiyBucket, Arn]

    # the s3 bucket to store sketch files and receive applications
    # on creation, s3 forecefully checks if it has permission to setup
    # the specified notification configurations
    DiyBucket:
      DependsOn:
        - ResizeLambdaPermissionPhotosS3
      Type: AWS::S3::Bucket
      Properties:
        NotificationConfiguration:
          LambdaConfigurations:
            - Event: "s3:ObjectCreated:*"
              Function:
                Fn::GetAtt: [TransformOnBucketUploadLambdaFunction, Arn]

But this is a problem because now the function’s iamRoleStatements and its environment vars cannot be set because they depend on the s3 bucket. The s3 bucket depends on the invoke permission, and the invoke permission depends on the function…