S3 bucket Notification Configuration and SNS/SQS creation in same serverless configuration file

Serverless Framework
1

Hi there!

I am trying to create a S3 Bucket, SNS Topic, SQS Queue that should be subscribed to the SNS Topic and when their is a new message in SQS Queue it should trigger a lambda function.

Here is my serverless config:

service: eshot

plugins:
  - serverless-step-functions
  - serverless-pseudo-parameters

provider:
  name: aws
  runtime: go1.x
  iamRoleStatements:
    - Effect: Allow
      Action:
        - s3:getObject
        - s3:putObject
        - s3:putBucketPolicy
      Resource: "*"
  stage: ${opt:stage}
  region: ${file(./config.yml):${opt:stage}.REGION}
  environment:
    ESHOT_UPLOADS_BUCKET: ${file(./config.yml):${opt:stage}.ESHOT_UPLOADS_BUCKET}

package:
  individually: true
  exclude:
    - ./**

functions:
  diffAndSave:
    handler: bin/diff-and-save
    memorySize: 128
    timeout: 60
    package:
      individually: true
      include:
        - ./bin/diff-and-save
    layers:
      - {Ref: DiffLambdaLayer}
    events:
      - sqs: "arn:aws:sqs:${file(./config.yml):${opt:stage}.REGION}:#{AWS::AccountId}:s3-txt-upload"


layers:
  diff:
    package:
      artifact: diff.zip
    name: ${opt:stage}-diff
    description: Diff is used to generated unified diff output
    compatibleRuntimes:
      - go1.x
    allowedAccounts:
      - '*'

resources:
  - Resources:
      CFNSQSQueue:
        Type: AWS::SQS::Queue
        Properties:
          MessageRetentionPeriod: 60
          QueueName: "s3-txt-upload"
          VisibilityTimeout: 60
      MySQSQueuePolicy:
        Type: AWS::SQS::QueuePolicy
        Properties:
          PolicyDocument:
            Version: '2008-10-17'
            Statement:
            - Effect: Allow
              Principal:
                AWS: "*"
              Action:
              - SQS:SendMessage
              Resource: "*"
              Condition:
                ArnEquals:
                  aws:SourceArn:
                    Ref: S3SNSTopic
          Queues:
          - Ref: CFNSQSQueue
      S3SNSTopic:
        Type: AWS::SNS::Topic
        Properties:
          Subscription:
          - Endpoint:
              Fn::GetAtt:
              - CFNSQSQueue
              - Arn
            Protocol: sqs
          TopicName: "s3-txt-upload"
      EmapApiTopicPolicy:
        Type: AWS::SNS::TopicPolicy
        Properties:
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Sid: EmapApiTopicPolicy
                Effect: Allow
                Principal:
                  Service: s3.amazonaws.com
                Action: sns:Publish
                Resource: "*"
          Topics:
            - Ref: S3SNSTopic
      CFNBucket:
        Type: AWS::S3::Bucket
        DependsOn: EmapApiTopicPolicy
        Properties:
          AccessControl: Private
          BucketName: "${file(./config.yml):${opt:stage}.EMAP_UPLOADS_BUCKET}"
          NotificationConfiguration:
             TopicConfigurations:
                - Event: s3:ObjectCreated:
                  Topic:
                  Ref: S3SNSTopic

But this returns an error because notification configuration options are set before the resources(SNS, SQS) are created.

If you create the target resource and related permissions in the same template, you might have a circular dependency.
More information here: link

The resolution to this issue in AWS Docs is

To avoid this dependency, you can create all resources without specifying the notification configuration. Then, update the stack with a notification configuration.

Is there some way to keep NotificationConfiguration in serverless configuration but only deploy it after the resources are created? Or maybe only create it on an “update” operation(As in when it’s the configuration is re-deployed after it’s ran for the first time).

This’ll save us the trouble of manually setting up events on S3 buckets.

3 Likes