"LogGroup - <resourceName> already exists" after deploy command

after running sls deploy -v && sls s3deploy as I normally do, I ran into this issue:

  Serverless Error ---------------------------------------

  An error occurred: StartTextractStateMachineLogGroup - /aws/lambda/textract-service-dev-startTextractStateMachine already exists.

  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
     Issues:        forum.serverless.com

  Your Environment Information ---------------------------
     OS:                     linux
     Node Version:           12.2.0
     Serverless Version:     1.42.3

I don’t remember changing anything at all before deployment, other than adding Retry sections to my Step Function States, which shouldn’t have messed with CF logs at all.

I tried running sls remove and then my deployment command again, no luck there.

Also tried adding cfLogs: true to my provider section in the YAML, no love!

I tried manually deleting the LogGroup in CloudFormation, but it’s not even there in the AWS CloudFormation Console.

Here is my serverless.yml:

service: textract-service

provider:
  name: aws
  runtime: python3.7
  timeout: 10
  region: us-east-1
  cfLogs: true
  environment:
    STATE_MACHINE_ARN: ${self:resources.Outputs.TextractStepFunctions.Value}
  iamRoleStatements:
    - Effect: "Allow"
      Action:
        - "s3:PutObject"
        - "states:*"
      Resource:
        Fn::Join:
          - ""
          - - "the-chumiest-bucket-ARN or *"
            - ${self:resources.Outputs.TextractStepFunctions.Value}

plugins:
  - serverless-plugin-existing-s3
  - serverless-step-functions
  - serverless-pseudo-parameters
  - serverless-plugin-existing-s3 
layers:
  boto3Layer:
    package:
      artifact: boto3_layer.zip
    allowedAccounts:
      - "*"

functions:
  startTextractStateMachine:
    handler: src/start_textract_state_machine.lambda_handler
    role: the-chumiest-bucket-role
    layers:
      - {Ref: Boto3LayerLambdaLayer}
    events:
      - existingS3:
          bucket: the-chumiest-bucket
          events:
            - s3:ObjectCreated:*
          rules:
            - prefix: input1/
            - suffix: .pdf
  callTextract:
    handler: src/call_textract.lambda_handler
    role: the-chumiest-bucket-role

    layers:
      - {Ref: Boto3LayerLambdaLayer}
  getTextractOutput:
    handler: src/get_textract_output.lambda_handler
    role: the-chumiest-bucket-role

    layers:
      - {Ref: Boto3LayerLambdaLayer}
  parseTextractOutput:
    handler: src/parse_textract_output.lambda_handler
    role: the-chumiest-bucket-role

    layers:
      - {Ref: Boto3LayerLambdaLayer}

stepFunctions:
  stateMachines:
    textractStepFunc:
      name: TextractStepFunctions
      definition:
        Comment: A state machine for the Textract OCR process.
        StartAt: StartTextractStateMachine
        States:
          StartTextractStateMachine:
            Type: Task
            Resource: "arn:aws:lambda:#{AWS::Region}:#{AWS::AccountId}:function:#{AWS::StackName}-startTextractStateMachine"
            Next: CallTextract
            Retry:
              - ErrorEquals:
                - HandledError
                IntervalSeconds: 1
                MaxAttempts: 1
                BackoffRate: 2
              - ErrorEquals:
                - States.ALL
                IntervalSeconds: 1
                MaxAttempts: 1
                BackoffRate: 2
          CallTextract:
            Type: Task
            Resource: "arn:aws:lambda:#{AWS::Region}:#{AWS::AccountId}:function:#{AWS::StackName}-callTextract"
            Next: GetTextractOutput
            Retry:
              - ErrorEquals:
                - HandledError
                IntervalSeconds: 1
                MaxAttempts: 1
                BackoffRate: 2
              - ErrorEquals:
                - States.ALL
                IntervalSeconds: 1
                MaxAttempts: 1
                BackoffRate: 2
          GetTextractOutput:
            Type: Task
            Resource: "arn:aws:lambda:#{AWS::Region}:#{AWS::AccountId}:function:#{AWS::StackName}-getTextractOutput"
            Next: ParseTextractOutput
            Retry:
              - ErrorEquals:
                - HandledError
                IntervalSeconds: 1
                MaxAttempts: 1
                BackoffRate: 2
              - ErrorEquals:
                - States.ALL
                IntervalSeconds: 1
                MaxAttempts: 1
                BackoffRate: 2
          ParseTextractOutput:
            Type: Task
            Resource: "arn:aws:lambda:#{AWS::Region}:#{AWS::AccountId}:function:#{AWS::StackName}-parseTextractOutput"
            Retry:
              - ErrorEquals:
                - HandledError
                IntervalSeconds: 1
                MaxAttempts: 1
                BackoffRate: 2
              - ErrorEquals:
                - States.ALL
                IntervalSeconds: 1
                MaxAttempts: 1
                BackoffRate: 2
            End: true

resources:
  Outputs:
    TextractStepFunctions:
      Description: The ARN of the state machine
      Value:
        Ref: TextractStepFunctions

FWIW, I’ve seen sls remove fail to completely pull down a stack, leaving some crumbs behind.
I was noy able to get this teardown working 100%, but try logging into cloudformation and the you can:

  • see logs for hints as to what is blocking the stack deletion, or
  • retry deleting the stack manually

Recently, I run into same problem several times.

Probably this is a #bug of #serverless-framework . It looks like it didn’t remove LogGroup

I removed previous deployment using sls removed and subsequent deployment failed due to

Guys,

In a secured environment you cannot remove log groups. Our deployer permissions allow log group creation but not removal. How it should work is:

  1. If the log group does not exist, create it
  2. If the log group does exist, use it
  3. Have a boolean that specifies whether to remove log groups

Hey shederman,

How can I correct the situation if that’s not the behavior I’m seeing? I’m getting the error “An error occurred: ApiGatewayLogGroup - /aws/api-gateway/{api_name} already exists.”

I manually deleted the associated API Gateway object and eventually deleted all Serverless-related items in my AWS environment but the issue still persists.

Any advice?

Thanks

1 Like