[Fixed] Can't add two indexes in GlobalSecondaryIndexes

[bill]

Met this issue today, I added two indexes in part of GlobalSecondaryIndexes when defined a dynamodb resource.

Serverless Error ---------------------------------------
An error occurred: TodosDynamoDbTable - Cannot perform more than one GSI creation or deletion in a single update.

The table has been created successfully with previous sls deploy, I got the error after add below codes

    GlobalSecondaryIndexes:
      - IndexName: category-index
        KeySchema:
          - AttributeName: category
            KeyType: HASH
        Projection:
          ProjectionType: KEYS_ONLY
        ProvisionedThroughput:
          ReadCapacityUnits: 1
          WriteCapacityUnits: 1
      - IndexName: name-index
        KeySchema:
          - AttributeName: name
            KeyType: HASH
        Projection:
          ProjectionType: KEYS_ONLY
        ProvisionedThroughput:
          ReadCapacityUnits: 1
          WriteCapacityUnits: 1

I have to remove the whole stack with sls remove and run sls deploy again , no error reported, all resources are created without issue.

I still got the same error, if I only delete one dynamodb with two indexes.

Anything I can do to avoid delete the stack?

[bill]

Did anyone meet this issue and how to fix it?

I give whole change I did.

original serverless.yml is exact same file of serverless/example/aws-node-rest-api-with-dynamodb/serverless.yml

I run sls deploy successfully.

Then I add the GSI part as below:

service: serverless-rest-api-with-dynamodb

frameworkVersion: ">=1.1.0 <2.0.0"

provider:
  name: aws
-  runtime: nodejs4.3
+  runtime: nodejs6.10
  environment:
    DYNAMODB_TABLE: ${self:service}-${opt:stage, self:provider.stage}
  iamRoleStatements:
    - Effect: Allow
      Action:
        - dynamodb:Query
        - dynamodb:Scan
        - dynamodb:GetItem
        - dynamodb:PutItem
        - dynamodb:UpdateItem
        - dynamodb:DeleteItem
      Resource: "arn:aws:dynamodb:${opt:region, self:provider.region}:*:table/${self:provider.environment.DYNAMODB_TABLE}"

functions:
  create:
    handler: todos/create.create
    events:
      - http:
          path: todos
          method: post
          cors: true

  list:
    handler: todos/list.list
    events:
      - http:
          path: todos
          method: get
          cors: true

  get:
    handler: todos/get.get
    events:
      - http:
          path: todos/{id}
          method: get
          cors: true

  update:
    handler: todos/update.update
    events:
      - http:
          path: todos/{id}
          method: put
          cors: true

  delete:
    handler: todos/delete.delete
    events:
      - http:
          path: todos/{id}
          method: delete
          cors: true

resources:
  Resources:
    TodosDynamoDbTable:
      Type: 'AWS::DynamoDB::Table'
      DeletionPolicy: Retain
      Properties:
        AttributeDefinitions:
          -
            AttributeName: id
            AttributeType: S
+          - AttributeName: category
+            AttributeType: S
+          - AttributeName: name
+            AttributeType: S
        KeySchema:
          -
            AttributeName: id
            KeyType: HASH
+        GlobalSecondaryIndexes:
+          - IndexName: category-index
+            KeySchema:
+              - AttributeName: category
+                KeyType: HASH
+            Projection:
+              ProjectionType: KEYS_ONLY
+            ProvisionedThroughput:
+              ReadCapacityUnits: 1
+              WriteCapacityUnits: 1
+          - IndexName: name-index
+            KeySchema:
+              - AttributeName: name
+                KeyType: HASH
+            Projection:
+              ProjectionType: KEYS_ONLY
+            ProvisionedThroughput:
+              ReadCapacityUnits: 1
+              WriteCapacityUnits: 1
        ProvisionedThroughput:
          ReadCapacityUnits: 1
          WriteCapacityUnits: 1
        TableName: ${self:provider.environment.DYNAMODB_TABLE}

move to button on above file, you will see the changes I did.

After I added GlobalSecondaryIndexes (GSI) codes and deploy again, I got the error Cannot perform more than one GSI creation or deletion in a single update.

Remove the whole stack and deploy from beginning is not idea solution, because I don’t expect this behavior in production environment, if I need to change something later.

[alexdebrie1]

@bill Can you do it in two steps – add the first GSI, then add the second GSI once it’s finished?

[bill]

Thanks @alexdebrie1

I tested as you suggested.

The correct steps are:

1. add one GSI, sls deploy
2. waiting for the first GSI created and ready (active status), it takes several minutes.
3. add second GSI, sls deploy

[tommedema]

@bill

The correct steps are:

add one GSI, sls deploy
waiting for the first GSI created and ready (active status), it takes several minutes.
add second GSI, sls deploy

This does not seem correct. Assuming you want your infrastructure to be fully version controlled, you wouldn’t want to have to “remember” strange deployment procedures like described.

I guess the question to ask is why the second global secondary index has to be registered after the first global secondary index is deployed. Is this because of cloudformation or because of a serverless bug? If it is because of cloudformation, perhaps it is possible to define a DependsOn as a property of the second index? I’m not sure if that would work, but if it does, your deployment procedure would be version controlled again.

[alexdebrie1]

@tommedema That’s a CloudFormation issue. Not sure if DependsOn would work but worth trying.

[harleyguru]

hey @alexdebrie1 this issue has been fixed?
I have the same issues and I have more tables with one or 2 indexes.
Its so painful to deploy my serverless project.
Looking forward your help.

[harleyguru]

Hey @tommedema, @bill, @alexdebrie1 I think this issue should be fixed by AWS team for sure.
By the way, even if CloudFormation doesn’t allow creation or update more than one index or 10 over tables, we can make this working by queueing those requests internally in Serverless framework.

Or we should wait until CloudFormation issue would be fixed?
Why should we keep this strange deployment flow?

Only one way is to serialize the deployment of resources by using DependsOn property, Thanks @tommedema
It works!!!

[xiangzprime8]

Hi @harleyguru, I wonder how you used DependsOn to add 2 GSIs in one deployment. I tried the following yaml but met error saying the table “already exists in stack …”:

MyTableResource01:
Type: AWS::DynamoDB::Table
Properties:
TableName: MyTable
[properties … with first GSI]

MyTableResource02:
Type: AWS::DynamoDB::Table
DependsOn:
- MyTableResource01
Properties:
TableName: MyTable
[properties … with second GSI]