Serverless deploy works in local, but not within GitLab CICD. The erorr message seems empty

I’m getting this output while deploying the serverless stack through GitLab CICD

ServerlessError: Invalid variable reference syntax for variable . You can only reference env vars, options, & files. You can check our docs for more info. at Variables.getValueFromSource (/usr/local/lib/node_modules/serverless/lib/classes/Variables.js:589:30) at Variables.splitAndGet (/usr/local/lib/node_modules/serverless/lib/classes/Variables.js:448:17) at /usr/local/lib/node_modules/serverless/lib/classes/Variables.js:387:40 at Array.map (<anonymous>) at Variables.populateMatches (/usr/local/lib/node_modules/serverless/lib/classes/Variables.js:387:20) at Variables.populateValue (/usr/local/lib/node_modules/serverless/lib/classes/Variables.js:418:30) at /usr/local/lib/node_modules/serverless/lib/classes/Variables.js:300:38 at Array.map (<anonymous>) at Variables.populateVariables (/usr/local/lib/node_modules/serverless/lib/classes/Variables.js:299:8) at Variables.populateObjectImpl (/usr/local/lib/node_modules/serverless/lib/classes/Variables.js:333:30) at /usr/local/lib/node_modules/serverless/lib/classes/Variables.js:329:40 at Variables.initialCall (/usr/local/lib/node_modules/serverless/lib/classes/Variables.js:112:20) at Variables.populateObject (/usr/local/lib/node_modules/serverless/lib/classes/Variables.js:329:17) at /builds/koshyviv/coda-learn/node_modules/serverless-step-functions/lib/yamlParser.js:26:62

It works completely fine in my local, and the error seems to be incomplete - thus, I’m not able to figure which variable is exactly at fault.

My serverless.yml is present here - serverless.yml · dev · Vivek George Koshy / coda-learn · GitLab

service: coda-learn-vivek
frameworkVersion: ‘2’

provider:
name: aws
runtime: nodejs12.x
lambdaHashingVersion: 20201221
region: ap-south-1
stage: ${opt:stage, ‘dev’}
bucketname: codalearn-vivek-rawfile-${self:provider.stage}

functions:
LoginUser:
handler: backend/userAuth.login
role: CodaLearnCognitoLambdaRole
environment:
USERPOOLID: !Ref CodaLearnCognitoUserPool
CLIENTID: !Ref CodaLearnCognitoUserPoolClient
REGION: ${self:provider.region}
events:
- http:
path: user
method: post
cors: true
UploadFile:
handler: backend/uploadFile.handler
role: CodaLearnCognitoLambdaRole
environment:
BUCKET: ${self:provider.bucketname}
TABLE_NAME: users-${self:provider.stage}
statemachine_arn: ${self:resources.Outputs.SFParseDataDB.Value}
events:
- http:
path: file
method: post
cors: true
authorizer:
type: COGNITO_USER_POOLS
authorizerId:
Ref: CodaLearnApiGatewayAuthorizer
GetUserFile:
handler: backend/readFile.handler
role: CodaLearnCognitoLambdaRole
environment:
BUCKET: ${self:provider.bucketname}
TABLE_NAME: users-${self:provider.stage}
DATA_TABLE_NAME: users-data-${self:provider.stage}
events:
- http:
path: file
method: get
cors: true
authorizer:
type: COGNITO_USER_POOLS
authorizerId:
Ref: CodaLearnApiGatewayAuthorizer
ParseDataDB:
handler: backend/parseDataDb.handler
role: CodaLearnCognitoLambdaRole
environment:
TABLE_NAME: users-data-${self:provider.stage}

stepFunctions:
stateMachines:
SFParseDataDBSomething:
name: SFParseDataDB
definition:
Comment: “Func to asynchronously upload data to DB”
StartAt: SaveDatatoDB
States:
SaveDatatoDB:
Type: Task
Resource:
Fn::GetAtt: [ParseDataDB, Arn]
End: true

resources:
Resources:

#S3
CodaLearnBucket:
  Type: AWS::S3::Bucket
  Properties:
    BucketName: ${self:provider.bucketname}
    WebsiteConfiguration:
      IndexDocument: index.html
CodaLearnBucketPolicy:
  Type: AWS::S3::BucketPolicy
  Properties:
    Bucket:
      Ref: 'CodaLearnBucket'
    PolicyDocument:
      Statement:
        - Effect: 'Allow'
          Principal: '*'
          Action:
            - 's3:GetObject'
          Resource:
            Fn::Join:
              - ''
              - - 'arn:aws:s3:::'
                - Ref: 'CodaLearnBucket'
                - '/*'

# Dynamo
CodaLearnDynamoDBTable:
  Type: AWS::DynamoDB::Table
  Properties:
    TableName: users-${self:provider.stage}
    AttributeDefinitions:
      - AttributeName: id
        AttributeType: S
      - AttributeName: username
        AttributeType: S
    KeySchema:
      - AttributeName: id
        KeyType: HASH
      - AttributeName: username
        KeyType: RANGE
    ProvisionedThroughput:
      ReadCapacityUnits: 5
      WriteCapacityUnits: 5
CodaLearnDataDynamoDBTable:
  Type: AWS::DynamoDB::Table
  Properties:
    TableName: users-data-${self:provider.stage}
    AttributeDefinitions:
      - AttributeName: id
        AttributeType: S
    KeySchema:
      - AttributeName: id
        KeyType: HASH
    ProvisionedThroughput:
      ReadCapacityUnits: 5
      WriteCapacityUnits: 5


# Cognito
CodaLearnCognitoUserPool:
  Type: AWS::Cognito::UserPool
  Properties:
    UserPoolName: CodaLearn-${self:provider.stage}
CodaLearnCognitoUserPoolClient:
  Type: AWS::Cognito::UserPoolClient
  Properties:
    ClientName: CodaLearnWebApp-${self:provider.stage}
    GenerateSecret: false
    UserPoolId:
      Ref: 'CodaLearnCognitoUserPool'


# Roles
CodaLearnCognitoLambdaRole:
  Type: AWS::IAM::Role
  Properties:
    RoleName: CodaLearnCognitoLambda-${self:provider.stage}
    AssumeRolePolicyDocument:
      Version: '2012-10-17'
      Statement:
        - Effect: Allow
          Principal:
            Service:
              - lambda.amazonaws.com
          Action: sts:AssumeRole
    Policies:
      - PolicyName: CogntioFullAccess
        PolicyDocument:
          Version: '2012-10-17'
          Statement:
            - Effect: Allow
              Action:
                - cognito-identity:*
              Resource:
                - "*"
      - PolicyName: LogWriteAccess
        PolicyDocument:
          Version: '2012-10-17'
          Statement:
            - Effect: Allow
              Action:
                - logs:CreateLogGroup
                - logs:CreateLogStream
                - logs:PutLogEvents
              Resource:
                - 'Fn::Join':
                    - ':'
                    - - 'arn:aws:logs'
                      - Ref: 'AWS::Region'
                      - Ref: 'AWS::AccountId'
                      - 'log-group:/aws/lambda/*:*:*'
      - PolicyName: S3BucketFullAccess
        PolicyDocument:
          Version: '2012-10-17'
          Statement:
            - Effect: Allow
              Action:
                - s3:*
              Resource: '*'
      - PolicyName: DynamoDBFullAccess
        PolicyDocument:
          Version: '2012-10-17'
          Statement:
            - Effect: Allow
              Action:
                - dynamodb:*
              Resource: 
                - 'Fn::GetAtt': [CodaLearnDynamoDBTable, Arn]
                - 'Fn::GetAtt': [CodaLearnDataDynamoDBTable, Arn]
      - PolicyName: StepFnFullAccess
        PolicyDocument:
          Version: '2012-10-17'
          Statement:
            - Effect: Allow
              Action:
                - states:*
              Resource: '*'

# API Gateway
CodaLearnApiGatewayAuthorizer:
  Type: AWS::ApiGateway::Authorizer
  Properties:
    Name: CodaLearn-${self:provider.stage}
    RestApiId:
      Ref: ApiGatewayRestApi
    Type: COGNITO_USER_POOLS
    ProviderARNs:
      - Fn::GetAtt: [CodaLearnCognitoUserPool, Arn]
    IdentitySource: method.request.header.Authorization

Outputs:
CodaLearnCognitoUserPoolId:
Description: ‘Wild Rydes Cognito User Pool ID’
Value:
Ref: ‘CodaLearnCognitoUserPool’
CodaLearnCognitoUserPoolClientId:
Description: ‘Wild Rydes Cognito User Pool Client ID’
Value:
Ref: ‘CodaLearnCognitoUserPoolClient’
CodaLearnDynamoDbARN:
Description: ‘Wild Rydes DynamoDB ARN’
Value:
‘Fn::GetAtt’: [CodaLearnDynamoDBTable, Arn]
SFParseDataDB:
Description: The ARN of the example state machine
Value:
Ref: SFParseDataDB

plugins:

  • serverless-step-functions

The complete error logs can be found in the pipeline logs here - deploy-backend-dev (#1068555958) · Jobs · Vivek George Koshy / coda-learn · GitLab

My gitlab-ci.yml is as follows:

default:
  image: node:latest
stages:
  - deploy
deploy-backend-dev:
  stage: deploy
  rules:
    - if: $CI_MERGE_REQUEST_ID
      when: never
    - if: $CI_COMMIT_REF_NAME =~ /dev/
      changes:
        - backend/**/*
    - if: $CI_COMMIT_REF_NAME =~ /dev/
      changes:
        - .gitlab-ci.yml
  before_script:
    - npm config set prefix /usr/local
    - npm install -g serverless
  script:
    - export AWS_ACCESS_KEY_ID=$DEV_AWS_ACCESS_KEY_ID
    - export AWS_SECRET_ACCESS_KEY=$DEV_AWS_SECRET_ACCESS_KEY
    - export AWS_REGION=$DEV_AWS_REGION
    - npm i
    - export SLS_DEBUG=*
    - sls deploy --verbose
    - sls info --verbose

I’m a little new to serverless, hope these help. Thanks in advance. :slight_smile:

Would you be able to paste your serverless.yml as the issue is likely there.

Sure, i had included the link. I have copied the contents too now.

Hi. I have similar problem. But just similar. Not same. Exceptions are different. Could it be a general CI/CD problem?
My question link

I’m really not sure whats going wrong/right - but it started working for me again on its own.
Now im stuck with a different issue already listed in GitHub Unsettled promises :( · Issue #5972 · serverless/serverless · GitHub

Again, this is only happening in my GitLab pipelines