SSM references failing build

Serverless Framework
1

Hey everyone,

I’ve just tried to implement ssm secrets. I think I’ve done everything correctly but it looks like serverless.yml just won’t accept the { ${ssm:} prefix the build fails (using serverless-webpack.

The failure says :

Profile ${self:custom.profiles.${self:provider.stage}} does not exist

which is an unrelated entry in the serverless file.

it appears to bork at the mere presence of ssm: within the ${} structure.

Every package is up to date, serverless set to ^1.22.0 which is present both locally in the project and globally.

Here is a snip example. It doesn’t matter if I do complex lookups of external files or just reference the parameter directly as below.

Any ideas or links to examples of some config I may be missing from my yml.

cheers
b

custom:
  local: ${file(../developerSpecific.yml)} #a file for developer specific values
  webpackIncludeModules: true
  grub: paymentgateways-api
  defaultStage: dev
  defaultBasePath: paymentgateways
  defaultDomain: ${self:custom.local.customDomainName} #points to developer domain
  profiles:
    dev: default
    uat: mpa-uat
    prod: mpa-serverless-admin
  #defaultRegion: us-east-1
  regions:
    dev: us-east-1
    uat: ap-southeast-2
    prod: us-east-1
  basePath:
    dev: paymentgateways
    uat: paymentgateways
    prod: paymentgateways
  domainName:
    dev: ${self:custom.local.customDomainName}
    uat: ${self:custom.local.customDomainNameUat}
    prod: api.tobedefined.com
  gateways:
    integraPay:
      api-username:
        dev: ${self:custom.local.integraPayApiUsername-uat}
        uat: ${self:custom.local.integraPayApiUsername-uat}
        prod: ${self:custom.local.integraPayApiUsername-uat}
      api-password:
        dev: ${ssm:${self:custom.local.integraPayApiPassword}-true}
        uat: ${self:custom.local.integraPayApiPassword-uat}
        prod: ${self:custom.local.integraPayApiPassword-uat}
2

@delprofundo

Just for clarity, it looks like you’re trying to use SSM here: dev: ${ssm:${self:custom.local.integraPayApiPassword}-true}.

Does ${self:custom.local.integraPayApiPassword} resolve to an actual password, or a key name in SSM? The way it’s written now, that should resolve to a key name that will then be grabbed from SSM.

I’d suggest trying a simpler config file just to isolate the SSM issue:

service: my-service

provider:
  name: aws
  runtime: nodejs6.10
  stage: dev
  environment:
    MY_SSM_SECRET: ${ssm:my-key-name}
functions:
  hello:
    handler: handler.hello
3

Hey Sorry Alex two different places! Here probably smarter

Yep as per my post in Gitter I pulled that complex reference out to just use the simple reference to a key name first before trying the more complex approach.

using v 1.22.0

an example is

dev: ${ssm:/mpa/uat/integra/api/pass-true}

and the error

Profile ${self:custom.profiles.${self:provider.stage}} does not exist

Error --------------------------------------------------

  Profile ${self:custom.profiles.${self:provider.stage}} does not exist

     For debugging logs, run again after setting the "SLS_DEBUG=*" environment variable.

  Stack Trace --------------------------------------------

Error: Profile ${self:custom.profiles.${self:provider.stage}} does not exist
    at Object.addProfileCredentials (/usr/local/lib/node_modules/serverless/lib/plugins/aws/provider/awsProvider.js:79:15)
    at AwsProvider.getCredentials (/usr/local/lib/node_modules/serverless/lib/plugins/aws/provider/awsProvider.js:209:12)
    at AwsProvider.request (/usr/local/lib/node_modules/serverless/lib/plugins/aws/provider/awsProvider.js:150:30)
    at Variables.getValueFromSsm (/usr/local/lib/node_modules/serverless/lib/classes/Variables.js:365:6)
    at Variables.getValueFromSource (/usr/local/lib/node_modules/serverless/lib/classes/Variables.js:184:19)
    at property.match.forEach (/usr/local/lib/node_modules/serverless/lib/classes/Variables.js:98:40)
    at Array.forEach (<anonymous>)
    at Variables.populateProperty (/usr/local/lib/node_modules/serverless/lib/classes/Variables.js:89:43)
    at deepMapValues (/usr/local/lib/node_modules/serverless/lib/classes/Variables.js:69:45)
    at deepMapValues (/usr/local/lib/node_modules/serverless/lib/classes/Variables.js:64:14)
    at deepMapValuesIteratee (/usr/local/lib/node_modules/serverless/lib/classes/Variables.js:54:25)
    at /usr/local/lib/node_modules/serverless/node_modules/lodash/lodash.js:13379:38
    at /usr/local/lib/node_modules/serverless/node_modules/lodash/lodash.js:4944:15
    at baseForOwn (/usr/local/lib/node_modules/serverless/node_modules/lodash/lodash.js:3001:24)
    at Function.mapValues (/usr/local/lib/node_modules/serverless/node_modules/lodash/lodash.js:13378:7)
    at deepMapValues (/usr/local/lib/node_modules/serverless/lib/classes/Variables.js:62:39)
    at deepMapValuesIteratee (/usr/local/lib/node_modules/serverless/lib/classes/Variables.js:54:25)
    at /usr/local/lib/node_modules/serverless/node_modules/lodash/lodash.js:13379:38
    at /usr/local/lib/node_modules/serverless/node_modules/lodash/lodash.js:4944:15
    at baseForOwn (/usr/local/lib/node_modules/serverless/node_modules/lodash/lodash.js:3001:24)
    at Function.mapValues (/usr/local/lib/node_modules/serverless/node_modules/lodash/lodash.js:13378:7)
    at deepMapValues (/usr/local/lib/node_modules/serverless/lib/classes/Variables.js:62:39)
    at deepMapValuesIteratee (/usr/local/lib/node_modules/serverless/lib/classes/Variables.js:54:25)
    at /usr/local/lib/node_modules/serverless/node_modules/lodash/lodash.js:13379:38
    at /usr/local/lib/node_modules/serverless/node_modules/lodash/lodash.js:4944:15
    at baseForOwn (/usr/local/lib/node_modules/serverless/node_modules/lodash/lodash.js:3001:24)
    at Function.mapValues (/usr/local/lib/node_modules/serverless/node_modules/lodash/lodash.js:13378:7)
    at deepMapValues (/usr/local/lib/node_modules/serverless/lib/classes/Variables.js:62:39)
    at deepMapValuesIteratee (/usr/local/lib/node_modules/serverless/lib/classes/Variables.js:54:25)
    at /usr/local/lib/node_modules/serverless/node_modules/lodash/lodash.js:13379:38
    at /usr/local/lib/node_modules/serverless/node_modules/lodash/lodash.js:4944:15
    at baseForOwn (/usr/local/lib/node_modules/serverless/node_modules/lodash/lodash.js:3001:24)
    at Function.mapValues (/usr/local/lib/node_modules/serverless/node_modules/lodash/lodash.js:13378:7)
    at deepMapValues (/usr/local/lib/node_modules/serverless/lib/classes/Variables.js:62:39)
    at deepMapValuesIteratee (/usr/local/lib/node_modules/serverless/lib/classes/Variables.js:54:25)
    at /usr/local/lib/node_modules/serverless/node_modules/lodash/lodash.js:13379:38
    at /usr/local/lib/node_modules/serverless/node_modules/lodash/lodash.js:4944:15
    at baseForOwn (/usr/local/lib/node_modules/serverless/node_modules/lodash/lodash.js:3001:24)
    at Function.mapValues (/usr/local/lib/node_modules/serverless/node_modules/lodash/lodash.js:13378:7)
    at deepMapValues (/usr/local/lib/node_modules/serverless/lib/classes/Variables.js:62:39)
    at Variables.populateObject (/usr/local/lib/node_modules/serverless/lib/classes/Variables.js:67:5)
    at Variables.populateService (/usr/local/lib/node_modules/serverless/lib/classes/Variables.js:43:17)
    at Serverless.run (/usr/local/lib/node_modules/serverless/lib/Serverless.js:86:27)
    at serverless.init.then (/usr/local/lib/node_modules/serverless/bin/serverless:39:50)
    at <anonymous>
From previous event:
    at runCallback (timers.js:781:20)
    at tryOnImmediate (timers.js:743:5)
    at processImmediate [as _immediateCallback] (timers.js:714:5)
From previous event:
    at __dirname (/usr/local/lib/node_modules/serverless/bin/serverless:25:46)
    at Object.<anonymous> (/usr/local/lib/node_modules/serverless/bin/serverless:43:4)
    at Module._compile (module.js:624:30)
    at Object.Module._extensions..js (module.js:635:10)
    at Module.load (module.js:545:32)
    at tryModuleLoad (module.js:508:12)
    at Function.Module._load (module.js:500:3)
    at Function.Module.runMain (module.js:665:10)
    at startup (bootstrap_node.js:201:16)
    at bootstrap_node.js:626:3

  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
     Forums:        forum.serverless.com
     Chat:          gitter.im/serverless/serverless

  Your Environment Information -----------------------------
     OS:                     darwin
     Node Version:           8.5.0
     Serverless Version:     1.22.0

and my custom section:

custom:
  local: ${file(../developerSpecific.yml)} #a file for developer specific values
  webpackIncludeModules: true
  grub: paymentgateways-api
  defaultStage: dev
  defaultBasePath: paymentgateways
  defaultDomain: ${self:custom.local.customDomainName} #points to developer domain
  profiles:
    dev: default
    uat: mpa-uat
    prod: mpa-serverless-admin
  #defaultRegion: us-east-1
  regions:
    dev: us-east-1
    uat: ap-southeast-2
    prod: us-east-1
  basePath:
    dev: paymentgateways
    uat: paymentgateways
    prod: paymentgateways
  domainName:
    dev: ${self:custom.local.customDomainName}
    uat: ${self:custom.local.customDomainNameUat}
    prod: api.tobedefined.com
  gateways:
    integraPay:
      api-username:
        dev: ${self:custom.local.integraPayApiUsername-uat}
        uat: ${self:custom.local.integraPayApiUsername-uat}
        prod: ${self:custom.local.integraPayApiUsername-uat}
      api-password:
        #dev: ${self:custom.local.integraPayApiPassword-uat}
        dev: ${ssm:/mpa/uat/integra/api/pass-true}
        #dev: ${ssm:${self:custom.local.integraPayApiPassword}-true}
        uat: ${self:custom.local.integraPayApiPassword-uat}
        prod: ${self:custom.local.integraPayApiPassword-uat}
      api-url:
        dev: https://apitest.integrapay.com.au/basic/PayLinkService.svc?WSDL
        uat: https://apitest.integrapay.com.au/basic/PayLinkService.svc?WSDL
        prod:  https://api.integrapay.com.au/basic/PayLinkService.svc?WSDL
      api-vault:
        dev: https://testpayments.integrapay.com.au/API/API.ashx
        uat: https://testpayments.integrapay.com.au/API/API.ashx
        prod:  https://payments.integrapay.com.au/API/API.ashx
4

@delprofundo

One thing to note is that it’s actually a tilde (~) not a dash (-) for the ~true if you want to decrypt a secure string.

So it should be:

dev: ${ssm:/mpa/uat/integra/api/pass~true}

I’m guessing this isn’t causing your problem, but can you fix that first and give it a shot?

5

Ah should have mentioned that yes I’ve changed it to ~ and no change, its the presence of the prefix thats borking.

6

@delprofundo Could you create a new, separate Serverless service with a simple serverless.yml just for testing?

Use:

service: my-service

provider:
  name: aws
  runtime: nodejs6.10
  stage: dev
  environment:
    MY_SSM_SECRET: ${ssm:/mpa/uat/integra/api/pass~true}
functions:
  hello:
    handler: handler.hello

And see if it resolves.

Also, can you paste your provider block?

7

Hey Alex,

Yep that deployed. Below is my entire serverless.yml for the project, that might be the best bet.

Some extra fooling around I did. I hard coded the reference to the profile that was boring oringinally. that then leaves me with the following message:

Inaccessible host: `ssm.'. This service may not be available in the `${opt:region, self:custom.regions.${self:provider.stage}}' region.

strange. feels like the custom’s aren’t being exploded.

It could be something to do with the webpack-serverless plugin as if I remove it from the plugin sections the the build doesn’t bork right at the start.

service: paymentGatewayService
frameworkVersion: ^1.22.0

provider:
  name: aws
  runtime: nodejs6.10
  stage: ${opt:stage, self:custom.defaultStage}
  profile: ${self:custom.profiles.${self:provider.stage}}
  region: ${opt:region, self:custom.regions.${self:provider.stage}}
  memorySize: 128 # this parameter doesnt only change how much memory is available but also CPU's and container life.
  timeout: 20
  environment:
    DEPLOY_REGION: ${opt:region, self:provider.region}
    INTEGRA_API_PASS: ${self:custom.gateways.integraPay.api-password.${self:provider.stage}}
    INTEGRA_USER_NAME: ${self:custom.gateways.integraPay.api-username.${self:provider.stage}}
    INTEGRA_API_URL: ${self:custom.gateways.integraPay.api-url.${self:provider.stage}}
    INTEGRA_VAULT_URL: ${self:custom.gateways.integraPay.api-vault.${self:provider.stage}}
    PAYMENTGATEWAY_TABLE_NAME: paymentgateways-${opt:stage, self:provider.stage}
    PAYMENTS_TABLE_NAME: payments-${opt:stage, self:provider.stage}
    #functions that we dont want exposed via HTTP
    POST_PAYMENT: ${self:provider.stage}-${self:custom.grub}-postPayment
    #internal lambda links for non-http endpoints
    POST_TRANSACTION_STATUS_UPDATE: ${self:provider.stage}-transactions-api-postTransactionStatusUpdate
  iamRoleStatements:
    - Effect: Allow
      Action:
        - lambda:InvokeFunction
        - lambda:InvokeAsync
      Resource: "*"
package:
  #individually: true
  include:
    - configurators
    - lib
  exclude:
    - tmp
    - .git
    - .idea
plugins:
  - serverless-mocha-plugin
  #- serverless-snyk
  - serverless-dynamodb-local
  #- serverless-offline
  - serverless-webpack
  - serverless-plugin-bind-deployment-id
custom:
  local: ${file(../developerSpecific.yml)} #a file for developer specific values
  webpackIncludeModules: true
  grub: paymentgateways-api
  defaultStage: dev
  defaultBasePath: paymentgateways
  defaultDomain: ${self:custom.local.customDomainName} #points to developer domain
  profiles:
    dev: default
    uat: mpa-uat
    prod: mpa-serverless-admin
  #defaultRegion: us-east-1
  regions:
    dev: us-east-1
    uat: ap-southeast-2
    prod: us-east-1
  basePath:
    dev: paymentgateways
    uat: paymentgateways
    prod: paymentgateways
  domainName:
    dev: ${self:custom.local.customDomainName}
    uat: ${self:custom.local.customDomainNameUat}
    prod: api.tobedefined.com
  gateways:
    integraPay:
      api-username:
        dev: ${self:custom.local.integraPayApiUsername-uat}
        uat: ${self:custom.local.integraPayApiUsername-uat}
        prod: ${self:custom.local.integraPayApiUsername-uat}
      api-password:
        dev: ${self:custom.local.integraPayApiPassword-uat}
        #dev: ${ssm:mpa/uat/integra/api/pass~true}
        #dev: ${ssm:${self:custom.local.integraPayApiPassword}~true}
        uat: ${self:custom.local.integraPayApiPassword-uat}
        prod: ${self:custom.local.integraPayApiPassword-uat}
      api-url:
        dev: https://apitest.integrapay.com.au/basic/PayLinkService.svc?WSDL
        uat: https://apitest.integrapay.com.au/basic/PayLinkService.svc?WSDL
        prod:  https://api.integrapay.com.au/basic/PayLinkService.svc?WSDL
      api-vault:
        dev: https://testpayments.integrapay.com.au/API/API.ashx
        uat: https://testpayments.integrapay.com.au/API/API.ashx
        prod:  https://payments.integrapay.com.au/API/API.ashx

functions:
  pingPaymentgateways:
    handler: handler.ping
    name: ${self:provider.stage}-${self:custom.grub}-ping
    events:
    - http:
        path: ping
        method: get
  getPayments:
    handler: handler.getPayments
    name: ${self:provider.stage}-${self:custom.grub}-getPayments
    description: returns a list of payments. could be called by other services
    events:
    - http:
        path: /
        method: get
  getPaymentById:
    handler: handler.getPaymentById
    name: ${self:provider.stage}-${self:custom.grub}-getPaymentById
    description: get a single payment record. could be called by other services
    events:
    - http:
        path: /{paymentId}
        method: get
  postPayment:
    handler: handler.postPayment
    name: ${self:provider.stage}-${self:custom.grub}-postPayment
    description: The endpoint to post payments to internally
  postPaymentInstrument:
    handler: handler.postPaymentInstrument
    name: ${self:provider.stage}-${self:custom.grub}-postPaymentInstrument
    description: The endpoint to post new or updated payment instruments to internally for gateway configuration
  postUser:
    handler: handler.postUser
    name: ${self:provider.stage}-${self:custom.grub}-postUser
    description: The endpoint to post new users to internally for gateway configuration
  processPayment:
    handler: handler.processPayment
    name: ${self:provider.stage}-${self:custom.grub}-processPayment
    description: Triggered to push transactions to the external gateways
    events:
      - stream:
        "Fn::Join": ["", ["arn:aws:dynamodb:", {"Ref": "AWS::Region"}, ":", {"Ref": "AWS::AccountId"}, ":table/", "payments-${opt:stage, self:provider.stage}", "/stream/*"]]
  processGatewayUpdates:
    handler: handler.processGatewayUpdates
    timeout: 60
    name: ${self:provider.stage}-${self:custom.grub}-processGatewayUpdates
    description: Scheduled function that polls the gateways and processes their responses
    events:
    - schedule: rate(5 minutes)
  requestVaultSession:
    handler: handler.requestVaultSession
    name: ${self:provider.stage}-${self:custom.grub}-requestVaultSession
    description: Function to get a vault response from a gateway
  checkVaultSession:
    handler: handler.checkVaultSession
    name: ${self:provider.stage}-${self:custom.grub}-checkVaultSession
    description: simple internal endpoint that looks up a user to see if they have a card attached.
  #processVaultSession:
  #  handler: handler.processVaultSession
  #  name: ${self:provider.stage}-${self:custom.grub}-processVaultSession
  #  description: Function to get card details
  getGateways:
    handler: handler.getGateways
    name: ${self:provider.stage}-${self:custom.grub}-getGateways
    events:
    - http:
        path: gateways
        method: get
        cors: true
  getGatewayById:
    handler: handler.getGatewayById
    name: ${self:provider.stage}-${self:custom.grub}-getGatewayById
    events:
    - http:
        path: gateways/{gatewayId}
        method: get
        cors: true
  getGatewayPaymentMethodConfigurations:
    handler: handler.getGatewayPaymentMethodConfigurations
    name: ${self:provider.stage}-${self:custom.grub}-getGatewayPaymentMethodConfigurations
    events:
    - http:
        path: gateways/{gatewayId}/paymentMethodConfigurations
        method: get
        cors: true
  getGatewayPaymentMethodConfigurationById:
    handler: handler.getGatewayPaymentMethodConfigurationById
    name: ${self:provider.stage}-${self:custom.grub}-getGatewayPaymentMethodConfigurationById
    events:
    - http:
        path: gateways/{gatewayId}/paymentMethodConfigurations/{paymentMethodConfigurationId}
        method: get
        cors: true
  deleteGatewayPaymentConfigurationById:
    handler: handler.deleteGatewayPaymentConfigurationById
    name: ${self:provider.stage}-${self:custom.grub}-deleteGatewayPaymentConfigurationById
    events:
    - http:
        path: gateways/{gatewayId}/paymentMethodConfigurations/{paymentMethodConfigurationId}
        method: delete
        cors: true
# Resources are essentially CloudFormation scripts. These are kept in the configurators folder
resources:
  Resources:
    __deployment__:
      Properties:
        Description: Deployment alias for resources that depend on deployment id
    PaymentsTable:
      $ref: ./configurators/paymentsTable.yaml
    PushPaymentLambdaMapping:
      $ref: ./configurators/processPaymentEventSourceMapping.yaml
    PaymentGatewaysTable:
      $ref: ./configurators/paymentGatewaysTable.yaml
    IamPolicyLambdaDynamo:
      $ref: ./configurators/iamPolicyLambdaDynamo.yaml
    PathMapping:
      $ref: ./configurators/pathMapping.yaml
    ApiGatewayStage:
      $ref: ./configurators/apiGatewayStage.yaml
8

Hi @delprofundo,

Did you figure this out? I’m having the exact same issue.

Cheers,
Adrian