SSM references failing build

Hey everyone,

I’ve just tried to implement ssm secrets. I think I’ve done everything correctly but it looks like serverless.yml just won’t accept the { ${ssm:} prefix the build fails (using serverless-webpack.

The failure says :

Profile ${self:custom.profiles.${self:provider.stage}} does not exist

which is an unrelated entry in the serverless file.

it appears to bork at the mere presence of ssm: within the ${} structure.

Every package is up to date, serverless set to ^1.22.0 which is present both locally in the project and globally.

Here is a snip example. It doesn’t matter if I do complex lookups of external files or just reference the parameter directly as below.

Any ideas or links to examples of some config I may be missing from my yml.

cheers
b

custom:
  local: ${file(../developerSpecific.yml)} #a file for developer specific values
  webpackIncludeModules: true
  grub: paymentgateways-api
  defaultStage: dev
  defaultBasePath: paymentgateways
  defaultDomain: ${self:custom.local.customDomainName} #points to developer domain
  profiles:
    dev: default
    uat: mpa-uat
    prod: mpa-serverless-admin
  #defaultRegion: us-east-1
  regions:
    dev: us-east-1
    uat: ap-southeast-2
    prod: us-east-1
  basePath:
    dev: paymentgateways
    uat: paymentgateways
    prod: paymentgateways
  domainName:
    dev: ${self:custom.local.customDomainName}
    uat: ${self:custom.local.customDomainNameUat}
    prod: api.tobedefined.com
  gateways:
    integraPay:
      api-username:
        dev: ${self:custom.local.integraPayApiUsername-uat}
        uat: ${self:custom.local.integraPayApiUsername-uat}
        prod: ${self:custom.local.integraPayApiUsername-uat}
      api-password:
        dev: ${ssm:${self:custom.local.integraPayApiPassword}-true}
        uat: ${self:custom.local.integraPayApiPassword-uat}
        prod: ${self:custom.local.integraPayApiPassword-uat}

@delprofundo

Just for clarity, it looks like you’re trying to use SSM here: dev: ${ssm:${self:custom.local.integraPayApiPassword}-true}.

Does ${self:custom.local.integraPayApiPassword} resolve to an actual password, or a key name in SSM? The way it’s written now, that should resolve to a key name that will then be grabbed from SSM.

I’d suggest trying a simpler config file just to isolate the SSM issue:

service: my-service

provider:
  name: aws
  runtime: nodejs6.10
  stage: dev
  environment:
    MY_SSM_SECRET: ${ssm:my-key-name}
functions:
  hello:
    handler: handler.hello

Hey Sorry Alex two different places! Here probably smarter

Yep as per my post in Gitter I pulled that complex reference out to just use the simple reference to a key name first before trying the more complex approach.

using v 1.22.0

an example is

dev: ${ssm:/mpa/uat/integra/api/pass-true}

and the error

Profile ${self:custom.profiles.${self:provider.stage}} does not exist

Error --------------------------------------------------

  Profile ${self:custom.profiles.${self:provider.stage}} does not exist

     For debugging logs, run again after setting the "SLS_DEBUG=*" environment variable.

  Stack Trace --------------------------------------------

Error: Profile ${self:custom.profiles.${self:provider.stage}} does not exist
    at Object.addProfileCredentials (/usr/local/lib/node_modules/serverless/lib/plugins/aws/provider/awsProvider.js:79:15)
    at AwsProvider.getCredentials (/usr/local/lib/node_modules/serverless/lib/plugins/aws/provider/awsProvider.js:209:12)
    at AwsProvider.request (/usr/local/lib/node_modules/serverless/lib/plugins/aws/provider/awsProvider.js:150:30)
    at Variables.getValueFromSsm (/usr/local/lib/node_modules/serverless/lib/classes/Variables.js:365:6)
    at Variables.getValueFromSource (/usr/local/lib/node_modules/serverless/lib/classes/Variables.js:184:19)
    at property.match.forEach (/usr/local/lib/node_modules/serverless/lib/classes/Variables.js:98:40)
    at Array.forEach (<anonymous>)
    at Variables.populateProperty (/usr/local/lib/node_modules/serverless/lib/classes/Variables.js:89:43)
    at deepMapValues (/usr/local/lib/node_modules/serverless/lib/classes/Variables.js:69:45)
    at deepMapValues (/usr/local/lib/node_modules/serverless/lib/classes/Variables.js:64:14)
    at deepMapValuesIteratee (/usr/local/lib/node_modules/serverless/lib/classes/Variables.js:54:25)
    at /usr/local/lib/node_modules/serverless/node_modules/lodash/lodash.js:13379:38
    at /usr/local/lib/node_modules/serverless/node_modules/lodash/lodash.js:4944:15
    at baseForOwn (/usr/local/lib/node_modules/serverless/node_modules/lodash/lodash.js:3001:24)
    at Function.mapValues (/usr/local/lib/node_modules/serverless/node_modules/lodash/lodash.js:13378:7)
    at deepMapValues (/usr/local/lib/node_modules/serverless/lib/classes/Variables.js:62:39)
    at deepMapValuesIteratee (/usr/local/lib/node_modules/serverless/lib/classes/Variables.js:54:25)
    at /usr/local/lib/node_modules/serverless/node_modules/lodash/lodash.js:13379:38
    at /usr/local/lib/node_modules/serverless/node_modules/lodash/lodash.js:4944:15
    at baseForOwn (/usr/local/lib/node_modules/serverless/node_modules/lodash/lodash.js:3001:24)
    at Function.mapValues (/usr/local/lib/node_modules/serverless/node_modules/lodash/lodash.js:13378:7)
    at deepMapValues (/usr/local/lib/node_modules/serverless/lib/classes/Variables.js:62:39)
    at deepMapValuesIteratee (/usr/local/lib/node_modules/serverless/lib/classes/Variables.js:54:25)
    at /usr/local/lib/node_modules/serverless/node_modules/lodash/lodash.js:13379:38
    at /usr/local/lib/node_modules/serverless/node_modules/lodash/lodash.js:4944:15
    at baseForOwn (/usr/local/lib/node_modules/serverless/node_modules/lodash/lodash.js:3001:24)
    at Function.mapValues (/usr/local/lib/node_modules/serverless/node_modules/lodash/lodash.js:13378:7)
    at deepMapValues (/usr/local/lib/node_modules/serverless/lib/classes/Variables.js:62:39)
    at deepMapValuesIteratee (/usr/local/lib/node_modules/serverless/lib/classes/Variables.js:54:25)
    at /usr/local/lib/node_modules/serverless/node_modules/lodash/lodash.js:13379:38
    at /usr/local/lib/node_modules/serverless/node_modules/lodash/lodash.js:4944:15
    at baseForOwn (/usr/local/lib/node_modules/serverless/node_modules/lodash/lodash.js:3001:24)
    at Function.mapValues (/usr/local/lib/node_modules/serverless/node_modules/lodash/lodash.js:13378:7)
    at deepMapValues (/usr/local/lib/node_modules/serverless/lib/classes/Variables.js:62:39)
    at deepMapValuesIteratee (/usr/local/lib/node_modules/serverless/lib/classes/Variables.js:54:25)
    at /usr/local/lib/node_modules/serverless/node_modules/lodash/lodash.js:13379:38
    at /usr/local/lib/node_modules/serverless/node_modules/lodash/lodash.js:4944:15
    at baseForOwn (/usr/local/lib/node_modules/serverless/node_modules/lodash/lodash.js:3001:24)
    at Function.mapValues (/usr/local/lib/node_modules/serverless/node_modules/lodash/lodash.js:13378:7)
    at deepMapValues (/usr/local/lib/node_modules/serverless/lib/classes/Variables.js:62:39)
    at Variables.populateObject (/usr/local/lib/node_modules/serverless/lib/classes/Variables.js:67:5)
    at Variables.populateService (/usr/local/lib/node_modules/serverless/lib/classes/Variables.js:43:17)
    at Serverless.run (/usr/local/lib/node_modules/serverless/lib/Serverless.js:86:27)
    at serverless.init.then (/usr/local/lib/node_modules/serverless/bin/serverless:39:50)
    at <anonymous>
From previous event:
    at runCallback (timers.js:781:20)
    at tryOnImmediate (timers.js:743:5)
    at processImmediate [as _immediateCallback] (timers.js:714:5)
From previous event:
    at __dirname (/usr/local/lib/node_modules/serverless/bin/serverless:25:46)
    at Object.<anonymous> (/usr/local/lib/node_modules/serverless/bin/serverless:43:4)
    at Module._compile (module.js:624:30)
    at Object.Module._extensions..js (module.js:635:10)
    at Module.load (module.js:545:32)
    at tryModuleLoad (module.js:508:12)
    at Function.Module._load (module.js:500:3)
    at Function.Module.runMain (module.js:665:10)
    at startup (bootstrap_node.js:201:16)
    at bootstrap_node.js:626:3

  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
     Forums:        forum.serverless.com
     Chat:          gitter.im/serverless/serverless

  Your Environment Information -----------------------------
     OS:                     darwin
     Node Version:           8.5.0
     Serverless Version:     1.22.0

and my custom section:

custom:
  local: ${file(../developerSpecific.yml)} #a file for developer specific values
  webpackIncludeModules: true
  grub: paymentgateways-api
  defaultStage: dev
  defaultBasePath: paymentgateways
  defaultDomain: ${self:custom.local.customDomainName} #points to developer domain
  profiles:
    dev: default
    uat: mpa-uat
    prod: mpa-serverless-admin
  #defaultRegion: us-east-1
  regions:
    dev: us-east-1
    uat: ap-southeast-2
    prod: us-east-1
  basePath:
    dev: paymentgateways
    uat: paymentgateways
    prod: paymentgateways
  domainName:
    dev: ${self:custom.local.customDomainName}
    uat: ${self:custom.local.customDomainNameUat}
    prod: api.tobedefined.com
  gateways:
    integraPay:
      api-username:
        dev: ${self:custom.local.integraPayApiUsername-uat}
        uat: ${self:custom.local.integraPayApiUsername-uat}
        prod: ${self:custom.local.integraPayApiUsername-uat}
      api-password:
        #dev: ${self:custom.local.integraPayApiPassword-uat}
        dev: ${ssm:/mpa/uat/integra/api/pass-true}
        #dev: ${ssm:${self:custom.local.integraPayApiPassword}-true}
        uat: ${self:custom.local.integraPayApiPassword-uat}
        prod: ${self:custom.local.integraPayApiPassword-uat}
      api-url:
        dev: https://apitest.integrapay.com.au/basic/PayLinkService.svc?WSDL
        uat: https://apitest.integrapay.com.au/basic/PayLinkService.svc?WSDL
        prod:  https://api.integrapay.com.au/basic/PayLinkService.svc?WSDL
      api-vault:
        dev: https://testpayments.integrapay.com.au/API/API.ashx
        uat: https://testpayments.integrapay.com.au/API/API.ashx
        prod:  https://payments.integrapay.com.au/API/API.ashx

@delprofundo

One thing to note is that it’s actually a tilde (~) not a dash (-) for the ~true if you want to decrypt a secure string.

So it should be:

dev: ${ssm:/mpa/uat/integra/api/pass~true}

I’m guessing this isn’t causing your problem, but can you fix that first and give it a shot?

Ah should have mentioned that yes I’ve changed it to ~ and no change, its the presence of the prefix thats borking.

@delprofundo Could you create a new, separate Serverless service with a simple serverless.yml just for testing?

Use:

service: my-service

provider:
  name: aws
  runtime: nodejs6.10
  stage: dev
  environment:
    MY_SSM_SECRET: ${ssm:/mpa/uat/integra/api/pass~true}
functions:
  hello:
    handler: handler.hello

And see if it resolves.

Also, can you paste your provider block?

Hey Alex,

Yep that deployed. Below is my entire serverless.yml for the project, that might be the best bet.

Some extra fooling around I did. I hard coded the reference to the profile that was boring oringinally. that then leaves me with the following message:

Inaccessible host: `ssm.'. This service may not be available in the `${opt:region, self:custom.regions.${self:provider.stage}}' region.

strange. feels like the custom’s aren’t being exploded.

It could be something to do with the webpack-serverless plugin as if I remove it from the plugin sections the the build doesn’t bork right at the start.

service: paymentGatewayService
frameworkVersion: ^1.22.0

provider:
  name: aws
  runtime: nodejs6.10
  stage: ${opt:stage, self:custom.defaultStage}
  profile: ${self:custom.profiles.${self:provider.stage}}
  region: ${opt:region, self:custom.regions.${self:provider.stage}}
  memorySize: 128 # this parameter doesnt only change how much memory is available but also CPU's and container life.
  timeout: 20
  environment:
    DEPLOY_REGION: ${opt:region, self:provider.region}
    INTEGRA_API_PASS: ${self:custom.gateways.integraPay.api-password.${self:provider.stage}}
    INTEGRA_USER_NAME: ${self:custom.gateways.integraPay.api-username.${self:provider.stage}}
    INTEGRA_API_URL: ${self:custom.gateways.integraPay.api-url.${self:provider.stage}}
    INTEGRA_VAULT_URL: ${self:custom.gateways.integraPay.api-vault.${self:provider.stage}}
    PAYMENTGATEWAY_TABLE_NAME: paymentgateways-${opt:stage, self:provider.stage}
    PAYMENTS_TABLE_NAME: payments-${opt:stage, self:provider.stage}
    #functions that we dont want exposed via HTTP
    POST_PAYMENT: ${self:provider.stage}-${self:custom.grub}-postPayment
    #internal lambda links for non-http endpoints
    POST_TRANSACTION_STATUS_UPDATE: ${self:provider.stage}-transactions-api-postTransactionStatusUpdate
  iamRoleStatements:
    - Effect: Allow
      Action:
        - lambda:InvokeFunction
        - lambda:InvokeAsync
      Resource: "*"
package:
  #individually: true
  include:
    - configurators
    - lib
  exclude:
    - tmp
    - .git
    - .idea
plugins:
  - serverless-mocha-plugin
  #- serverless-snyk
  - serverless-dynamodb-local
  #- serverless-offline
  - serverless-webpack
  - serverless-plugin-bind-deployment-id
custom:
  local: ${file(../developerSpecific.yml)} #a file for developer specific values
  webpackIncludeModules: true
  grub: paymentgateways-api
  defaultStage: dev
  defaultBasePath: paymentgateways
  defaultDomain: ${self:custom.local.customDomainName} #points to developer domain
  profiles:
    dev: default
    uat: mpa-uat
    prod: mpa-serverless-admin
  #defaultRegion: us-east-1
  regions:
    dev: us-east-1
    uat: ap-southeast-2
    prod: us-east-1
  basePath:
    dev: paymentgateways
    uat: paymentgateways
    prod: paymentgateways
  domainName:
    dev: ${self:custom.local.customDomainName}
    uat: ${self:custom.local.customDomainNameUat}
    prod: api.tobedefined.com
  gateways:
    integraPay:
      api-username:
        dev: ${self:custom.local.integraPayApiUsername-uat}
        uat: ${self:custom.local.integraPayApiUsername-uat}
        prod: ${self:custom.local.integraPayApiUsername-uat}
      api-password:
        dev: ${self:custom.local.integraPayApiPassword-uat}
        #dev: ${ssm:mpa/uat/integra/api/pass~true}
        #dev: ${ssm:${self:custom.local.integraPayApiPassword}~true}
        uat: ${self:custom.local.integraPayApiPassword-uat}
        prod: ${self:custom.local.integraPayApiPassword-uat}
      api-url:
        dev: https://apitest.integrapay.com.au/basic/PayLinkService.svc?WSDL
        uat: https://apitest.integrapay.com.au/basic/PayLinkService.svc?WSDL
        prod:  https://api.integrapay.com.au/basic/PayLinkService.svc?WSDL
      api-vault:
        dev: https://testpayments.integrapay.com.au/API/API.ashx
        uat: https://testpayments.integrapay.com.au/API/API.ashx
        prod:  https://payments.integrapay.com.au/API/API.ashx

functions:
  pingPaymentgateways:
    handler: handler.ping
    name: ${self:provider.stage}-${self:custom.grub}-ping
    events:
    - http:
        path: ping
        method: get
  getPayments:
    handler: handler.getPayments
    name: ${self:provider.stage}-${self:custom.grub}-getPayments
    description: returns a list of payments. could be called by other services
    events:
    - http:
        path: /
        method: get
  getPaymentById:
    handler: handler.getPaymentById
    name: ${self:provider.stage}-${self:custom.grub}-getPaymentById
    description: get a single payment record. could be called by other services
    events:
    - http:
        path: /{paymentId}
        method: get
  postPayment:
    handler: handler.postPayment
    name: ${self:provider.stage}-${self:custom.grub}-postPayment
    description: The endpoint to post payments to internally
  postPaymentInstrument:
    handler: handler.postPaymentInstrument
    name: ${self:provider.stage}-${self:custom.grub}-postPaymentInstrument
    description: The endpoint to post new or updated payment instruments to internally for gateway configuration
  postUser:
    handler: handler.postUser
    name: ${self:provider.stage}-${self:custom.grub}-postUser
    description: The endpoint to post new users to internally for gateway configuration
  processPayment:
    handler: handler.processPayment
    name: ${self:provider.stage}-${self:custom.grub}-processPayment
    description: Triggered to push transactions to the external gateways
    events:
      - stream:
        "Fn::Join": ["", ["arn:aws:dynamodb:", {"Ref": "AWS::Region"}, ":", {"Ref": "AWS::AccountId"}, ":table/", "payments-${opt:stage, self:provider.stage}", "/stream/*"]]
  processGatewayUpdates:
    handler: handler.processGatewayUpdates
    timeout: 60
    name: ${self:provider.stage}-${self:custom.grub}-processGatewayUpdates
    description: Scheduled function that polls the gateways and processes their responses
    events:
    - schedule: rate(5 minutes)
  requestVaultSession:
    handler: handler.requestVaultSession
    name: ${self:provider.stage}-${self:custom.grub}-requestVaultSession
    description: Function to get a vault response from a gateway
  checkVaultSession:
    handler: handler.checkVaultSession
    name: ${self:provider.stage}-${self:custom.grub}-checkVaultSession
    description: simple internal endpoint that looks up a user to see if they have a card attached.
  #processVaultSession:
  #  handler: handler.processVaultSession
  #  name: ${self:provider.stage}-${self:custom.grub}-processVaultSession
  #  description: Function to get card details
  getGateways:
    handler: handler.getGateways
    name: ${self:provider.stage}-${self:custom.grub}-getGateways
    events:
    - http:
        path: gateways
        method: get
        cors: true
  getGatewayById:
    handler: handler.getGatewayById
    name: ${self:provider.stage}-${self:custom.grub}-getGatewayById
    events:
    - http:
        path: gateways/{gatewayId}
        method: get
        cors: true
  getGatewayPaymentMethodConfigurations:
    handler: handler.getGatewayPaymentMethodConfigurations
    name: ${self:provider.stage}-${self:custom.grub}-getGatewayPaymentMethodConfigurations
    events:
    - http:
        path: gateways/{gatewayId}/paymentMethodConfigurations
        method: get
        cors: true
  getGatewayPaymentMethodConfigurationById:
    handler: handler.getGatewayPaymentMethodConfigurationById
    name: ${self:provider.stage}-${self:custom.grub}-getGatewayPaymentMethodConfigurationById
    events:
    - http:
        path: gateways/{gatewayId}/paymentMethodConfigurations/{paymentMethodConfigurationId}
        method: get
        cors: true
  deleteGatewayPaymentConfigurationById:
    handler: handler.deleteGatewayPaymentConfigurationById
    name: ${self:provider.stage}-${self:custom.grub}-deleteGatewayPaymentConfigurationById
    events:
    - http:
        path: gateways/{gatewayId}/paymentMethodConfigurations/{paymentMethodConfigurationId}
        method: delete
        cors: true
# Resources are essentially CloudFormation scripts. These are kept in the configurators folder
resources:
  Resources:
    __deployment__:
      Properties:
        Description: Deployment alias for resources that depend on deployment id
    PaymentsTable:
      $ref: ./configurators/paymentsTable.yaml
    PushPaymentLambdaMapping:
      $ref: ./configurators/processPaymentEventSourceMapping.yaml
    PaymentGatewaysTable:
      $ref: ./configurators/paymentGatewaysTable.yaml
    IamPolicyLambdaDynamo:
      $ref: ./configurators/iamPolicyLambdaDynamo.yaml
    PathMapping:
      $ref: ./configurators/pathMapping.yaml
    ApiGatewayStage:
      $ref: ./configurators/apiGatewayStage.yaml

Hi @delprofundo,

Did you figure this out? I’m having the exact same issue.

Cheers,
Adrian