# Can I access outputs from custom resources as variables in serverless.yml?

**URL:** <https://forum.serverless.com/t/can-i-access-outputs-from-custom-resources-as-variables-in-serverless-yml/508>\
**Category:** Serverless Framework\
**Created:** [October 12, 2016, 11:27pm UTC](https://forum.serverless.com/t/can-i-access-outputs-from-custom-resources-as-variables-in-serverless-yml/508 "2016-10-12T23:27:11Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![viz](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.serverless.com/viz/32/200_2.png) [@viz](https://forum.serverless.com/u/viz)\
**Post date:** [October 12, 2016, 11:27pm UTC](https://forum.serverless.com/t/can-i-access-outputs-from-custom-resources-as-variables-in-serverless-yml/508/1 "2016-10-12T23:27:11Z")

</div>

I’m creating an AWS elasticsearch domain as a custom resource and I want to be able to pass the domain endpoint into my lambda functions.

Is there a way to access a property of a generated resource as a variable in serverless.yml?

I’m guessing not as the endpoint wouldn’t be generated until the CF deployment is done by which time I’d expect that the value couldn’t be made available to serverless-plugin-write-env-vars plugin that I’m using to setup the env vars for the lambda functions.

I’m hoping someone has come up with a way to do this…

---

<div class="post-metadata">

**Author:** ![andrew.beck](https://avatars.discourse-cdn.com/v4/letter/a/e99b99/32.png) [@andrew.beck](https://forum.serverless.com/u/andrew.beck)\
**Post date:** [October 12, 2016, 11:45pm UTC](https://forum.serverless.com/t/can-i-access-outputs-from-custom-resources-as-variables-in-serverless-yml/508/2 "2016-10-12T23:45:05Z")

</div>

I just signed up to ask exactly this. I’ve rolled my own deployment tools, which use a CF custom resource to grab my lambda’s zip from S3, inject a simple json file containing references to other CF resources and then replace the original zip. The lambda resource DependsOn this pre-processing step. Note that its possible to save these references as a separate S3 config file, but the latency is way better when its injected directly into he lambda.

I’d like to replicate this capability in serverless in order to complete the transition from my own tools to serverless. Its not clear to me if I can add the required DependsOn to the current function declaration, if I need to create a plugin or alternatively if there is some other technique which is clearly better. I do a similar thing to inject KMS encoded secrets.

---

<div class="post-metadata">

**Author:** ![rowanu](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.serverless.com/rowanu/32/75_2.png) [@rowanu](https://forum.serverless.com/u/rowanu)\
**Post date:** [October 13, 2016, 1:22am UTC](https://forum.serverless.com/t/can-i-access-outputs-from-custom-resources-as-variables-in-serverless-yml/508/3 "2016-10-13T01:22:58Z")

</div>

Yeah, this has been discussed many times in the the Gitter.

I don’t think it’s possible to refer to CloudFormation Outputs in `serverless.yml` due to the reason @viz mentioned - there’s a circular dependency; You can’t deploy your service without a defined `serverless.yml`, and you don’t know your outputs until you’ve deployed…

The way to deal with this would be to create two stacks/services, one with the dependent resources (exposed via CFN outputs) and the other one to refer to it using [cross stack references](https://aws.amazon.com/blogs/aws/aws-cloudformation-update-yaml-cross-stack-references-simplified-substitution/).

---

<div class="post-metadata">

**Author:** ![flomotlik](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.serverless.com/flomotlik/32/3_2.png) [@flomotlik](https://forum.serverless.com/u/flomotlik)\
**Post date:** [October 13, 2016, 3:09pm UTC](https://forum.serverless.com/t/can-i-access-outputs-from-custom-resources-as-variables-in-serverless-yml/508/4 "2016-10-13T15:09:29Z")

</div>

As @rowanu mentioned this is not possible yet, but a super important feature for us. To really make this work though we need some way for passing those variables to your lambda functions inside your Cloudformation template. Basically AWS needs to implement this feature so we can use it to expose that data to you (we can’t really do anything about that ourselves because we’d always have to make this a two step process because we don’t know the data ourselves beforehand)

Hopefully this is something we can provide soon, we’re definitely telling AWS that this is necessary.

---

<div class="post-metadata">

**Author:** ![andrew.beck](https://avatars.discourse-cdn.com/v4/letter/a/e99b99/32.png) [@andrew.beck](https://forum.serverless.com/u/andrew.beck)\
**Post date:** [October 13, 2016, 10:58pm UTC](https://forum.serverless.com/t/can-i-access-outputs-from-custom-resources-as-variables-in-serverless-yml/508/5 "2016-10-13T22:58:23Z")

</div>

@flomotlik I have a solution for this now. Basically I have a custom CloudFormation resource which grabs the zip file and injects a file containing the template references I need and then replaces the original zip in S3. The lambda function DependsOn this pre-processing step.

The code for my solution is below. In this case, I’m deploying “LambdaFunction” and I want to inject a reference to “ABucket” into that function. I’ve created a custom resource "LambdaSettings to specify the settings I want to inject. The other lambda, “SettingsFunction” simply grabs the zip from S3, injects the settings and replaces the original zip. The only prior knowledge required is the zip file location. Obviously I’ve omitted the permission and role resources etc. I inject a json file below, but a [settings.py](http://settings.py) would be better.

```
"SettingsFunction": {
    "Type" : "AWS::Lambda::Function",
    "Properties" : {
        "Code": {
            "ZipFile": {"Fn::Join": ["\n", [
                "import boto3",
                "import cfnresponse",
                "import io",
                "import json",
                "import zipfile",

                "def handler(event, context):",
                " bucket = event['ResourceProperties']['Bucket']",
                " key = event['ResourceProperties']['Key']",

                " client = boto3.client('s3')",

                " if event['RequestType'] == 'Delete':",
                " return cfnresponse.send(event, context, 'SUCCESS', {})",

                " buff = io.BytesIO()",
                " client.download_fileobj(bucket, key, buff)",

                " with zipfile.ZipFile(buff, 'a') as zip_file:",
                " info = zipfile.ZipInfo('settings.json')",
                " info.external_attr = 0777 << 16L",
                " zip_file.writestr(info, json.dumps(event['ResourceProperties']['Settings']))",
                " zip_file.close()",

                " buff.seek(0)",
                " client.upload_fileobj(buff, bucket, key)",

                " return cfnresponse.send(event, context, 'SUCCESS', {})"
            ]]}
        },
        "Handler": "index.handler",
        "Runtime": "python2.7",
        "Timeout": "300"
    },
},

"ABucket": {
    "Type": "AWS::S3::Bucket",
},

"LambdaSettings" : {
    "Type": "Custom::Settings",
    "Properties": {
        "ServiceToken" : {"Fn::Join": [":", [
            "arn:aws:lambda",
            {"Ref": "AWS::Region"},
            {"Ref": "AWS::AccountId"},
            "function",
            {"Ref": "SettingsFunction"}
        ]]},
        "Bucket": deploy_bucket,
        "Key": deploy_key,
        "Settings": {
            "ABucket": {"Ref": "ABucket"}
        }
    }
},

"LambdaFunction": {
    "Type" : "AWS::Lambda::Function",
    "DependsOn": {"Ref": "LambdaSettings"},
    "Propertes": {
        "Code": {
            'S3Bucket': deploy_bucket,
            'S3Key': deploy_key
        }
    }
}

```

I would like to extend serverless to do exactly this so I can ditch my custom deployment tools and switch to serverless. Its not clear to me if I need to extend serverless itself or if I can implement this feature via a plugin.

---

<div class="post-metadata">

**Author:** ![flomotlik](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.serverless.com/flomotlik/32/3_2.png) [@flomotlik](https://forum.serverless.com/u/flomotlik)\
**Post date:** [October 14, 2016, 9:40am UTC](https://forum.serverless.com/t/can-i-access-outputs-from-custom-resources-as-variables-in-serverless-yml/508/6 "2016-10-14T09:40:40Z")

</div>

@andrew.beck awesome stuff. Its certainly possible to implement this as a plugin (basically everything in Serverless is a plugin, all of our AWS deployment integrations for example are no different than any plugin you could write).

But we can’t merge this into master because we’re working on a solution with AWS that should make this much easier. We currently have to wait until AWS gets ready to release something in this direction though. This is why we’d love to see this as an option, but we can’t include it in Serverless directly.

If thats fine with you please go ahead and let us know when the plugin is ready, we’d be happy to include it in our README list until we have something else in place that makes this easier.

---

<div class="post-metadata">

**Author:** ![dom\_hutton](https://avatars.discourse-cdn.com/v4/letter/d/c0e974/32.png) [@dom\_hutton](https://forum.serverless.com/u/dom_hutton)\
**Post date:** [February 20, 2017, 6:49am UTC](https://forum.serverless.com/t/can-i-access-outputs-from-custom-resources-as-variables-in-serverless-yml/508/7 "2017-02-20T06:49:30Z")

</div>

For anyone looking for a way to access their queues you can use the AWS SDK in your function body to retrieve queues by QueueNamePrefix, given that you can define the queue name in the serverless.yml resources section this makes a workaround for this issue fairly straightforward.

```
 var params = {
  QueueNamePrefix: "slss-ms-orchestrator"
 };
 sqs.listQueues(params, function(err, data) {
   if (err) console.log(err, err.stack); // an error occurred
   else console.log(data); // successful response
   /*
   data = {
    QueueUrls: [
       "https://queue.amazonaws.com/80398EXAMPLE/SLSSMSOrchestratorQueue"
    ]
   }
   */
 });

resources:
  Resources:
    debounceStateVerifierQueue:
      Type: AWS::SQS::Queue
      Properties:
        QueueName: slss-ms-orchestrator-debounce-queue
        Tags:
          -
            Key: environment
            Value: multi-staging

```

In the meantime I hope AWS and the Serverless team continue to work together.

---

<div class="post-metadata">

**Author:** ![mputilin](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.serverless.com/mputilin/32/966_2.png) [@mputilin](https://forum.serverless.com/u/mputilin)\
**Post date:** [September 1, 2017, 10:27am UTC](https://forum.serverless.com/t/can-i-access-outputs-from-custom-resources-as-variables-in-serverless-yml/508/8 "2017-09-01T10:27:45Z")

</div>

Guys, have Amazon released native solution or I should use @andrew.beck’s one? @flomotlik may be you know?

---

<div class="post-metadata">

**Author:** ![kalinchernev](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.serverless.com/kalinchernev/32/1041_2.png) [@kalinchernev](https://forum.serverless.com/u/kalinchernev)\
**Post date:** [November 7, 2017, 1:19pm UTC](https://forum.serverless.com/t/can-i-access-outputs-from-custom-resources-as-variables-in-serverless-yml/508/9 "2017-11-07T13:19:52Z")

</div>

No, I don’t think so. I’ve also spent few hours digging into the question and this thread is one of the most helpful together with this [github discussion](https://github.com/serverless/serverless/issues/2240#issuecomment-268411199) in a similar topic.

The solution of @andrew.beck seems a bit hard to maintain, I’d personally go for splitting the service into 2 and try using cross stack reference as @rowanu has mentioned.

The original question of @viz is about elasticsearch, just like my case. Only the deployment of the smallest instance in a self-managed service way takes at least 10 minutes, which is a good-enough reason to split service for CF and another one for the functionalities already.

Update after success, hopefully this will be useful for someone else or my future-self.

Service 1, the elasticsearch one, creating a domain:

```auto
service: service1

custom:
  ES_DOMAIN: ${self:provider.stage}-projects

provider:
  name: aws
  runtime: nodejs6.10
  stage: ${opt:stage, file(config.json):stage, 'dev'}
  region: ${opt:region, file(config.json):region, 'eu-central-1'}

resources:
  Resources:
    ProjectsElasticSearchDomain:
      Type: AWS::Elasticsearch::Domain
      Properties:
        DomainName: ${self:custom.ES_DOMAIN}
        ElasticsearchVersion: 5.5
        EBSOptions:
          EBSEnabled: true
          VolumeType: gp2
          VolumeSize: 10
        ElasticsearchClusterConfig:
          InstanceType: t2.small.elasticsearch
          InstanceCount: 1
          DedicatedMasterEnabled: false
          ZoneAwarenessEnabled: false
        AccessPolicies:
          Version: '2012-10-17'
          Statement:
          # Public can query for information
          - Effect: Allow
            Principal: "*"
            Action:
             - "es:ESHttpHead"
             - "es:ESHttpGet"
            Resource: "arn:aws:es:${self:provider.region}:*:domain/${self:custom.ES_DOMAIN}/*"
          # Lambda can take actions on the ES domain
          - Effect: Allow
            Principal:
              Service: lambda.amazonaws.com
            Action: sts:AssumeRole
          # Admin access to Kibana from an IP
          # https://goo.gl/eiGgpD
          - Effect: Allow
            Principal: "*"
            Action: es:*
            Condition:
              IpAddress:
                aws:SourceIp:
                - THE_IP
            Resource: "arn:aws:es:${self:provider.region}:*:domain/${self:custom.ES_DOMAIN}/*"

  Outputs:
    ServiceEndpoint:
      Description: The API endpoint of the projects' elasticsearch domain.
      Value:
        Fn::GetAtt: ["ProjectsElasticSearchDomain", "DomainEndpoint"]
      Export:
        Name: "${self:provider.stage}:${self:service}:ServiceEndpoint"

```

The `Outputs` are important as the `serverless` framework and `serverless-stack-output` will not give you the endpoint after successful deployment, [at least at the moment](https://github.com/sbstjn/serverless-stack-output/issues/6).

Then, in Service 2, where the endpiont is to be feeded in automatically:

```auto
service: service2

custom:
  index: projects

provider:
  name: aws
  runtime: nodejs6.10
  stage: ${opt:stage, file(config.json):stage, 'dev'}
  region: ${opt:region, file(config.json):region, 'eu-central-1'}
  iamRoleStatements:
    # https://goo.gl/U21zxP
    - Effect: "Allow"
      Action: "es:*"
      Resource: "arn:aws:es:${self:provider.region}:*:domain/*"

functions:
  onObjectCreated:
    handler: src/events/onObjectCreated.handler
    name: ${self:provider.stage}-${self:service}-onObjectCreated
    memorySize: 256
    environment:
      API:
        Fn::ImportValue: ${self:provider.stage}:elasticsearch:ServiceEndpoint
      INDEX: ${self:custom.index}
    events:
      - SOME_EVENT: http, sns, etc.

```

In the code of the function then, it’s safe to pull the necessary API information

```javascript
const { API, INDEX } = process.env;

```

---

<div class="post-metadata">

**Author:** ![tommedema](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.serverless.com/tommedema/32/1257_2.png) [@tommedema](https://forum.serverless.com/u/tommedema)\
**Post date:** [November 18, 2017, 9:26am UTC](https://forum.serverless.com/t/can-i-access-outputs-from-custom-resources-as-variables-in-serverless-yml/508/10 "2017-11-18T09:26:31Z")

</div>

@kalinchernev how do you orchestrate this such that service 1 is deployed prior to service 2 on a `sls deploy`? or do you use terraform etc?

personally I need to create a custom cloudformation resource backed by a lambda, where the lambda has to be pulled from an AWS s3 bucket as per the AWS docs. Of course I don’t want to manually create a s3 bucket and upload the lambda code – this has to be automated. I’m looking into creating a plugin to do this, but I have no idea how to have my serverless.yml reference to the s3 bucket created by the plugin.

---

<div class="post-metadata">

**Author:** ![kalinchernev](https://yyz2.discourse-cdn.com/flex036/user_avatar/forum.serverless.com/kalinchernev/32/1041_2.png) [@kalinchernev](https://forum.serverless.com/u/kalinchernev)\
**Post date:** [November 18, 2017, 9:56am UTC](https://forum.serverless.com/t/can-i-access-outputs-from-custom-resources-as-variables-in-serverless-yml/508/11 "2017-11-18T09:56:50Z")

</div>

For the moment, a shell script

```auto
#!/bin/sh

# Exit the script on any command with non 0 return code
set -ex

# Go to project root
cd "$(dirname "$0")"
cd ..

# Deploy first service
cd ./services/one
./node_modules/.bin/serverless deploy -v

# Deploy second service
cd ./services/two
./node_modules/.bin/serverless deploy -v

...

```

It might be that there are better ways I don’t know of, but that’s the current very basic orchestration, in which [serverless-stack-output](https://github.com/sbstjn/serverless-stack-output) is configured in each service so that services can rely on the endpoint provided by the previous one.