How to set up a lambda function for dynamic cloudwatch event rules

Serverless Framework
1

I have code that dynamically creates scheduled CloudWatch Event rules, which are supposed to invoke a lambda function. This means that there is no static rule that is set up to invoke the lambda, instead rules are created and removed as needed.

When setting up this manually through the AWS console I have no issues - the lambda function is displayed as without any trigger, but implicitly the console created a resource-based function policy for my lambda function with lambda:InvokeFunction from the source principal events.amazonaws.com .

However, when setting up this through Serverless, I first tried leaving the event: parameter empty for my function. CloudWatch Event rules are created fine, with the lambda function as target seemingly looking OK, but they will silently fail to be invoked due to the function not having any resource-based policy matching the event.

Then I tried creating adding a cloudWatchEvent event to the function event parameter, but that ended up creating a static CloudWatch Event rule and attaching a resource-based policy for it to my lambda function. The policy has a Condition: ArnLike: AwsSourceArn: . This doesn’t help me as I want my function to be invokable by my dynamically created rules.

How can I set serverless up to create my lambda function, don’t set any trigger for it but give it a resource-based policy so that it can be triggered by CloudWatch Event rules that doesn’t exist at deploy time (any events:* would be fine)? Is there any way to explicitly define a resource-based function policy, or can it only be defined implicitly through the list of events for a function?

Or do I have to do this in two steps, so that my function is always triggered by one static event, which in turn can be triggered by the dynamically added events instead of them directly triggering the function?

1 Like
2

FWIW I ended up skipping the event, and instead dynamically calling lambda.addPermission in runtime, to add resource-based permissions to the lambda function I’m attaching to dynamically created CloudWatch event rules.
Would still be interesting to know if it’s possible to solve it all in sls.

1 Like
3

I’m trying to figure out this exact issue right now. I was hoping the resource-based policy could specify an arn prefix with a wildcard for the SourceArn but it doesn’t seem to be working. I’m also not sure if I need to pass the Lambda’s IAM role as RoleArn when creating the event rules or if that should be left blank… when I set it, I get an error about iam:PassRole and I’m not sure if that should be necessary…

I don’t love the idea of calling addPermission but if it worked for you I might go that route, although I don’t know if that will even scale, I think there is a very low limit on the number of those policies that can be added to a single resource… in my case, I might need up to the max of 100 event rules pointing to my single Lambda.

4

Sorry for not responding earlier, but to follow this up:

  1. I don’t pass the Lambda’s IAM role as RoleArn
  2. I do use wildcards for the SourceArn, so the policy will not grow over time.

Essentially I do this:

const functionArn = <my-scheduled-function-arn>;
const ruleName = <rule-name>;

// Add CW rule
const {RuleArn} = await cwe.putRule({
  Name: ruleName,
  ScheduleExpression: '<cron expression>',
}).promise();

const [ruleArnPrefix] = RuleArn.split('/');
try {
  // Allow function to be invoked by CW event
  await lambda.addPermission({
    Action: 'lambda:invokeFunction',
    FunctionName: functionArn, // note that ARN is acceptable for FunctionName
    StatementId: 'scheduled-invocation',
    Principal: 'events.amazonaws.com',
    SourceArn: `${ruleArnPrefix}*` // Wildcard allowing any CW rule to invoke this function
  }).promise();
} catch (err) {
  // Statement already added, ignore
}

// Tell CW rule to invoke function
await cwe.putTargets({
  Rule: ruleName,
  Targets: [
    {
      Id: '1',
      Arn: functionArn,
      Input: JSON.stringify({
        ruleName,
        data: { ... }
      })
    }
  ]
}).promise();

Then in the invoked function, I use the passed ruleName to remove the rule since it’s now obsolete.

This way, the function will never have more than one policy related to CW rules, but I try-and-silently-fail to add that policy every time. Of course that policy could have been set up statically at deploy time by invoking a script from SLS.

Did you end up doing something similar or did you find a better approach?

5

Even longer overdue, but the reason my code above adds the lambda permission in runtime is that I couldn’t find how to set a resource policy on a lambda function statically. Since then I’ve found how that can be done, so the code above can be simplified into:

serverless.yml

provider:
  [...]
  iamRoleStatements:
    - Effect: Allow
      Action:
        - events:putRule
        - events:putTargets
        - events:deleteRule
        - events:removeTargets
[...]
functions:
  MyScheduledFunction:
    [...]
    # Note no events at all, this will be triggered by CW rules created in runtime

resources:
  Resources:
    MyScheduledFunctionResourcePolicy:
      Type: 'AWS::Lambda::Permission'
      Properties:
        FunctionName:
          Fn::GetAtt: [MyScheduledFunctionLambdaFunction, Arn]
       Action: 'lambda:invokeFunction'
       Principal: 'events.amazonaws.com'
       SourceArn:
         Fn::Sub: 'arn:aws:events:${AWS::Region}:${AWS::AccountId}:rule/*'

scheduler.js

const functionArn = <my-scheduled-function-arn>;
const ruleName = <rule-name>;

// Add CW rule
await cwe.putRule({
  Name: ruleName,
  ScheduleExpression: '<cron expression>',
}).promise();

// Tell CW rule to invoke function
await cwe.putTargets({
  Rule: ruleName,
  Targets: [
    {
      Id: '<my-target-id>',
      Arn: functionArn,
      Input: JSON.stringify({
        rule: {name: ruleName},
        data: { ... }
      })
    }
  ]
}).promise();

To avoid adding lots and lots of CW rules, the invoked scheduled function may clean up by removing the rule that triggered it:

my-scheduled-function.js

exports.handler = async event => {
  const ruleName = event.rule.name;

  try {
    [...]
  } finally {
    await cwe.removeTargets({
      Rule: ruleName,
      Ids: ['<my-target-id>']
    }).promise();

    await cwe.deleteRule({
      Name: ruleName
    }).promise();
  }
}
2 Likes
6

You probably saved me at least few hours of banging my head against the keyboard…