Lambda logs -> Kinesis via CloudWatch Logs subscription

Serverless Framework
1

Good morning - has anyone setup Cloudwatch Logs from Lambda to stream to Kinesis via subscriptions? http://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/SubscriptionFilters.html – I’m at a bit of a loss on how to translate this to my serverless.yml
especially the perms - which perms do I associate with the iamRole I define, which have to be defined on the user that I execute sls with?
and then the logs subscription -> kinesis, I’m assuming there isn’t a primitive in the DSL for that at this point, so I probably need to define that as a custom CF resource?
I assume it’s basically the same premise to send Lambda logs to AWS ElasticSearch (ES), but I’ve never done that, either - what are folks doing for this?

Here’s what I have so far to define this w/ custom resources:

KinesisLogsStream:
  Type: AWS::Kinesis::Stream
  Properties:
    Name: my-function-lambda-logs-stream-${opt:stage}
    ShardCount: 1
CWLtoKinesisRole:
  Type: AWS::IAM::Role
  Properties:
    RoleName: CWLtoKinesisRole-${opt:stage}
    AssumeRolePolicyDocument:
      Version: '2012-10-17'
      Statement:
        - Effect: Allow
          Principal:
            Service: logs.us-east-1.amazonaws.com
          Action: sts:AssumeRole
PermissionsPolicyForCWL:
  Type: AWS::IAM::Policy
  DependsOn:
    - CWLtoKinesisRole
    - KinesisLogsStream
  Properties:
    PolicyName: PermissionsPolicyForCWL-${opt:stage}
    Roles:
      - CWLtoKinesisRole
    PolicyDocument:
      Version: '2012-10-17'
      Statement:
        - Effect: Allow
          Action: kinesis:PutRecord
          Resource:
            Fn::GetAtt:
              - KinesisLogsStream
              - Arn
        - Effect: Allow
          Action: iam:PassRole
          Resource:
            Fn::GetAtt:
              - CWLtoKinesisRole
              - Arn
CWLtoKinesisSubscription:
  Type: AWS::Logs::SubscriptionFilter
  DependsOn:
    - CWLtoKinesisRole
    - PermissionsPolicyForCWL
    - KinesisLogsStream
  Properties:
    DestinationArn:
      Fn::GetAtt:
        - KinesisLogsStream
        - Arn
    RoleArn:
      Fn::GetAtt:
        - CWLtoKinesisRole
        - Arn
    FilterPattern: "{$.userIdentity.type = Root}"
    LogGroupName: CloudTrail
2

Here’s what I ended up with (some of the previous values were miscues from the AWS docs I was following):

KinesisLogsStream:
  Type: AWS::Kinesis::Stream
  Properties:
    Name: my-example-lambda-logs-stream-${opt:stage}
    ShardCount: 1
CWLtoKinesisRole:
  Type: AWS::IAM::Role
  Properties:
    RoleName: CWLtoKinesisRole-${opt:stage}
    AssumeRolePolicyDocument:
      Version: '2012-10-17'
      Statement:
        - Effect: Allow
          Principal:
            Service: logs.us-east-1.amazonaws.com
          Action: sts:AssumeRole
PermissionsPolicyForCWL:
  Type: AWS::IAM::Policy
  DependsOn:
    - CWLtoKinesisRole
    - KinesisLogsStream
  Properties:
    PolicyName: PermissionsPolicyForCWL-${opt:stage}
    Roles:
      - Ref: CWLtoKinesisRole
    PolicyDocument:
      Version: '2012-10-17'
      Statement:
        - Effect: Allow
          Action: kinesis:PutRecord
          Resource:
            Fn::GetAtt:
              - KinesisLogsStream
              - Arn
        - Effect: Allow
          Action: iam:PassRole
          Resource:
            Fn::GetAtt:
              - CWLtoKinesisRole
              - Arn
CWLtoKinesisSubscription:
  Type: AWS::Logs::SubscriptionFilter
  DependsOn:
    - CWLtoKinesisRole
    - PermissionsPolicyForCWL
    - KinesisLogsStream
  Properties:
    DestinationArn:
      Fn::GetAtt:
        - KinesisLogsStream
        - Arn
    RoleArn:
      Fn::GetAtt:
        - CWLtoKinesisRole
        - Arn
    FilterPattern: ""
    LogGroupName: "/aws/lambda/my-example-log-group-dev"
1 Like