RDS access, policy roles serverless.yml setup

Serverless Framework
1

Hi,

I cannot get my Lambda functions to access my public RDS Postgresql database. I can connect locally and via tests. I was using a IAM User which had AdministratorAccess.

I need a clean way to configure this moving forward. How would I set this up in my .yml file:

iamRoleStatements:
    -  Effect: "Allow"
       Action: 
         - "ec2:CreateNetworkInterface"
         - "ec2:DescribeNetworkInterfaces"
         - "ec2:DeleteNetworkInterface"
       Resource: "*"     
    -  Effect: "Allow"
       Action:
         - "s3:ListBucket"
       Resource:
         Fn::Join:
           - ""
           - - "arn:aws:s3:::"
             - Ref: ServerlessDeploymentBucket
    -  Effect: "Allow"
       Action:
         - "s3:PutObject"
       Resource:
         Fn::Join:
           - ""
           - - "arn:aws:s3:::"
             - Ref: ServerlessDeploymentBucket
             - "/*"

Does this look right? Are the EC2 actions correct?

2

not working still, trying to burrow into the cloudwatch logs

3

This is my current serverless.yml:

provider:
  name: aws
  runtime: nodejs4.3
  profile: serverless-bankstatement-test
  timeout: 30
  iamRoleStatements:
    -  Effect: "Allow"
       Action: 
         - "rds:*"
         - "cloudwatch:DescribeAlarms"
         - "cloudwatch:GetMetricStatistics"
         - "ec2:CreateNetworkInterface"
         - "ec2:DescribeNetworkInterfaces"
         - "ec2:DeleteNetworkInterface"
         - "ec2:DescribeAccountAttributes"
         - "ec2:DescribeAvailabilityZones"
         - "ec2:DescribeSecurityGroups"
         - "ec2:DescribeSubnets"
         - "ec2:DescribeVpcs"
         - "sns:ListSubscriptions"
         - "sns:ListTopics"
         - "logs:DescribeLogStreams"
         - "logs:GetLogEvents"
       Resource: "*"     
    -  Effect: "Allow"
       Action:
         - "s3:ListBucket"
       Resource:
         Fn::Join:
           - ""
           - - "arn:aws:s3:::"
             - Ref: ServerlessDeploymentBucket
    -  Effect: "Allow"
       Action:
         - "s3:PutObject"
       Resource:
         Fn::Join:
           - ""
           - - "arn:aws:s3:::"
             - Ref: ServerlessDeploymentBucket
             - "/*"

Not working, but I noticed my RDS is available in ap-southeast-2b whilst my lambda functions are deployed to US East (N. Virginia) . Could this be an issue?

4

Your RDS database and Lambda function are required to be within the same VPC. The instruction on the AWS side on how to do so is here and how to specify VPC details in the Serverless framework is here.

Hope that helps

2 Likes
5

The second url is the same mate. Thanks

6

Thanks. Fixed. Was in a hurry as I was busy heading out as I posted that.

1 Like
7

Thanks again. I created a new IAM Role, which before I was adding the policies to the existing IAM User which was created for the serverless deployment:

This is the provider part of my serverless.yml:

provider:
  name: aws
  runtime: nodejs4.3
  profile: serverless-deploy
  role: arn:aws:iam::{account}:role/{role-name}
  timeout: 30
  vpc:
      securityGroupIds:
        - {my SG id, same for the RDS VPC}
      subnetIds:
        - {VPC subnet 1, same for the RDS VPC}
        - {VPC subnet 2, same for the RDS VPC}
  iamRoleStatements:
    -  Effect: "Allow"
       Action: 
         - "ec2:CreateNetworkInterface"
         - "ec2:DescribeNetworkInterfaces"
         - "ec2:DeleteNetworkInterface"
         - "logs:CreateLogGroup"
         - "logs:CreateLogStream"
         - "logs:PutLogEvents"
       Resource: "*"     
    -  Effect: "Allow"
       Action:
         - "s3:ListBucket"
       Resource:
         Fn::Join:
           - ""
           - - "arn:aws:s3:::"
             - Ref: ServerlessDeploymentBucket
    -  Effect: "Allow"
       Action:
         - "s3:PutObject"
       Resource:
         Fn::Join:
           - ""
           - - "arn:aws:s3:::"
             - Ref: ServerlessDeploymentBucket
             - "/*"

This doesn’t work. I am concerned that the profile and role have conflicts.

UPDATE: This works it was the IP of the security group on AWS that needed to be set for internal access.

8

I also needed to set my Security Group Postgres TCP IP to accept calls from IP
0.0.0.0/0 which is all internal calls basically.

Thanks for the help

9

Hmm.

If I create a database on aws. Should it then be possible for me to deploy a nodejs function (lambda) and with configuration in serverless.yml give it access to my database?

If I deploy my function and then go to my database and allow everyone to connect to it then it works. But is that nesseserry?

By reading here I have read I should configure this

vpc:
securityGroupIds:
- sg-9999999
subnetIds:
- subnet-9999999
- subnet-9999999
- subnet-9999999

The security group is from my mysql instance details page. And so are the subnets. Is that correct? It still doesnt work unless I allow access to my db from all ips.

10

Ok. Just learned a little more. The vpc securityGroup I add for my functions serverless.yml is the group which will be added to that function. And then what I need to do before that is to allow that group access to that vpc I use.

Is that the same way you guys do it? or is there a smarter way :slight_smile:

11

RDS + Lambda is complicated. It feels like you need to chose between security or fast cold start times. If your Lambda is inside the VPC your database is secure but your Lamdba cold start time rises from 3s to 13s. Deploy Lambda outside the VPC and you need to open your database to the world.

1 Like
12

Thx for replying :grinning:. Can you tell me how I decide if my lambda runs on the vpc or “normal”. Is that by giving the lambdas security group access to the vpc resource?

13

Ultimately I decided that Lambda + RDS was a bad idea for a web app and stayed with DynamoDB.

If I had to connect to RDS from Lambda then I would run the Lambda inside the VPC for security reasons.

14

Ill take a look at DynamoDB. Im starting up a new project and still havent chosen the technology stack.

15

Do you code with nodejs? whcih library do you use? On this page there are so many to choose from https://aws.amazon.com/dynamodb/community/

16

I’ve been using http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/DynamoDB/DocumentClient.html

1 Like
17

Cool thx. The last days I have ben googling a bit about dynamodb it seems a few of the week points are transaction handling and backup of the database. Is that something you also experience as a problem?

18

Ok looks like Im to eager to ask questions. Just found out there is a batch put and batch delete. So I guess that solves the transactional issue.

19

Backup is fine. There are no transactions. The eventual consistency can also be a problem if you don’t design your app to deal with it.

1 Like