Connection to RDS fails from outside VPC

Serverless Framework
1

Hi. New-ish to AWS configuration.

Edited: The lambda’s inside the VPC also time out and do not return the database results. I don’t know how to fix my setup as I’m unfamiliar with the concepts here.

I can connect to my RDS Postgres instance when running it from Lambda. I would like this database to be accessible outside of the VPC as well, so I can connect to it with an external client (like PSQL). I’m not sure how to set this up using the Cloudformation config.

Can anyone help me out with this?

The serverless.yml file:

provider:
  name: aws
  runtime: nodejs6.10

# you can overwrite defaults here
  stage: ${self:custom.currentStage}
  region: ap-southeast-2
  environment: 
    DATABASE_URL:
      Fn::Join: [":", [Fn::GetAtt: [ServerlessRDSCluster, Endpoint.Address], Fn::GetAtt: [ServerlessRDSCluster, Endpoint.Port]]]
# you can add statements to the Lambda function's IAM Role here
  iamRoleStatements:
  - Effect: "Allow"
    Action:
      - "ec2:CreateNetworkInterface"
      - "ec2:DescribeNetworkInterfaces"
      - "ec2:DeleteNetworkInterface"
    Resource: "*"

functions:
  authWithKey: 
    handler: handler.authWithKey
    vpc:
      securityGroupIds:
        - "Fn::GetAtt": ServerlessSecurityGroup.GroupId
      subnetIds:
        - Ref: ServerlessSubnetA
        - Ref: ServerlessSubnetB
        - Ref: ServerlessSubnetC

  apiKeyEndpoint:
    handler: handler.apiKeyEndpoint
    vpc:
      securityGroupIds:
        - "Fn::GetAtt": ServerlessSecurityGroup.GroupId
      subnetIds:
        - Ref: ServerlessSubnetA
        - Ref: ServerlessSubnetB
        - Ref: ServerlessSubnetC
    events:
      - http:
          path: apiKey
          method: get
          integration: lambda
          private: true
          request:
            parameters:
              headers:
                'X-Api-Key': true
          authorizer: 
            name: authWithKey
            identitySource: method.request.header.X-Api-Key
          cors:
            origins:
              - '*'
            headers:
              - Content-Type
              - X-Amz-Date
              - Authorization
              - X-Api-Key
              - X-Amz-Security-Token

# cloud formation stuff
resources:
  Resources:
    ServerlessVPC:
      Type: AWS::EC2::VPC
      Properties:
        CidrBlock: "10.0.0.0/16"
    ServerlessSubnetA:
      DependsOn: ServerlessVPC
      Type: AWS::EC2::Subnet
      Properties:
        VpcId:
          Ref: ServerlessVPC
        AvailabilityZone: ${self:provider.region}a
        CidrBlock: "10.0.0.0/24"
    ServerlessSubnetB:
      DependsOn: ServerlessVPC
      Type: AWS::EC2::Subnet
      Properties:
        VpcId:
          Ref: ServerlessVPC
        AvailabilityZone: ${self:provider.region}b
        CidrBlock: "10.0.1.0/24"
    ServerlessSubnetC:
      DependsOn: ServerlessVPC
      Type: AWS::EC2::Subnet
      Properties:
        VpcId:
          Ref: ServerlessVPC
        AvailabilityZone: ${self:provider.region}c
        CidrBlock: "10.0.2.0/24"
    ServerlessSecurityGroup:
      DependsOn: ServerlessVPC
      Type: AWS::EC2::SecurityGroup
      Properties:
        GroupDescription: SecurityGroup for Serverless Functions
        VpcId:
          Ref: ServerlessVPC
    ServerlessStorageSecurityGroup:
      DependsOn: ServerlessVPC
      Type: AWS::EC2::SecurityGroup
      Properties:
        GroupDescription: Ingress for RDS Instance
        VpcId:
          Ref: ServerlessVPC
        SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: '5432'
          ToPort: '5432'
          SourceSecurityGroupId:
            Ref: ServerlessSecurityGroup
        - IpProtocol: tcp
          FromPort: '11211'
          ToPort: '11211'
          SourceSecurityGroupId:
            Ref: ServerlessSecurityGroup
    ServerlessRDSSubnetGroup:
      Type: AWS::RDS::DBSubnetGroup
      Properties:
        DBSubnetGroupDescription: "RDS Subnet Group"
        SubnetIds:
        - Ref: ServerlessSubnetA
        - Ref: ServerlessSubnetB
        - Ref: ServerlessSubnetC
    ServerlessRDSCluster: 
      DependsOn: ServerlessStorageSecurityGroup
      Type: "AWS::RDS::DBInstance"
      Properties: 
        DBName: "telematics${self:custom.currentStage}"
        AllocatedStorage: 10
        DBInstanceClass: "db.t2.micro"
        Engine: "postgres"
        EngineVersion: "9.6.2"
        MasterUsername: "<user>"
        MasterUserPassword: "<pass>"
        VPCSecurityGroups:
        - "Fn::GetAtt": ServerlessStorageSecurityGroup.GroupId
        DBSubnetGroupName:
          Ref: ServerlessRDSSubnetGroup
      DeletionPolicy: "Snapshot"
2

It may be beyond what you want to setup in your serverless CloudFormation template, but the way we solve this problem is to have a small EC2 instance in the VPC called a bastion host. This enables us to establish ssh tunnels to the JDBC endpoint of the database (RDS or Redshift) in the VPC. Then we can connect psql or SQL dashboards running on our laptops to the database. You might check out this QuickStart for creating such a box: https://aws.amazon.com/quickstart/architecture/linux-bastion/

3

Sorry no, I the functions I have that need to VPC to connect to postgres also don’t reach out - they time out even though I get results from my database queries.

I’m not sure how to get this to connect, there’s talk of NAT’s and allowing traffic from 0.0.0.0/0 but I’m clueless how I go about adding that to the CloudFormation template.

Advice would be appreciated.

4

To make a db reachable on the public Internet you would need to make the subnet it is in public (by adding an internet gateway) and then adding the rds resource flags to make your db publicly accessible.

This of course is not very secure, so you would minimally want to add ip whitelisted security groups attached to the db.

Alternatively, you can do the bastion host setup as mentioned above, put that into a public subnet and add security group rules that allow communication between the host and the db. Also, make sure that there are no subnet nacls that block traffic, by default there aren’t any.

If you do the bastion host, you can then do a forwarded port on your system from the db port to the host forwarding it to the db, then use your pg client to connect to localhost which will forward that to the db.

NAT is only for outbound access of hosts in a private subnet so they can make outbound calls, but they don’t allow in traffic.

Bastion host is the preferred, open db is a bad idea.

5

I’m dealing with a similar topic, so these are actually ideas, not proven examples.

  1. Assign the RDS instance to the default VPC.
    There is a note within the console which makes me thing this may work. In the console you can select the default security group for an RDS instance. Don’t know how to try this in CloudFormation.

All AWS Lambda functions run securely inside a default system-managed VPC.

  1. ClassicLink
    I’ve only read about this, but ClassicLink will open a VPC within the same region by private IPs. Can this be done without EC2… no idea :slight_smile:

ClassicLink removes the need to make use of public IPv4 addresses or Elastic IP addresses to enable communication between instances in these platforms.

Please share if you find a solution!

6

Okay, I just re-read you question. I think you want to connect to your DB from your desktop?

I believe you need configure security group inbound rules to whitelist your IP.

7

Hey Sime,

Yes, it was about migrating an existing database, which ideally is done through psql. I’ll check whitelisting an IP on the security group.

8

Whitelisted security groups will do nothing if the DB is not publicly accessible and inside of a public subnet. There is a specific flag for PubliclyAccessible: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-rds-database-instance.html#cfn-rds-dbinstance-publiclyaccessible

See my answer above.