Aurora Serverless Cluster

Hello, I’ve been searching around the forums in hopes someone had an answer to my question but basically I’ve noticed that at the moment, there doesn’t seem to be a means of deploying an Aurora Serverless Cluster through Serverless Components at the moment. I’ve sifted through all of the posted npm modules and I’m wondering if anyone knows when this will happen or if there’s some type of workaround with regards to this? Preferably; I would like to use Serverless Components for this thanks to its Open-Source Licensing as opposed to Amazon’s CloudFormation.

I wonder if that is because Aurora RDS would look like MySQL or PostgreSQL to npm? I’ve used node12.x/RDS/MySql done manually and used the mysql npm modules vs anything specific for RDS/Aurora.

Although you do have different end points for ReadOnly vs the Write.


Hi Nytingale,

I’ve managed to get an Aurora Serverless Cluster to deploy with CloudFormation (not Components). However, I should note there are some caveats and prerequisites.

One thing to note is that Aurora Serverless Clusters exist in a VPC and as far as I recall, cannot exist outside of one.

This means that in order to connect to the cluster, your serverless functions would also have to deploy in the same VPC with an Internet Gateway etc… This results in extraordinarily execution times and for a production service is not really fit for purpose, at least in my opinion, particularly when coupled with the cold start of the cluster and the lambda function it can easily be in excess of 10 seconds.

This can be solved using the RDSDataService which allows SQL commands to be executed on the Aurora RDS using HTTP requests authenticated with a Secret ARN. This removes the requirement for the lambda being deployed in the Cluster VPC.

Also, as serverless infrastructure itself, the RDS Cluster does have cold starts which in my experience will exceed the default APIGateway timeout when you attempt to connect from one of your serverless functions and results in a 500 error. This can be confusing at first so be sure to manage the timeouts on requests to the RDS.

Right, on to the actual explanation of how to get an Aurora Serverless Cluster deployed. You’re going to need a few things.

I’ve added a link to each of the corresponding template documentation pages. Please review this and adjust the templates accordingly, the examples below do not include delete protection etc…

VPC and Subnets

VPC & Subnets CloudFormation Example
  Type: AWS::EC2::VPC

    EnableDnsSupport: true
    EnableDnsHostnames: true
    InstanceTenancy: default

  Type: AWS::EC2::Subnet

    AvailabilityZone: ${ self:provider.region }a
    MapPublicIpOnLaunch: true
      Ref: VPC

  Type: AWS::EC2::Subnet

    AvailabilityZone: ${ self:provider.region }b
    MapPublicIpOnLaunch: true
      Ref: VPC

  Type: AWS::EC2::Subnet

    AvailabilityZone: ${ self:provider.region }c
    MapPublicIpOnLaunch: true
      Ref: VPC

Security Group

Security Group CloudFormation Example
  Type: AWS::EC2::SecurityGroup

    GroupName: ${ self:service }-security-group

      - IpProtocol: tcp
        FromPort: 3306
        ToPort: 3306

      Ref: VPC

RDS Subnet Group

RDS Subnet Group CloudFormation Example
  Type: AWS::RDS::DBSubnetGroup

      - Ref: SubnetA
      - Ref: SubnetB
      - Ref: SubnetC

Secret Manager Secret

Secret Manager Secret CloudFormation Example
  Type: AWS::SecretsManager::Secret

    Name: ${ self:service }-aurora-secret
    Description: ${ self:service } serverless aurora secret

      SecretStringTemplate: !Join ['', ['{ "username": "', '${ self:provider.environment.DATABASE_USERNAME }', '" }']]
      GenerateStringKey: 'password'
      PasswordLength: 30
      ExcludeCharacters: '"@/\'

Serverless Aurora Cluster

Serverless Aurora Cluster CloudFormation Example
  Type: AWS::RDS::DBCluster

    MasterUsername: !Join ['', ['{{resolve:secretsmanager:', !Ref AuroraSecret, ':SecretString:username}}' ]]
    MasterUserPassword: !Join ['', ['{{resolve:secretsmanager:', !Ref AuroraSecret, ':SecretString:password}}' ]]

    DatabaseName: ${ self:provider.environment.DATABASE_NAME }

    Engine: aurora-postgresql
    EngineMode: serverless

      Ref: RDSSubnetGroup

      - Fn::GetAtt: SecurityGroup.GroupId

    EnableHttpEndpoint: true # Enable the Data API

  Type: AWS::SecretsManager::SecretTargetAttachment

    SecretId: !Ref AuroraSecret
    TargetId: !Ref AuroraServerlessPostgresCluster
    TargetType: AWS::RDS::DBCluster

For the purposes of this example, I’m assuming that the above templates are defined in separate yaml files.

With that, in your serverless.yml they would need to be included in resources section.

  - ${ file(infrastructure/vpc/vpc.yml) }
  - ${ file(infrastructure/aurora/subnet-group.yml) }
  - ${ file(infrastructure/aurora/security-group.yml) }
  - ${ file(infrastructure/aurora/secret.yml) }
  - ${ file(infrastructure/aurora/serverless-cluster.yml) }

You will also need to add the relevant IAMRoleStatements for the RDS Data API and the Secrets Manager.

The AWS SDK includes an RDSDataService that can be used to connect to and excute commands to your RDS Instance.

Here’s the Javascript RDSDataService SDK for reference.

I hope this was helpful, it took me a while to initially figure out.

1 Like

Thankfully enough the really long wait times is no longer an issue with Lambda as AWS has made some changes under the hood to correct that issue as of November last year if I recall.

I wasn’t aware of this, that’s awesome!

You can see the massive difference that this has made in function execution duration, with it dropping to 933 ms from 14.8 seconds!