Serverless Octopus: How to orchestrate many repos and environments into a singular deployable product?

Our awesome team has split up a rails monolith into many serverless microservices.

Each microservice has it’s own repo, usually with a client component and a databasey-crud component. The components/microservices can be mixed and match on the client side to do very cool things indeed.

This is really great for development and deployment, as serverless lets us separate our concerns and deploy in isolated environments, and it’s really very awesome and fun.

The problem is, all these microservices and clients are accessed via cloud front, which is manually configured, to map to the api gateways of each microservice.

That means to create a whole new product environment (for example, feature testing or the like), it requires basically a full day of poking around the aws console to carefully map all these new deployments (feature-x-new-test-env-2, feature-y-new-test-env-2, …) to the new cloudfront behaviors and such.

Does anyone know the best practices to get around this monotony?

Ideally I would like a singular way to configure and deploy all the repos of microservices in one swell foop, already assigned their place in a new tangle of freshly minted cloud front behaviors, alive and raring to go, inside their new cloudfront distribution.

This way we could create new environments and tear them down when we need to, for example, show a 3rd party stakeholder some new progress or the like.

Inspiration please :slight_smile:

If you can create a CloudFormation template to do all that provisioning and configuration, you could add that as a resource in one of your serverless.yml files…

1 Like

I think there are three ways to deal with it.

Cloudfront is just another layer , you can:

  1. in serverless framework

add cloudfront resources in serverless.yml directly, they use cloudfornation yaml template and syntax.

you can references api gateway resources to cloudfront

  1. write a seperate cloudformation

output serverless api gateway resource in serverless.yml
deal with cloudfront deployment, reference api gateway resources from cloudformation outputs variables

  1. write a seperate terraform stack to manage it.

you still need set outputs from serverless, so terraform can reference them

maybe this is the simplest way, but depand on your team skillsets on master terraform.

Please note, cloudfront deployment takes long time, normaly about 15~20 minutes, I recommend split api gateway and cloudfront deployment in two layers

2 Likes

These are both great responses, thanks for the help. @bill @Java1Guy

I don’t think I can allow one serverless.yml to coordinate the deployment because there are several repos each with their own serverless deployments each running on CI. Maybe I should use a different CI strategy, instead of at the individual repo level. Any ideas?

There is no limits for you. My current serverless project has serveral serverless stacks, you just need define the outputs, so others can reference them easily.