CICD from Github to API Gateway using CodePipeline (or simpler CICD)

Serverless Framework
, , ,
1

Hello,

I’m currently trying to configure a CodePipeline that deploy a serverless application (lambda + API Gateway using CloudFormation) but i’m really struggling in the process. The hook on Github is working, the build seems to work (i used the buildspec.yml described here ) but the deploy is not working throwing many different issues (like logGroup already exists or role invalid).

When I tried to deploy again using serverless deploy it wasn’t working anymore so I had to delete everything, and it was still throwing errors. I have changed the name of the project so it works again.

Now I would like to know if anyone figured out a simple way to have a CICD (using codepipeline or else) to deploy from Github to Lambda + API Gateway ?

Thanks in advance !

2

Looks like service name conflict, have u tried changing the stage name by environment name or changing the service name by environment name.

3

Thanks for answering.

We actually tried many things but everything was broken after all. So we’ve changed the name of the project and re-deploy it using serverless deploy command.

What we would like is to have some tutorial and/or template of serverless.yml and maybe buildspec.yml to know the full process to use a CI/CD (CodePipeline or another) in order to build and deploy FROM a Github repository TO Api Gateway + Lambdas (so using CloudFormation).

Thanks in advance if you have anything that could help ! :slightly_smiling_face:

4

Right now the process I have in place is github as source to codepipeline using with webhooks, next run codebuild project were I have a container with sls installed as environment, in build stage do sls package —package /path/to/package next in buildspec.yml set the zip generated and serverless.yml to be the artifacts now you have a immutable artifact which u can build once and deploy to all other environments. In codepipeline one pipeline does build, dev, qa and finally stage to s3. Another codepipeline then gets triggered with s3 upload and starts production deployment which has manual approval as the first stage after source.
i do not have the template to share at the moment build will try to publish sometime this weekend.

5

Hey @bishwash-devops,
I’m actually doing this right now too, but I’m currently calling sls deploy for each separate stage (Dev, staging, prod) which I’m not totally happy with.

I’m interested about how you generate an immutable artifact that you can deploy to each stage further down the pipeline?
My understanding (which may be wrong) is that “sls package” generates a stage-specific artifact? So I’d need to have the build step call “sls package” 3 times?
How do you handle this?

6

Hey @Julien.
I’m currently building a CICD process which uses CodePipeline and CodeBuild to deploy a serverless.yml service (Lambdas + APIGW) using the following pipeline:

  1. Merge to GitHub master branch
  2. Run lint + unit tests, Deploy to Dev, Run acceptance tests
  3. Deploy to Staging, Run acceptance tests
  4. Deploy to Prod, run acceptance tests.

It’s almost fully working now but I want to optimise a few things. I’m going to write it up as a blog post with source code included (serverless.yml, buildspec.yml + CDK constructs for creating pipeline). I’ll link to it here once I have it ready.

7

Thanks you all for your answers !
Hope to see your works on this subjects soon :smiley: If you want to share some draft, don’t hesitate, i will be glad to help you in the process !

8

After you run package the cloudformation template generated is stage/env specific but the .zip should not be, which you have total control over. so in buildspec artifacts i have

  • ./package/*.zip
  • serverless.yml
  • package.json

so when i am in another stage the ouput artifact has already packaged lambda function along with serverless.yml.

Then run : sls deploy —artifact ./package/name.zip

the cloudformation will be regenerated but the package does not have to be.

One addition is required in serverless.yml for this to work I had posted about it at medium a while back:
https://link.medium.com/W46tiZIfrV

1 Like
9

Thanks for this Bishwash, that makes sense and I’ll probably update my build steps to use your approach :slight_smile:

@Julien here’s the link to my blog post which might be helpful for you: https://winterwindsoftware.com/serverless-cicd-pipelines-with-aws-cdk/

1 Like
10

Hi,

When are you planning to share your work?
I am looking for something similar.

11

Hello,

I actually use only CodeBuild now, and not the full CodePipeline. It can be triggererd by a push on a remote repository and then run the commands you put in your buildspec.yaml. You only need to install your package (eg: npm install) and then run serverless deploy. Using the AWS console, the configuration is made quite easy if you’re familar with AWS services.

Hope it helps !

12
13

Hey, I wrote an article about deploying Serverless application using Github Actions, hope it helps!