It’s got to the stage where I need some continuous integration to ensure changes are deployed predictably. I have this script:
#!/bin/bash
for i in */
do
( cd "$i" && npm i && npm run deployprod )
done
First off, I probably should have one huge serverless.yml instead of separate lambda functions. WDYT?
Secondly, most painfully, I don’t want to deploy unless something has changed! Else sls deploy list becomes really hard to follow. Also my deployment stage takes like 5 minutes with the 10 lambdas or so that I have.
I have one serverless.yml for each service rather than one for each Lambda function. A service being a collection of related Lambda functions (microservice). To deploy a service I use npm run deploy which runs a bunch of tasks (compile, lint, test, etc) before deploying the service. While each service is deployed separately all of the functions inside a service are deployed together.
To support different stages I have npm run deploy:STAGE for example npm run deploy:dev. Thinking about it, I should probably make npm run deploy use dev by default instead of prod and make prod more explicit.
Sidenote: Has anyone noticed these TypeError: Cannot read property 'pipesCount' of undefined errors?
> sls deploy -s production
Serverless: Packaging service...
_stream_readable.js:545
switch (state.pipesCount) {
^
TypeError: Cannot read property 'pipesCount' of undefined
at module.exports.Readable.pipe (_stream_readable.js:545:16)
at module.exports.ZipArchiveOutputStream._smartStream (/home/travis/build/Spuul/video-ingestion/lambda/admin-update/node_modules/serverless/node_modules/compress-commons/lib/archivers/zip/zip-archive-output-stream.js:184:11)
at module.exports.ZipArchiveOutputStream._appendBuffer (/home/travis/build/Spuul/video-ingestion/lambda/admin-update/node_modules/serverless/node_modules/compress-commons/lib/archivers/zip/zip-archive-output-stream.js:82:10)
at module.exports.ArchiveOutputStream.entry (/home/travis/build/Spuul/video-ingestion/lambda/admin-update/node_modules/serverless/node_modules/compress-commons/lib/archivers/archive-output-stream.js:86:10)
at module.exports.ZipStream.entry (/home/travis/build/Spuul/video-ingestion/lambda/admin-update/node_modules/serverless/node_modules/zip-stream/index.js:138:49)
at Zip.append (/home/travis/build/Spuul/video-ingestion/lambda/admin-update/node_modules/serverless/node_modules/archiver/lib/plugins/zip.js:53:15)
at Archiver._moduleAppend (/home/travis/build/Spuul/video-ingestion/lambda/admin-update/node_modules/serverless/node_modules/archiver/lib/core.js:172:16)
at Archiver._onQueueTask (/home/travis/build/Spuul/video-ingestion/lambda/admin-update/node_modules/serverless/node_modules/archiver/lib/core.js:370:8)
at /home/travis/build/Spuul/video-ingestion/lambda/admin-update/node_modules/serverless/node_modules/archiver/node_modules/async/dist/async.js:4045:9
at process (/home/travis/build/Spuul/video-ingestion/lambda/admin-update/node_modules/serverless/node_modules/archiver/node_modules/async/dist/async.js:2316:17)
at Immediate._onImmediate (/home/travis/build/Spuul/video-ingestion/lambda/admin-update/node_modules/serverless/node_modules/archiver/node_modules/async/dist/async.js:66:16)
at runCallback (timers.js:800:20)
at tryOnImmediate (timers.js:762:5)
at processImmediate [as _immediateCallback] (timers.js:733:5)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! video-admin@0.0.1 deployprod: `sls deploy -s production`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the video-admin@0.0.1 deployprod script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
When it comes to managing many functions we have taken the approach of breaking things up into distinct business domains and each domain is essnetially a single Serverless service. For example, Customer data management is a single service which has exposed API’s for receiving new customer data, editing it, allowing API calls to retrieve it as well as event based functions to add customer to mailing list if appropriate, log all interaction CRM-wise, etc. This way if we need to make a change to something related to “Customer” its only one service that needs updating and deploying.
Speaking of which, we also currently use Bitbucket for SCM and with that comes Pipelines whcih we use for our CI/CD, well, pipeline. This allows us to, as soon code is pushed into the repo, run all our tests and if they pass run sls deploy to then have all changes automatically pushed live. As a part of our process, developers run their tests locally (using a combination of serverless plugins to run offline and that wrap Lambda functions for unit testing purposes), they then push their code at the develop branch which runs our HTTP integration tests (a very limited number) as well as the unit tests again before the code being accepted by me into the master branch which then automatically goes live.
Doing things this way we don’t really care how long it takes to go live because we can test all code offline and not have to worry about deployments. Any communication between services is done over HTTP and a RESTful architecture.
Type Error ---------------------------------------------
TypeError: Cannot read property 'pipesCount' of undefined at ReadStream.Readable.pipe (_stream_readable.js:637:13) at Readable.<anonymous> (/opt/atlassian/pipelines/agent/build/node_modules/lazystream/lib/lazystream.js:33:12) at Readable.instance.<computed> (/opt/atlassian/pipelines/agent/build/node_modules/lazystream/lib/lazystream.js:18:14) at Readable.read (/opt/atlassian/pipelines/agent/build/node_modules/readable-stream/lib/_stream_readable.js:443:10)
at resume_ (/opt/atlassian/pipelines/agent/build/node_modules/readable-stream/lib/_stream_readable.js:790:12)
at processTicksAndRejections (internal/process/task_queues.js:84:21) For debugging logs, run again after setting the "SLS_DEBUG=*" environment variable.
Get Support --------------------------------------------