I’d be interested in hearing how you all are dealing with debugging. Lambda errors are notoriously difficult to interpret and contain minimal information. I’m using Atom which has little in the way of syntax checking. Debugging has been a challenge for me. How are you dealing with it?
1 Like
I run my functions locally, and if I get syntax errors I enable SLS_DEBUG=* to get more detailed information. If the functions are running, but I’m getting service failures, I’ll dump errors to the logs in debug mode.
What types of errors are you trying to debug? Connecting to AWS services? Syntax errors? Are you running a test suite to unit test your functions? What language are you using?
Hi All,
As @jeremydaly said “What types of errors are you trying to debug? Connecting to AWS services? Syntax errors? Are you running a test suite to unit test your functions? What language are you using?”
what type of errors you are encounting…! I need to add one more question to this… I.e., Run-time Errors.
Actually, “A run-time error is generated when a VBA program attempts to execute an invalid statement at run time.”
If you are clear about what type of errors you are getting. It will be easy to get your answer…
Thanks,
Kimberly
1 Like
Hi!
I don’t have a clear answer, but I’ll share my experience so far with debugging functions.
I’m currently building a medium sized platform with serverless, using typescript + webpack.
I opted to do integration tests instead of unit tests in order to have some sort of flows covered. For this I piggybacked on webpack and I use the bundles it creates and run them locally in some plain node.js scripts.
This way I can build flows using responses from one function to another. I’m using VSCode and its debugger is very helpful when it comes to environment variables.
Obviously there are some catches to this approach:
Catch 1 - My tests are not covering the serverless.yml configuration
Since I’m using only the code that will be deployed to the Lambda function, I often encounter errors generated by serverless.yml misconfigurations, mostly forgotten environment variables.
Possible solution: I created flows in postman that work directly with API that would reveal any endpoint misconfiguration and I can use them as working examples and for demo purposes.
Catch 2 - A webpack generated bundle is a bit hard to debug
Webpack bundles everything in one file. I tried to keep the number of modules used at minimum, even avoided using lodash 
, but still one file has 1-2 MB. You can imagine that these files are a bit hard to debug but not impossible.
Possible solution: I’m thinking to transpile the typescript code to js and use those files without bundling and tree shaking.
Catch 3 - The process of preparing the files for testing is long
The bundle creation time increases with every new function you add. I have now 60 functions that are bundled every time I need to run the tests, and it takes 4 minutes to finish. I have to run this after every update I do to the functions code, so TDD is not easily achievable.
Possible solution: Bundle only the functions needed for a certain test. This way I don’t have to bundle every thing every time. This way I can get a shorter bundling time and still use the final product that would run in the lambda function. The downside is to configure every test with what needs to bundle.
I know this is not probably the answer you were looking for, but it might give you an insight on how others are doing it. Also I think this course might help with debugging and in general about using serveless https://livevideo.manning.com/course/38/production-ready-serverless?
2 Likes
@IulianGioada great answer. enlightening. I’ll share how I’m doing tests and the problems I encounter.
A bit about my stack:
- Python + Django
- Using Zappa
- My AWS services: S3, Lambda, RDS, ES
As you can see I have 2 services that are not completely serverless, they are managed, but there is an actual server behind the scene, it greatly affects my testing strategy.
I use the standard testing pyramid, e.g. unit testing my Lambda functions, integration tests for my end to end testing and exploratory testing to find hidden “gems”. Pay attention that except unit tests I run all of my tests in real environment, I try to avoid mocks and simplify my dev environment and components I’m dealing with. For actual debugging during exploratory testing I use CloudWatch --> ES integration to send my logs for improved digestion. CloudWatch is horrible and unusable, ELK is a god sent service.
The issues that I encountered during testing:
- Each of our developers have a separate environment to test their code in live cloud environment, it’s expensive, each one needs to provision RDS + ES, using AWS organisation features does not really help, in addition provision each time a new environment to a new developer is a nightmare.
- Network is a real problem, as you mentioned preparing, uploading the fixes take a lot of time, it’s really annoying and takes the fun out of it.
1 Like
@IulianGioada If you’re using Typescript & webpack then I can recommending
- Use the webpack-node-externals plugin so that it doesn’t bundle your node_modules. I’ve had problems with this.
- Enable source maps
- Add the source-map-support NPM package as a dependency
- Add
import "source-map-support/register";to the top of your files - Set a minimum of 256MB for your Lambda
- Don’t use tree shaking. It will make your files a lot smaller but it mucks with source maps.
Using this approach my CloudWatch logs now show errors from the original source code instead of the JavaScript file that webpack creates.
1 Like
I’m using Node.js Lambdas that need to take weird real-world raw input events from fairly dumb IoT devices and then do some processing before sending them on to a legacy system. It’s a bit hacky and local debug weren’t doing the job because they were so different from the real use-cases.
I switched over to using Rookout for live Lambdas a while back. It lets you do step debugging on the Lambda by setting oldskool breakpoints that dump the stack without stopping the function execution.
Adding my 2c here. I have found unit tests the easiest way to debug locally. To run your Lambda you will probably need some kind of wrapper script to pass it parameters to execute so you might as well make it a unit test. Now you can execute tests with breakpoints any time you want. There are a number of node modules I use to help with this:
- serverless-mocha-plugin: Helps wrap your lambda in a way that makes passing it event data easy as well as then performing assertions.
- aws-sdk-mock: You often need to communicate with AWS services but when running tests you don’t want to do this. Also, its difficult to test AWS service failure if you are pointing at a live service so this module lets you simulate specific responses
If you want anymore clarification feel free to ask 
1 Like
I ran into the same issues so I created “serverless-plugin-local-env” which is an npm module that uses your credentials to get all the environment variables you reference either from your resources section or import from other cloudformation templates.
I also use “serverless-vs-code” to help debug my functions, as it allows you to setup your test input JSON pretty easily, and have multiple teest cases.
https://marketplace.visualstudio.com/items?itemName=MutualofEnumclaw.serverless-vs-code
Hi, Thundra debugger is one of the debugging solutions for AWS lambda, you can try it.
I will probably be a minority, but I just try as hard as possible to keep my code as flat as possible. When I have tests, this means I can write a test, reproducing the issue, in fifteen minutes or less and go wild with log.Printf from there. And, I’ll have a test case after snaptube vidmate I’m done. When I don’t have tests, I just add slightly more of log.Printf (and don’t have a test case afterwards 
).
1 Like
I just got an email telling me this topic I had posted two years ago just got updated. I had long forgotten I had posted it.
Two years later, I’m still very interested in this. Since I posted this, I’ve written countless thousands of lines of node.js on Lambda. I’ve moved my entire operation to Lambda (and other parts of AWS), including some fairly large web applications. I even moved my development to AWS Cloud9. I’ve drunk the AWS koolaid. My debugging is still primitive, though. I stick console.log in my code and spend time in Cloudwatch looking at logs.
I only recently moved to Cloud9 and I know that it has some debugging capabilities for Lambda but I admit, I haven’t really investigated them yet. I have too much to do (like most of us) and I’ve gotten used to my primitive Cloudwatch-oriented debugging and find it doesn’t materially slow me down. That doesn’t mean I’m not looking to improve, however.
If any of you have interesting solutions or want to share how you debug your Lambdas, I’d be very interested to read them.
At this point in time I would recommend taking a look at using Serverless Framework Pro with its (stinn in beta) Studio feature. Studio lets you connect a service you are building on your local machine to a stack in the cloud and watches for changes to your Lambda functions. As soon as the file is saved, it is updated in the cloud within seconds. Then, studio also provides you a testing interface, very similar to Postman, so that you can trigger your freshly updated Lambda function. In this way you can develop locally but actually test in the cloud instead of trying to find ways to create inaccurate copies of the locally or mocking services. Actual testing that shows you actual results from the real cloud services you will use.
I’ve been using the hello world app made with AWS Serverless Application Model (I realize this is a different tool), but they include an ability to run a “API” locally to execute the containerized lambdas (running in a docker container) and can pass a debug flag when starting the API and then get a connection from vscode:
In VS code in launch.json:
{
// Use IntelliSense to learn about possible attributes.
// Hover to view descriptions of existing attributes.
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
"version": "0.2.0",
"configurations": [
{
"type": "java",
"name": "Debug (Attach) - Remote",
"request": "attach",
"hostName": "localhost",
"port": 9999
}
]
}
In a terminal in the hello-world SAM application
sam local start-api -d 9999
Then in VS Code set some breakpoints and hit the launch debug button.
Assuming using the hello world project you can then hit the URL: http://localhost:3000/hello and any breakpoints that are hit will allow you to stop and inspect and step through just like any other remote debugging. I’m curious if something similar exists within serverless (not SAM) since our projects are setup to already use serverless and would like to avoid replicating configuration just for the sake of using the debug tools with SAM.