Cron Job runs twice in Serverless using AWS Lambda

Serverless Framework
,
1

I am running a Cron Job which runs every Sunday at 10:00am UTC

It pushes a Text Message to a Telegram Group

Here’s my full code :point_right: https://github.com/deadcoder0904/lessons-of-hn-telegram-bot

Relevant parts below -

serverless.yml

service: lessons-of-hn-telegram-bot

provider:
  name: aws
  runtime: nodejs8.10
  environment:
    BOT_API_KEY: ${file(./env.json):BOT_API_KEY}
    CHANNEL_ID: ${file(./env.json):CHANNEL_ID}

functions:
  cron:
    handler: handler.run
    description: Cron job that runs every Sunday at 10 am UTC
    events:
      - schedule: cron(0 10 ? * SUN *)

handler.js

const axios = require("axios");

const { lessonsOfHN } = require("./src/lessonsOfHN");

exports.run = async (event, context, callback) => {
  const time = new Date();
  console.log(`Your cron function "${context.functionName}" ran at ${time}`);

  const lesson = await lessonsOfHN();
  const ENDPOINT = `https://api.telegram.org/bot${
    process.env.BOT_API_KEY
  }/sendMessage`;

  await axios({
    method: "get",
    url: ENDPOINT,
    data: {
      chat_id: process.env.CHANNEL_ID,
      parse_mode: "markdown",
      disable_web_page_preview: true,
      text: lesson
    }
  });

  callback(null, { lesson, success: true });
};

Nowhere in the code a request is sent to Telegram twice ,i.e, axios is only written once

The logs I got says the cronjob ran twice once at 10:00:09 am & at 10:01:08 am

I am using Serverless Framework with AWS Lambda

How to make sure it only runs once? Since people get 2 messages in the telegram group because of this :frowning:

2

First: If you declare your handler async then you can’t use callback to callback to send the response. You need to return { lesson, success: true };

Second: Your Lambdas need to be idempotent because they may be executed multiple times with the same input. This is baked into the basic execution model which guarantees a Lambda will be executed at least once.

From Invoke - AWS Lambda

Invocations occur at least once in response to an event and functions must be idempotent to handle this.

Most of the time it will only executed once but it will occasionally it will run multiple times.

3

Thank you :raised_hands: Didn’t knew about this

I don’t understand this. Is Idempotent same as Pure functions in Functional Programming?
Same Input => Same Output. If so then it does that.

So I have removed callback & did do return { lesson, success: true };. Anything else I have to do?

Because when I invoke the function sls invoke -f cron -l then it just runs once ,i.e, sends 1 message to the Telegram Group, but in Cron it runs twice :frowning:

Is there any way to prevent this?

4

I think you’re main problem here is using async and callback. The async handler syntax with Node 8.10 doesn’t include the callback parameter. Using return should solve the bulk of your problem.

Pure functions don’t have side effects like network calls.

Being idempotent is not the same as being pure.

For example: I could have a Lambda that creates a record in DynamoDB. If the Lambda generates a random string for the key then it’s not idempotent because subsequent calls with the same event will result in multiple records being added to DynamoDB but if I have a consistent way to generate the key (perhaps using an ID included in the event) so only one record is ever created and I always return success even if the record already exists then I’m idempotent.

5

Cool thanks man for all the explanation. I deployed the new version. Let’s see how that works this Sunday. I’ll post the updates if it works or if it doesn’t :smiley:

Also, if I can store the data in DynamoDB then I can make a boolean to not run Lambda the 2nd time

By default, runCron = true

On 1st run, it runs the cron & make runCron = false

On 2nd run, its runCron = false then don’t run it. Can I do this in Lambda?

Currently, it runs twice every Sunday (10:00 am & 10:01 am UTC)

6

A Lambda being executed multiple times for a single event should be very rare. If the async/callback change fixes your problem and the occasional duplicate message isn’t a problem then personally I wouldn’t worry about making the Lambda fully idempotent.

1 Like
7

Hey it just ran but again it ran twice. Any idea of how I can fix this now?

I also changed async/callback fix as stated :point_right: https://github.com/deadcoder0904/lessons-of-hn-telegram-bot

8

Can you simplify the function by removing the lines with await to confirm that it only runs once?

exports.run = async (event, context) => {
  const time = new Date();
  console.log(`Your cron function "${context.functionName}" ran at ${time}`);
  console.log("event ", event);

  return { message: "success" };
};

If it still executes multiple times are they for the same event?

If it works then is it possible your function is timing out during the first execution? Try adding each of the lines with await one at a time to see if it still only triggers once. The CloudWatch logs can also tell you this.

If your function is timing out then increase the timeout.

Are you sure the service only scheduled once? Check CloudWatch to make sure you don’t have two accidentally scheduled - perhaps in a different stage or region?

1 Like