Hi.
i’m working on a project built on serverless, (like everything).
And one thing we do is sending user behavior data to KinesisStream, and process that in Lambda, and put aggregated records to DynamoDB. and we use those data to optimize recommendations… etc
Problem we started to develop is DynamoDB hot-key. which means we’re accessing same record too frequently. Reason of that is KinesisStream - Lambda trigger only has “maximum batch size”, so that a lot of times it process relatively too small batch of records.
Like let’s say we want to aggregate user’s click count, and put it to DynamoDB table like (user_id: hashKey, count: number)
number of DynamoDB Table write access is "number of unique users in given batch size"
so, increasing batch size would definitely beneficial if process can be done with in 5 minutes, which is lambda execution limit
i’ve checked AWS documentations and things but couldn’t really find any. have any idea about this guys? thanks.