Dynamodb streaming with Lambda

Serverless Architectures
1

I am beginning to learn the patterns for using dynamodb streams to maintain ‘aggregrate’ derivative tables etc.

I see that dynamodb streams only deliver max 24 hrs worth of any data…

so what happens if way down the line, I figure out I need a new derivative table (populated by a dynamodb stream lambda processor) ? its only going to receive max 24hrs worth of data from the source table(s). I would need to replay all the data from the source tables some clean way…

is there a recommended pattern for this ?

2

This is a good question.

I think you would need to persist the stream elsewhere (s3/redshift ¯_(ツ)_/¯) for replays.

I don’t know if any stream service retains the transaction logs forever. (Would make event sourcing much easier =P)

3

thanks… it hard to get some solid examples of solutions to all the various dynamodb limitations - getting around transactions, replaying views etc :frowning: