Hello, I have the following DynamoDB record:
user_id, record_id, created, sample, balance
I also have one step function that is triggered when a file is saved on S3, this then checks if I have a parameter sample: true
if yes, it then executes the step one way, also it takes a different route, my question would then be, is what will be a correct approach to trigger the same function but only based on updating the sample
to false
on dynamoDb?
Similarly for balance
if the user does not have enough credit, I want to suspend the step-function and restart it when there is enough balance - for this i have a separate api endpoint, that checks the user’s balance - is it wise to have a loop to check this?
My use case is that i want to process the uploaded file, but ensure the user has enough credit before the work is done and want to find an efficient design to do this as one step-function for the entire ingest pipeline: upload -> transcoding -> transforming -> output
The sample
lets the user only sample partial part of the file so it does not require to check for balance
My initial idea was to set the triggers and persist the ingest state on dynamoDb, so that:
- when the file is added to S3, we have an even that creates the initial db entry, if sample,
true
by passbalance
check - else,
check
balance and if not enough - retry - this is where I get stuck as I don’t want to keep retrying indefinitely and would prefer to manually restart the step-function from that point - is this possible?
or thinking about it, as i am writing this, is to re-run the step-function but bypass some steps as these have already been executed - could be a better solution!
any advice is much appreciated