Implementing newly-launched synchronous workflows for step functions

Event Gateway
, , ,
1

Hi all I have a scenario that is perfect for the Dec 2020 update to step functions: “Synchronous express workflows”. Express step functions can now block until complete, which means you can link them to the API Gateway and return a result to a web browser synchronously. The alternatives were messy.

I’m looking to do this with Serverless com — but it’s a pretty new feature. However, I think I can get away by breaking out into a CloudFormation template to specify the extra parameters. What I need to do is add API Gateway configuration along the lines of this:

    post:
      x-amazon-apigateway-integration:
        integrationSubtype: "StepFunctions-StartSyncExecution"

Source

I wish there was a complete reference of Serverless.com documentation. The docs have a bunch of examples but there’s nothing definitive that I can find.

Anyone have thoughts on where to start?

2

@boxabirds you might have a look at this repo as an example GitHub - aws-samples/contact-form-processing-with-synchronous-express-workflows: This application uses a Synchronous Express workflow to analyse a contact form submission and provide customers with a case reference number. and add resources section into your serverless.yml

1 Like
3

Thanks – yes I saw that resource. Unfortunately I’m not sufficiently experienced to know how to translate AWS examples into Serverless com examples without a slow and error-prone test process.

4

I was stuck on this for awhile and couldn’t get an endpoint to use integration with StepFunctions… Instead I had to use Lambda to invoke a StepFunction state machine.

1 Like
5

Hi,

I just had similar use case. Like you said, it’s a new feature, serverless-step-functions plugin does not support it yet (https://github.com/serverless-operations/serverless-step-functions/issues/378).

I managed to make it work with CloudFormation so I though I will share it with you.
You need to use step functions of type express and Http API gateway.
Step functions need to be express type, because they offer synchronous execution.

You need to create:

  1. Step function, e.g. by using serverless-step-functions plugin

     stepFunctions:
   validate: true
  stateMachines:
    myStateMachine:
      type: EXPRESS
      tracingConfig:
        enabled: true
      id: MyStateMachine
     name: 'My step function
     definition: ${file(state-machines/state-machine.json)}

  2. Execution role for HttpApi integration

     InvokeStepFunctionRole:
    Type: AWS::IAM::Role
    Properties:
  AssumeRolePolicyDocument:
    Statement:
      - Effect: Allow
        Principal:
        Service:
          - apigateway.amazonaws.com
        Action:
         - sts:AssumeRole
  Policies:
   - PolicyName: InvokeStepFunctionPolicy
     PolicyDocument:
       Statement:
         - Effect: Allow
           Action:
             - states:DescribeExecution
             - states:StartSyncExecution
           Resource:
             - !Ref MyStateMachine

  3. Given you have a gateway already created (HttpApi) resource, you need to create integration

    HttpApiStepFunctionIntegration:
  Type: AWS::ApiGatewayV2::Integration
  Properties:
   ApiId: !Ref HttpApi
   CredentialsArn: !GetAtt InvokeStepFunctionRole.Arn
   Description: 'Start my step machine step function'
   IntegrationType: AWS_PROXY
   IntegrationSubtype: StepFunctions-StartSyncExecution
   PayloadFormatVersion: "1.0"
   RequestParameters: {"Input": "$request.body", "StateMachineArn": "#{MyStateMachine}"}
  • note: #{MyStateMachine} is serverless pseudo parameters syntax for !Ref MyStateMachine

  1. Http Api route with integration from point 3.

     HttpApiRoute:
   DependsOn:
     - HttpApiStepFunctionIntegration
     - HttpApi
   Type: AWS::ApiGatewayV2::Route
   Properties:
     ApiId: !Ref HttpApi
     RouteKey: POST /start-stepfunction-sync
     Target: !Join ['/', ['integrations', !Ref HttpApiStepFunctionIntegration]]

The only drawback I noticed is the format of the response which cannot be mapped at the moment according to documentation, only headers and status codes can be mapped: [Modifying the response of the StartSyncExecution · Issue #6 · aws-samples/contact-form-processing-with-synchronous-express-workflows · GitHub]

Cheers,
jimmy

7

This is pretty late but just wanted to leave the message here just in case if anyone is looking for in the future. I was able to implement using the serverless configuration itself, without going to CloudFormation. Here it is:

stepFunctions:
  validate: true
  stateMachines:
    numbergeneratestatemachine:
      events:
        - http:
            path: step-functions/generate-number
            method: post
            action: StartSyncExecution
            request:
              template:
                application/json: |
                  #set( $body = $util.escapeJavaScript($input.json('$')) )
                  {
                    "input": "$body",
                    "stateMachineArn": "arn:aws:states:${self:provider.region}:${opt:aws-account-id}:stateMachine:MyStateMachine-api"
                  }
      type: EXPRESS
      id: MyStateMachine
      name: MyStateMachine-api
      definition:
        Comment: "Generate a number every 10 minutes"
...

The response from the API call looks like this:

{
  "billingDetails": {
    "billedDurationInMilliseconds": 200,
    "billedMemoryUsedInMB": 64
  },
  "executionArn": "arn:aws:states:xxxxxxx:xxxxxxxxx:express:MyStateMachine-api:xxxxxxx:xxxxxxx",
  "input": "{\"test\":true}",
  "inputDetails": {
    "__type": "com.amazonaws.swf.base.model#CloudWatchEventsExecutionDataDetails",
    "included": true
  },
  "name": "xxxxxxxxx",
  "output": "{\"number\":46,\"message\":\"event number\"}",
  "outputDetails": {
    "__type": "com.amazonaws.swf.base.model#CloudWatchEventsExecutionDataDetails",
    "included": true
  },
  "startDate": 1.634283447457E9,
  "stateMachineArn": "arn:aws:states:xxxxxxxx:xxxxxxxxx:stateMachine:MyStateMachine-api",
  "status": "SUCCEEDED",
  "stopDate": 1.634283447561E9,
  "traceHeader": "Root=1-xxxxxxx;Sampled=1"
}
1 Like
9

This was very helpful, thank you very much!.. can I process response to return a shorter one? i.e. output

10

Thanks for posting your reply. To your point, I just posted a question a few minutes ago before I found this thread on this very subject. I will give your solution a try.

Jay

11

When Response Parameter mapping is too limited to satisfy requirements, responses may be transformed in one of the states in your state machine prior to returning the response thru HTTP API.

For simple, predictable response transformations, I utilize the built-in JsonPath features with a Pass State in lieu of, or, in addition to a success state.

For more complex transformation, implement lambda customized to transform responses to whatever degree needed before returning to the HTTP API.

The above approach works well with WS API too.

Cheers!

Jay

12

Sorry about the delay. Yes, you can. Set the response template as follows:

template:
  application/json: |
    #set( $output = $util.parseJson($input.json('$.output')) )
    $output

This will return just the output from the statefunction response.

13

How to make the value of stateMachineArn dynamic instead of hardcoding in the request?