APIG Proxy via Step Functions

I'm creating a platform which consists of 2 APIs, with the 1st acting as a proxy to route traffic to the 2nd layer (this could be one 1 of several possible providers all at the 2nd layer).

The Problem

We didn't want to use the in-built APIG proxy functionality with the route as part of the query path as we wanted to keep the URL as standard as possible, and constantly changing the URL would go against the definition of REST APIs for our usecase.

Sticking with the music provider example I wrote about in the previous post, we would want to specify the provider and also the action to be performed. This may look like;

• https://www.myMusicProviderIntegration.com/spotify/play
• https://www.myMusicProviderIntegration.com/amazon/play
• https://www.myMusicProviderIntegration.com/spotify/pause
• https://www.myMusicProviderIntegration.com/amazon/pause

The URL needs to change for each provider. A cleaner implementation, especially for our consumers that are using our platform, would be to have just the action change on the URL.

• https://www.myMusicProviderIntegration.com/play
• https://www.myMusicProviderIntegration.com/pause

For this we ended up putting the provider part in a Header parameter which keeps the URL unchanged for the consumer.

Behind the scenes, we have a unique URL for each provider, that we then route to;

• https://www.myMusicProviderIntegration-spotify-provider.com/play
• https://www.myMusicProviderIntegration-amazon-music-provider.com/play

The Solution

Part 1 - Step Function Provider Routing

Using an Express Step Function as the first integration within the APIG allows us to add custom routing logic based on the header value.

This step function inspects the header values, and retrieves x-provider (our chosen header property). If the property is found, it routes to the specified provider, and if the header is not found then it routes to Provider Not Found and End processing.

"Provider Router": {
  "Type": "Choice",
  "Choices": [
    {
      "Variable": "$.headers.x-provider",
      "IsPresent": false,
      "Next": "Provider Not Found"
    },
    {
      "Variable": "$.headers.x-provider",
      "StringEquals": "spotify",
      "Next": "Call Spotify Component"
    },
    {
      "Variable": "$.headers.x-provider",
      "StringEquals": "amazonmusic",
      "Next": "Call Amazon Music Component"
    }
  ],
  "Default": "Provider Not Found"
}

Each of the choice tasks to call provider use direct integration to directly call the correct provider API.

This then brought on the next challenge, in that we wanted to dynamically specify the path that is called, from within the direct integration step for each provider. Given that we were also using stage names within our URL for differing environments, we had some nuance around getting the path part at run-time.

Part 2 - Intrinsic Functions

Intrinsic functions within AWS Step Functions allows for processing of data at run time, giving more flexibility of the option of build time configuration which comes out of the box with this type of routing. AWS docs state that intrinsic functions "help you perform basic data processing operations without using a Task state" which is exactly what we needed.

Note : To use intrinsic functions you must specify .$ in the key value in your state machine definitions, as shown in the following example: "KeyId.$": "States.Array($.Id)"

( As reminded by AWS)

For our case, we need to set the Path.$ property on the api integration.

Because we are not using APIG Proxy integration, we need to be able to inspect and extract the route off the original request, and add that to the request which goes to the provider api.

States.Format('/{}', States.ArrayGetItem(States.StringSplit($.path, '/'), States.MathAdd(States.ArrayLength(States.StringSplit($.path, '/')), -1)))

They're not the cleanest thing to work with, especially when there are multiple nested functions (of which we can have 10 nested functions) but if we break down each of the functions being used it becomes clearer.

States.Format formats the input parameters into an output. The key part of this function is that the first parameter, which must be a string, uses a pattern akin to Format Specifiers that are common in other languages (e.g. in c. Knowing that, we can see then that we are just creating a string with the value of the 2nd input parameter.

States.ArrayGetItem takes an array and returns the specified indexed value. We're providing as the array input, the output from the next function

States.StringSplit takes a string and splits on instances of the provided 2nd parameter (again, similar to many other languages), resulting in an array (this is the input to the previous function)

States.MathAdd is being used as the 2nd input to the ArrayGetItem function. Here we are repeating the previous steps to figure out the length of the array of path parts, and then providing -1 as the amount we want to add. Thus removing 1 and working nicely with our zero-index based array.

By the end of all of this, we were left with the last path part of the URL which was called to the initial API. So if we had the URL https://www.customDomainName.com/spotify/play we would result in play.

Adding this to the step function definition for Path.$ would dynamically pass the path part, and tie-everything together.

Intrinsic Functions and CDK (node.js )& TypeScript

We are using AWS CDK for node.js to handle IaC, which has a slight nuance in that the Path.$ property isn't exposed on the types so we weren't able to directly set it. However the power of javascript made this fairly straight forward to solve, by creating a wrapper class and setting the Path.$ manually.

export interface ExtendedCallApiGatewayRestApiEndpointProps extends CallApiGatewayRestApiEndpointProps {
    dynamicPath: string;
}

export class ExtendedCallApiGatewayRestApiEndpoint extends CallApiGatewayRestApiEndpoint {
   constructor(scope: Construct, id: string, private _props: ExtendedCallApiGatewayRestApiEndpointProps) {
       super(scope, id, _props);
   }

   _renderTask(): {
       Resource: string;
       Parameters:
           | {
                 [key: string]: any;
             }
           | undefined;
   } {
       const rendered = super._renderTask();

       return {
           ...rendered,
           Parameters: FieldUtils.renderObject({
               ...rendered.Parameters,
               'Path.$': this._props.dynamicPath,
           }),
       };
   }
}

This class extends the AWS CDK Construct, calls super._renderTask() to keep consistency with what we had before, and the modifies the returned object to set Path.$ with the value of the passed in property, which we included on a new interface ExtendedCallApiGatewayRestApiEndpointProps.

Summary

Using AWS Step Functions we have routed the initial request into our platform proxy api, to a specified provider API. In the process we used Intrinsic Functions to extract the path part and set this on our direct integration calling the provider api.

The following images shows the updated architecture diagram, with the 1st API updated to show the step function, which will be used for the provider routing.