Creating a platform to keep internal consistency when integrating with differing 3rd party APIs

Integrating with a 3rd party can be a complex endeavour. Their API will have specific nuances that need to be taken into account, such as content type (JSON vs XML), webhooks & callbacks, payload field names, to name but a few. There is also the added challenge that their handling of the domain might try to dictate how you handle your domain. For example a music library service might first require the artist be created, 2nd an album is created and associated to the artist, and then finally some songs can be created and associated to the album. Another library providing the same service might require the artist be created, but then the songs are created next, and then finally the album is created and the songs are then associated to the album (after all, you can't have an album without any songs, right?).

This isn't too much of a problem if you're only integrating with one 3rd party API, but what if you are integrating with multiple 3rd parties, each providing the same end service, but depending on the internal business logic you might want to use service A or service B. Sticking with music libraries, you might want to use Spotify or Amazon music depending on specific parameters. And you'd want to be able to switch between the 2 with ease.

We don't want each area of the domain to have to do their own integration to each of the differing services. This would quickly get unwieldly. In OOP we would use an Interface (of SOLID fame), why wouldn't we do the same in a serverless approach?

Well, we can. Creating a platform that has a consistent internal layer with a single APIG, that routes through to a specified provider from a selection of multiple providers. We handle each provider and any specific implementation they have, separate from each of the other providers.

There are several layers to this architecture.

- Firstly, we have a top level API that serves as the entry point for everything which needs to communicate with our 3rd parties `Music Provider Gateway`. This has a single base URL, that remains consistent across all providers with the chosen provider passed in as a parameter (e.g. URL path using APIG Proxy).

- We use Cognito to auth the request coming into the platform

- Next we route the incoming message to the specific provider component

- The provider component then handles all the specific implementation for that provider. Mapping payloads, formatting, and subsequently calling the API

- The response is then normalised to a consistent format and returned to the caller

- The implementation in this example is within a step function, which would allow us to build in retry, error handling and extensibility in any number of ways.

- If the response is being processed async, then the response is emitted on an event bus for the original caller to listen for.

- If the response is being processed sync, then it just returns down the calling path out to the initiator

When more functionality needs to be added in, we can create a new endpoint and then configure the specific implementation in each provider. If a new provided is needed, we can do the leg work behind the scenes without our consumer needing to make major changes - they just need to pass the new provider-id to the gateway API and the rest is magic.