Blog

Integration and API news for developers

Conducting Data: The Art of API Orchestration

18th November 2017

APIs are an essential attribute for SaaS applications, expanding their internal and external abilities for a wider audience. However, the design of these data pathways can be a difficult task.

Creating several APIs, each of which is focused on a particular area of your application, allows for better data management and future updates.

An email marketing platform API, for example, may have the functionality to:

  • store contacts
  • support lists
  • ability to setup & deploy campaigns
  • store reporting data


API Orchestration

By creating these as separate APIs, and treating them as services, it allows for easier management and the ability to add new functionality in future releases without breaking users’ existing API usage. But how do we get these separate APIs to communicate with each other?

This is where API Orchestration comes in.

What is API Orchestration?

An orchestration layer steps in when multiple API services need to be coordinated.

It adds the ability to manage data formating between separate services, where requests and responses need to be split, merged or routed.

By adding this abstraction layer you provide your API with a level of intelligence for communication between services, while handling vital authentication tasks.

Adding an orchestration layer to your API gives you many benefits, including:

Data Transformation

Data TransformationNot all data requests are sent or received in a standard format, this can be true for both the client and the backend. Either may require data responses to be sent in a particular format, requiring data transformation to allow both client and server to communicate in a common call format.

The orchestration layer can step in to ensure payloads are correctly formatted using its transformation abilities.

Management Runtime Layer

runtime layer

By treating each part of your application as a series of microservices the orchestration layer can step in before requests reach your API backend. This provides many advantages when it comes to security (runtime policies checking for malicious code threats), ensuring protocols are standardised, data validation and error handling/logging.  

 

Integrating Legacy Systems

Legacy systemsMixing of API types, such as REST and SOAP, can lead to major headaches when trying to use multiple APIs together. In steps the orchestration layer to normalise the data structures, managing the flow of data letting you communicate with the most neglected APIs and microservices.

Handling Authentication

AuthenticationThe orchestration layer can be used to handle the OAuth flow between microservices, allowing for use of role based permissioning systems. This allows you to make only selected data accessible to selected users who fall into a given role.

Server Management

Server ManagementA public API can cause spikes in server usage, especially when integrated with external applications that heavily rely on your service. The orchestration layer can step in to help mitigate these spikes that can cause your application to grind to a halt.

Features such as API call restrictions can step in and limit over zealous applications spamming your server with requests.

While paging and cashing can be implemented to give you more control with how your data is delivered and received. Setting up cashing can reduce server CPU load by storing previous responses in memory, allowing for faster querying as additional calls to the API’s backend are not required.

The Missing Piece of the Orchestra

In our Email marketing platform example, as well as being able to communicate and query between your services, you can add external partner API services to enhance your application’s functionality.

For example, you could add a data enrichment service, such as FullContact, in order to enrich your contact data, allowing for the creation of more granular lists and more targeted messaging.


Adding services to orchestration layer
However, you would have to create individual connections from each of your APIs to allow them to communicate with your external partner API. This may not sound like a big task, but bare in mind that you would have to do the same for any other external partner API you want your service to connect to.

All of a sudden the number of connections to create and maintain begin to snowball, creating significant developer overhead.

This is why at Cyclr we propose another way of managing external partner API connections, by integrating only one API – ours.

The Cyclr API orchestrates all of your API connections, providing you with an orchestration layer to manage the flow of data and build automation workflows on top of. Our management console and visual, drag and drop integration builder also allows for the delegation of previously developer only integration tasks to other areas of your team, including marketing and sales, leaving developers to develop your product!