How to build your CI/CD pipeline for the full API lifecycle

For many, a continuous integration (CI) and continuous deployment (CD) pipeline is an important aspect of software development that provides wide benefits––from reducing error-prone deployment work to providing early visibility into bugs. This blog post will cover the API lifecycle through different phases––from designing and managing APIs to deploying and discovering them. The blog will provide a step-by-step guide for implementation and configuration using Anypoint Platform, including:

  1. Anypoint Design Center: Design
  2. Anypoint API Manager: Managing
  3. Anypoint Runtime Manager: Deploying
  4. Anypoint Exchange: Discovery

The following diagram outlines the high-level abstracted tasks usually involved in each phase in the API lifecycle.

ci cd anypoint platform

Phase 1 – Design

In the design phase, architects and lead developers can use Anypoint Design Center to design APIs. APIs are authored in RAML files, which include all different resources such as API fragments, specifications, and examples.

Once the API spec files are ready, API designer can publish the API RAML files to Anypoint Exchange. To publish the API RAML files to Exchange, you can use the “Publish” Option in Design Center. 

Phase 2 – Publish

Every resource published to Exchange is considered an asset, which means it has an asset ID, asset versioning, and other meta info.

Please note the following:

  • The “API version” field is derived from the “version” field in the main RAML file (value “v2” in the above example.) The “Asset ID” and “Asset version” are unique attributes associated with this API asset.
  • For every change published to Exchange, if the “version” field in the main RAML file is not changed, Exchange will only update the last digit of the Asset Version (PATCH info, i.e., if current asset version is 2.0.0 it will be changed to 2.0.1). But if the “version” value is changed, Exchange will consider the change as a major one. As a result, the asset version will will be increase by 1, i.e, 2.0.0 → 3.0.0

Phase 3 – Development

Once the design and contract are finalized, a developer would need an API asset from Exchange to start implementing the API in Anypoint Studio. Once the development is complete, we are ready to move to managing and deploying the API.

Phase 4 – Manage and deploy

The type of API endpoint in API Manager “Basic endpoint” versus “Endpoint with Proxy” will decide the deployment pipeline. In both cases, an API instance needs to be created with reference to the Exchange asset, but for the basic endpoint, you need auto-discovery properties generated by API Manager to pair an API instance with a Runtime Manager application.

ci cd anypoint platform 4

Please note this flow changed in November 2017, in the previous version of Anypoint auto-discovery properties were user-defined, but now those properties are provided by the platform.

In case of the “endpoint with proxy,” you can either deploy using API Manager or download the proxy application to be deployed with Runtime Manager Management API.

Additionally, the following information can also be provisioned in API Manager:

  • Client applications
  • Policies
  • SLA settings
  • Alerts

Pipeline in Action

First, we will define the system properties for username, password, organization, and environment ID of the Anypoint account, and we will also define two helper methods that will be used in the pipeline. This pipeline provides a basic CI/CD framework using Groovy scripting.

Getting the authentication token

The first step is to get the authentication token using Anypoint credentials, the snippet shows how to call the accounts/login API to obtain a token.

Getting Design Center project details

First, we need to get the project and branch ID from Design Center, these will be used to publish the asset to Exchange.

The Management API used in this use case is available on Anypoint Exchange.

Publishing to Exchange

This step publishes a Design Center project to Exchange. Please note this is only meant for RAML/OAS asset types. If you wish to manage a SOAP/HTTP asset type please publish the asset into Exchange directly using this API.

You can also use Anypoint CLI for automating design center projects, for more information please see this documentation.  

The Management API used in this use case is available on Anypoint Exchange.

Getting details of an asset in Exchange

Refer to this code in case you need to fetch the details of an asset published in Exchange, such as the Asset ID, API version, Asset version, and group ID.

Managing an asset from Exchange in API Manager (Get from Exchange)

This step is the equivalent UI step of “Get from Exchange,” we will manage an Exchange asset in API Manager in order to apply policies and other runtime governance.

The snippet below shows how to create a basic endpoint for a Mule 3.x API that is available at URI:

The management API used is available on Anypoint Exchange.

The response apiInstance contains an id unique identifier for an API in API Manager. This ID is used for other API operations like applying a policy, API promotion, API deprecation, deleting an API, finding out autodiscovery properties.

Tip: To create a Mule 3 proxy type API on CloudHub for an implementation endpoint  request body will change to the following:

To manage a Mule 4 API, you must set this property to true muleVersion4OrAbove.

Getting Auto-discovery details for an API

This step is optional and only needed if an API type is a basic endpoint type. This endpoint is invoked to get auto-discovery (API name and version for Mule 3), which will be used when deploying to Runtime Manager. The examples below get auto-discovery for a Mule 3.8.5 API.

Management API used is available on Anypoint Exchange.

Note: For Mule 3 APIs, you need api.name and api.version, but for Mule 4 APIs only api.id is needed. For more details, please see read this documentation.

Getting the details of an API instance

This step fetches all the available instances in an API environment and filters the result based on an Asset version and Asset ID.  

Management API used is available on Anypoint Exchange.

Applying policies to APIs

After creating an API instance, you can apply policies to secure access. For example, the following code shows how to use the “Client ID Enforcement policy” on an API

Management API used is available Anypoint Exchange.

Deploying an API

Once the parameters required for auto-discovery are discovered and an API is secured via policies, the next step is to deploy the application to Anypoint Runtime Manager and use auto-discovery properties to manage that API.

Basic Endpoint Type

In this example, we will deploy an application to CloudHub using Anypoint CLI and use property placeholders for configuring auto discovery properties.

ci cd anypoint platform 5

This step of the process is essential to managing a particular application implementing the API. The link between Mule runtime (implementing the API) and the API instance (in API Manager) is done thanks to four different properties:

  • API Name
  • API Version
  • client_id
  • Client_secret

The combination of these two values: API Name and API Version (see the diagram above), identify a unique instance in API Manager. This information is important to link this API instance to the running application that will implement the API. It also allows its management (apply policy, statistics, alerts, etc.).

The client_id and client_secret identify the environment where the Mule application is running. Therefore, these parameters should be passed to the application to be used by the component “API Auto-discovery” as part of the configuration management process prior to deploying any environment.

Deploying an API Proxy

This demonstrates how to deploy an API proxy for a proxy endpoint API to CloudHub 3.8.5 runtime using API Manager proxy API.

Management API used is available Anypoint Exchange.

Tip: There is an alternative for proxy deployments, which is to download the proxy application using the API Manager management API and then deploy the application using either Anypoint CLI or Runtime Manager API.

Promoting an API to a different environment

Once the developer has completed testing in the development environment, the next process is promoting the API from the development environment to the test environment in API Manager. This promotion can include policies, tiers, alerts.

Note: To promote an API from one environment to another, you must have access to both the source and target environment. The URI property contains the target environment. The source environment is specified by originApiId as shown in the following example:

Management API used is available on Anypoint Exchange.

After promoting the API, you may need to do either of the following:

  • If the API is of basic endpoint type, then promote the application in Runtime Manager to the test environment and use the new auto-discovery properties that will be retrieved using the “Getting Auto-discovery details for an API” step.
  • If the API is of proxy endpoint type, then deploy a new proxy application in the Runtime Manager TEST environment.

This concludes the basic of building a CI/CD pipeline using Anypoint platform.

Important resources

If there’s an API call that you’re planning to use for CI/CD, which is not covered in this blog, please refer to our API portal for more information or let us know in the comments

Anypoint CLI

Anypoint Platform provides a scripting and command line tool to interact with:

  • Anypoint Exchange
  • Access Management
  • Runtime Manager for Mule applications deployed to MuleSoft-hosted (CloudHub) and customer-hosted Mule runtimes
  • VPCs
  • CloudHub load balancers
  • Design Center
  • Anypoint Exchange

This is done through both interactive shell and standard CLI modes for both Anypoint Platform and Anypoint Platform Private Cloud Edition.

To know more about CLI and to install it on the server that runs the CI/CD tool, look at the documentation.

Which use case are you planning to use for CI/CD? Let us know in the comments!


 


We'd love to hear your opinion on this post


6 Responses to “How to build your CI/CD pipeline for the full API lifecycle”

  1. Hi Chirag,

    Thanks, a timely article for me – as I was looking for automating deployment. However the approach I was interested in – via Management APIs, is still not clear. You referred to Management API in you Tip but that URL doesn’t seem to support Mule 4. Is this conclusion correct ? Can we use management APIs for automating deployments ? That mock service requires parameters like organisation Id, environment id and environment app id. I couldn’t locate the last 2.

    • Hi Joy,

      It’s possible to use the same API for Mule 4. query params can be used to specify the gatewayVersion.

      EnvironmentAPIId refers to apiInstanceID which is covered in the section “Managing an asset from Exchange in API Manager (Get from Exchange)”
      EnvironmentID is the environment where that API instance is located.

      Hope this helps.

      Thanks,
      Chirag