With the influx of AI-powered business tools, companies have begun incorporating large language models (LLMs) into their existing systems, with APIs offering a convenient way to access and interact with these models.
LLMs are constantly being refined and improved using pre-trained models and large datasets. What’s great is that advanced LLMs are now accessible to the general public, allowing everyone to benefit from this groundbreaking technology. Because LLMs can generate various types of content, e.g. text, designs, and video material, what they create can be integrated with other applications to accomplish tasks, like data analysis and information retrieval, making business interest in LLMs a no-brainer.
Developers can easily utilize popular LLMs like those offered by Google and OpenAI through APIs that have free tier options. This enables them to seamlessly integrate natural language processing (NLP) capabilities into their applications, paving the way for the widespread usage of generative AI across various platforms and industries. What we present now concerns an open-source project on MuleSoft AI policies.
Architecture with Gateway and AI
The figure below is an example of an Architecture with Gateway and AI:
- Mule Gateway: Mule Runtime includes an embedded Mule Gateway. Using this gateway, any user can apply a basic authentication policy on top of a Mule application or a masking/toxicity policy, enrich an incoming/outgoing message, or add any other complex capability to an API without having to write any code.
- LLM APIs: These provide unprecedented access to natural language understanding capabilities for processing and evaluating.
- AI Engines: This is a Swiss Army knife for businesses that utilize technology. The engine includes a variety of tools and methods that enable computers to perform tasks that have, until now, required human intelligence.
AI custom policies
Because security is paramount for MuleSoft, using policies in Anypoint Platform can protect data and APIs. In addition to existing policies, users have the ability to create Custom Policies. We’ll go through a few custom-made policies created for Mule Gateway. Once created and deployed, they will be visible assets on Anypoint Exchange, and users can find them among the policies to apply using API Manager.
The Toxicity AI policy protects our data by checking whether the content contains toxic language. Today, it is very important to pay attention to the data that passes through in full compliance with the rules and code of ethics.
Among these, there is the Policy Tokenization (under Security group) that tokenizes a value or a group of values applying rules defined in a Tokenization Service. However, this Policy has a limit: it can only be applied to APIs deployed on Runtime Fabric Bare Metal/VM.
So what about the other APIs? The solution is to create a Custom Policy and use AI as a Tokenization Service, creating a Masking AI policy.
Masking AI policy
This custom policy is similar to the Tokenization Service that is present using Runtime Fabric, which masks sensitive data with special characters. This policy totally uses AI for masking the fields that the AI deems to be sensitive data to mask and for to do this use the OpenAI engine.
You need to insert a key to access OpenAI and insert the Payload Path, in the form of Expression, used to extract the payload value from API request. You can also choose API methods and resources where the policy will be applied.
This is a simple request that contains fields with a similar name, inserted specially to show how the AI works. In this case, AI considered that fields such as email, phone number, PIN, credit card number, and date of birth are sensitive data, so it masked them with a particular character. This means the AI is free to choose which fields have sensitive data, although the fields may differ from time to time.
Toxicity AI policy
In the MuleSoft messages, there are several references to the data that transit between the various systems. This data must be safe, but also controlled. So what does this policy do? Its primary goal is to check if the content of a payload is toxic, i.e. whether it’s offensive or disrespectful.
The Toxicity AI policy uses the Perspective AI engine; the configuration is very simple. A user must provide an access key to the Perspective AI and the path of the payload that contains the content that needs to be verified.
Here are two examples of inputs and responses to an API that applies the Toxicity AI policy:
- Input 1: “You are a stupid man, a real idiot! You will never be successful in your life, and you will always be a loser!”
- Response: The AI determines that the content input is highly toxic, arriving at a value of nearly 93% toxicity.
- Input 2: “You are a fantastic man [and] very intelligent! It’s a pleasure to meet you.”
- Response: The AI determines that there is no presence of toxicity, marking a less than 1% possibility of toxic content.
A future evolution of this policy might include masking words with characters that the AI should identify as offensive, or “toxic”.
API management tools play a crucial role in enhancing security measures
There are some AI vendors that offer complete security solutions for LLMs, such as Salesforce’s Einstein with its Trust Layer, that acts as a safety layer for users. When developers create secondary AI applications from platforms like Google Cloud or Open AI, they may prioritize features over security. This can present a challenge for organizations that want to integrate new AI technologies quickly without compromising security or managing new APIs.
Luckily, there are solutions available that can provide the necessary API protection, governance, and management to address this issue. With tools like MuleSoft Anypoint API Manager and Anypoint Flex Gateway or Mule Gateway, IT teams can configure and enforce security policies to monitor and control the flow of information in and out of LLMs, safeguarding sensitive company data and personal information.
