How Does Integration Enable AI? 3 Things You Need to Know

MuleSoft has evangelized the importance of integration in supporting AI for many years. In 2018, MuleSoft founder Ross Mason encouraged companies to take the AI brain “out of the jar”. Ross explained that siloed data stores and a lack of connectivity between enterprise applications severely restricted AI’s ability to influence the digital ecosystem around it, rendering it little more than a rather expensive brain in a jar. 

Fast forward to 2024, and rapid advances in AI have now pushed it into the mainstream. With breakthrough large language models (LLMs), the industry has made great strides in the intelligence of AI brains. What has not changed is the challenge of getting these brains to interact with the world around them. Now it’s as if we just have more expensive “brains in the jar”. 

Enterprises looking to unlock the power of generative AI will need to invest in integrating and securing the data assets around their AI brains. Yet, one of the biggest roadblocks to the adoption of AI is integrating existing systems together, not to mention the added security and privacy concerns. 

1. Fine-tuning LLMs on their data sets in order for them to understand their business 
2. Prompting trained LLMs with a specific business context (so they can generate accurate insights) 
3. Initiating actions based on these insights 

Integration helps not only in the assembly of the data required by each of these capabilities, but also by doing this in a way that ensures the confidentiality and integrity of enterprise data while maintaining privacy and compliance with customer data.

1. Fine-tune LLMs with the latest enterprise data  

LLMs learn through a number of training techniques including pre-training, fine-tuning, and reinforcement learning from human feedback (RLHF). Most LLMs will come pre-trained for specific tasks such as customer service or legal review. 

But to be effective an LLM must be fine-tuned on your organization’s business data. This is analogous to the process of loading your preferences into your mobile phone after you pull it out of the factory packing. 

Enterprises are challenged in providing LLMs the most up-to-date data from their enterprise systems. One of the biggest issues in this area is the ability to provide information to the LLM in a way that complies with the relevant security and privacy laws of the jurisdictions that the enterprise is operating. 

Most countries have unique privacy laws that dictate how personal information should be handled and stored. For example the European Union’s General Data Protection Regulation (GDPR) grants individuals the right to access, rectify, and erase their personal data. Today, it is very difficult to remove or obfuscate information from an LLM. Even one request to remove information from an LLM would require a costly retraining of the LLM. A similar challenge exists in the domain of intellectual property and LLMs. 

APIs and composable architectures, such as those provided by the MuleSoft Anypoint Platform, provide a governance layer around your enterprise data. This governance layer works by redacting personally identifiable information (PII) from data sets. 

Additionally, you can apply rules to APIs that audit the flow of data across physical and geographic boundaries as data flows out of systems and into the target LLMs. This allows enterprises to interact with models in a way that complies with local and federal regulations. 

2. Balance data access and security through prompting 

Once an LLM has been fine-tuned with compliant data, we can now begin to use it to support our customer scenarios. This is done via prompting the LLM. When we prompt an LLM we don’t change the model, we simply structure the request of the model to get the model to be more specific in how it responds. 

Better prompt data means better responses from the LLM. This is where integration helps. In building a prompt, you might want to include structured information such as product ownership records from SAP, together with unstructured information such as emails from the customer. 

Integration via APIs to enterprise systems, such as SAP, and to unstructured data stores, such as S3, increase my ability to provide high-quality prompting data to the LLM, thus increasing the accuracy of my LLM responses and positively impacting customer satisfaction. Many customers who have implemented a composable architecture in their enterprises are finding themselves well-equipped to support LLM prompting.

Another challenge customers face is access control and data leakage from LLMs. To enable LLMs to accurately respond to prompts we may need to fine-tune the model with information of varying levels of confidentiality. 

While it may be necessary to fine-tune a model with confidential information, there is limited ability to constrain the model from giving up these secrets when prompted – LLMs don’t provide fine-grained access controls. 

In theory, with the correct prompting, any user can access any data. An API management layer can provide a security framework around LLMs to control access and prevent leakage of intellectual property to unauthorized parties. 

3. Automate actions to take LLMs to the next level

While today’s LLMs are conversational in nature, they are quickly moving toward becoming autonomous agents. Autonomous agents promise to enhance one of the key business benefits of AI: automation. 

Einstein Copilot is the new generative AI-powered conversational assistant available to all Salesforce applications. A service agent or sales representative can use Einstein Copilot to show an optimized work schedule. Moving forward, Einstein Copilot will work to automate many of the tasks in this schedule. The key to becoming an autonomous agent is the ability to access and work with tools. Once again, integration comes to the rescue. 

Most web-based tools we use today – from mapping software to payment engines – can be accessed via APIs. If we instruct LLMs how to use tools via APIs and combine this tool access with workflow capability, LLMs will be able to automate everyday tasks. Not only will an LLM be able to see my calendar, but it will be able to resolve conflicts as they arise and make intelligent decisions about the best choices based on my past behavior. 

Where the LLM is in doubt, it may surface as a prompt to resolve the conflict. This will be important as GDPR also empowers individuals with the right to object to automated decision-making. These human responses instantly become part of the contextual data set moving forward; this is an example of how humans and machines will work together to streamline business and improve outcomes for citizens and consumers. 

Break free from the jar

So there you have it right in front of you: a brain in a jar. Take it out of the jar, and fine-tune it on your evolving business data by integrating your enterprise systems. Now that the brain understands your business, ask it for insights. Build prompts that are rich with context from your enterprise data, and finally allow your brain to action these insights through access to tools and workflows that automate functions within your enterprise. 

Intrigued? Find out how MuleSoft can help pull the “brain out of the jar” and start delivering on the business benefits of generative AI.