When it comes to getting data in and out of Salesforce using Anypoint Platform, there are a number of different options. Typically, using just one of them won’t give you everything you need and you might have to combine a number of them to to achieve a complete solution. In this post, we’ll summarize 4 options – Realtime, Custom, Bulk, and Data Virtualization – when to use which, and things to keep in mind for each.
Real Time
In terms of real time communication with Salesforce, it’s important to keep in mind that there are huge differences between how to handle inbound and outbound communication. Inbound communication is covered completely by the core API, which means that it’s also completely covered by MuleSoft’s Anypoint Connector for Salesforce. Salesforce limits the number of API calls you can execute per day according to this table. Due to this polling, it’s not a great option for getting data out of Salesforce. This is addressed by offering outbound messaging and the streaming api.
Inbound
Salesforce Core API
The core API is the normal Salesforce API that allows you to perform CRUD type operations. The Anypoint connector fully supports the Salesforce Core API.
- This is useful when…
you are interacting with Salesforce as part of an orchestration or automation process and you need to perform CRUD type operations against Salesforce. For example, if you have a case in Salesforce that has been resolved in another system such as Jira and the case needs to be closed in Salesforce. - This might not be the best option when…
you need to update a large amount of objects in one go. The core API only supports updates to 200 objects at a time. Also, if it doesn’t make sense to maintain duplicate data in Salesforce at all, a better option is to consume external services using Apex or using Lightning Connect.
Outbound
Outbound messaging
Salesforce outbound messaging is being configured as part of a workflow defined in Salesforce and it’s a callout to an external SOAP service defined by Salesforce. When configured in Salesforce, a WSDL will be generated that is available for download. It’s then up to you to implement and expose a SOAP service that Salesforce can make the callout to. From a MuleSoft perspective, the way to handle this is to implement that service using the CXF module or as XML over HTTP.
- This is useful when….
there is an transactional event Salesforce that needs to be processed, such as an opportunity being updated to “closed won”. The main benefit with outbound messages vs other options is that it has retries built in; so if for some reason the receiving application is unavailable, Salesforce will retry and report as an error if the defined retries have been exhausted. - This might not be the best option when…
you don’t have the ability to implement SOAP services. MuleSoft offers complete support for exposing SOAP services. However if the event triggering the callout is more informational and does not require an acknowledgement or retries, then the streaming API might be easier to implement.
Salesforce Streaming API
The way that you use the streaming API from a Salesforce admin perspective is that you create an instance of the entity PushTopic. In that PushTopic, you define a condition for when an event should be published to that topic. For example, when an opportunity goes to state “closed won”, an event will be published to a specific PushTopic, then you can have zero to many subscriptions to that push topic that will get notified directly once it happens.
- This is useful when….
you want to be notified asynchronously when a condition in Salesforce is fulfilled and there is no need for retries. From an Anypoint Studio development perspective, this is also a bit easier to setup compared to outbound messaging. - This might not be the best option when….
you need to make sure that the message is not being lost even if the recipient is not available. Since this is a publish subscribe exchange pattern, the message will be lost if your client/subscriber is unavailable once the event triggers.
Custom
Salesforce offers a great set of features when it comes to customization – some of which are very useful for integration. This option does however require you to be familiar with Apex and how to do custom extensions of Salesforce.
Inbound
Apex REST Service
You can expose REST services from Salesforce using custom Apex development and by default these services will be secured using the normal session token mechanism used by the core API. You can utilize the parts of the Anypoint connector that deals with the login and session token management, and then just use normal Mule functionality to call the service using JSON over HTTP. Currently, the login method is not exposed in the connector but you can get around this by programmatically calling the method (example here). The next release of the Salesforce connector will have full support for this so that no custom code will be needed.
- This is useful when….
you already have developed Apex services in Salesforce and you want to utilize the Anypoint connector to consume these services, or if you have very complex operations that needs to be executed inside the Salesforce platform due to performance reasons. - This might not be the best option when….
you are building services from scratch. A better option is to build and expose your APIs on a platform built for just that, like Anypoint Platform for APIs. This way, you have an abstraction layer for your API that is decoupled from the endpoint and being able to use the Salesforce connector to implement your API. Salesforce does not offer any capabilities around API Management, API Portals or API design.
Outbound
Consume REST Service using Apex
In the same way that you can expose a REST service using custom Apex code, you can consume one as well. You can tie that functionality to a custom button or any other Salesforce component. One example of this would be to add an available to promise check on the opportunity product object. In these cases, the Anypoint connector will not help you; however, Anypoint Platform for APIs would be central.
- This is useful when….
you want to display data that is not in Salesforce and does’tt make sense to maintain there and this needs to happen synchronously. - This might not be the best option when….
the data needs to be synchronized/duplicated in Salesforce rather than making a real time request.
Batch/Bulk
As the name suggest this is about moving large amounts of data in and out of Salesforce. This is typically done during the initial stages of a project where an existing dataset needs to be loaded into Salesforce.
Inbound
Bulk api
Using the bulk api which is fully supported using the MuleSoft Salesforce connector you can automate the upload of large number of records from a file, database or any other endpoint.
- This is useful when….
you have an export from a third party system and this data needs to be enriched or some more complex transformations need to be executed before it’s uploaded to Salesforce. - This might not be the best option when….
the data transformations that you need to do are very straight forward and there are no need for automation.
dataloader.io
dataloader.io is an online tool built for loading data into Salesforce. It’s a web application but at its core it’s powered by the MuleSoft runtime and connector. Currently it’s the most popular application on the Salesforce App Exchange.
- This is useful when….
you need to import csv (excel) formatted data into Salesforce. Dataloader provides you with a graphical UI that allows you to do simpler transformations.
This might not be the best option when….
you need to combine or enrich data from multiple sources before you upload it or if you need to do complex transformation or data aggregation.
Outbound
Salesforce Bulk api
Using the bulk api which is fully supported using the MuleSoft Salesforce connector you can automate the export of large number of records from sfdc as a csv or xml.
- This is useful when….
there is a need to have an automated export from Salesforce where the data needs to be inserted/updated in one or many endpoints on a regular basis. For example if you want to export the accounts from Salesforce and insert that to an ERP and a data warehouse on a scheduled basis. - This might not be the best option when….
when you need to do a one time data load from Salesforce to some other system that can accept a csv or a xml file as you get that functionality out of the box from dataloader.io.
dataloader.io
In addition to loading data into Salesforce dataloader.io can also be used to export data from Salesforce.
- This is useful when….
you need to do an export from Salesforce that does not require any complex data transformations and you can manually handle an export file. - This might not be the best option when….
you want to have a fully automated process that includes multiple endpoints and complex transformation.
Data Virtualization
Inbound
Salesforce1 Lightning Connect
Using the Anypoint Data Gateway you can read data from external systems such as SAP and databases within Salesforce in the same way you would be able to with any native entity. Currently the Anypoint Data Gateway only supports read but create and update is on the roadmap.
- This is useful when….
you only need to display an external entity and you don’t want to do any configuration or development at all. - This might not be the best option when….
you need to be able to update the data however this is being addressed in an upcoming release of Anypoint Data Gateway.
Thanks for taking the time to read. Leave a comment below and let us know what method you use the most!
