Today you’ll meet the newest member of our Training Talks series, Mark Nguyen. Mark joined the training team in November of 2016 as a Curriculum Developer, and will be a familiar face from now on! And yes, we have Mark’s fun fact too…are you ready?
Mark was part of the original team that launched Taco Bell’s Doritos Locos Tacos and, from what I heard, if you mention his name when ordering one, you’ll get an extra taco for free. (Just kidding, I tried it though!).
In this Training Talks segment, Mark will share some examples of flat files and show you how to import them into Anypoint Studio.
Demo Video
Instructions
Follow the demo step-by-step by reading below:
Flat files are fixed with text files that have hierarchies. You often find these flat files exported from legacy database systems. For example, this flat file (refer to video) contains account information for our customers. Line two contains the information about the account, while lines three and four contain detailed transaction information for the account.
How does Anypoint Studio process flat files? Anypoint Studio uses a file formatted in YAML. The YAML file describes data in a hierarchal way. The most important thing to note is that every column of data in a flat file can be mapped to a defined element in the YAML file. You can find more information on how to structure your YAML file by visiting the MuleSoft Documentations site.
Another important fact about flat files are the segment IDs associated with each line. These segment IDs help us effectively identify what type of data we’re processing. Anypoint Studio will look at the segment ID and match the ID with the segment IDs we define in our YAML file.
Anypoint Studio can also handle multi-level flat files as well. We can see here that for the account of Georgia Harbor, we have a call history that contains a list of past calls this customer made. The most important message is this: segments can have nested and repeated childs and elements.
Next, I’m now going to show you how to read a flat file and convert the data into a JSON structure in Anypoint Studio.
Let us first look at the files that we’re dealing with in Anypoint Studio. The first file that we’re dealing with is “customer.ff.” This is the flat file that will contain information for our customers, and our goal here is to convert this flat file into a JSON structure.
The second file that we need to also have is our schema definition. This is in YAML format, and this is how we will define and instruct Anypoint Studio to process our flat file. In this process, we will create an application that accepts file input, transform it, and then output the JSON structure into our console using a longer component.
First, let us set up our file connector. I’m going to process any files that show up in resources/input, process them, and then output the file to resources/output. Next, let us find our transform component. In the left-hand side, we are going to first define the metadata and add a customer type. This type will be a flat file. At the top, we will choose flat file and choose the “customer.ffd” as our schema definition.
As you can see, Anypoint Studio automatically pulls the schema definition and creates the appropriate metadata for our flat file. For the next step, we are going to change the input-output directive from application Java to application JSON, and we are going to conduct straight mapping. In the body of the DataWeave code, we will input payload.
Now, I want to do a preview to see how this code gets processed and what its output is. We are going to click “create required sample data to execute preview.” It will then expect us to provide the transform message sample data. To do this, we will take this information from the “customer.ff” then load the information, paste it, and go to the right-hand side and click on “preview.”
We’ve already confirmed that this code will convert our flat file schema into a JSON schema. We have three objects called end file, header file, and data, which is an array that contains outgoing objects for each of our accounts, with nested information as well.
The last step for us is to add a logger and output the message payload. We save our file and run it, then we can test it out. Let us drag and drop “customer.ff” to input, and we will be able to see the application transform the flat file into a JSON structure.
In summary, flat files are fixed width files within hierarchies. Every column of data can be mapped to an element in a flat file schema definite. Once you define the schema definition, processing the flat file in Anypoint Studio is easy.
Have questions for the Training team? Drop it in our forum or Training Twitter using the hashtag #TrainingTalks and stay tuned!
