LegStar for Mule Patterns

motif

Introduction

A pattern is a generic solution to a generic problem that is likely to occur over and over again. Patterns, for the purpose of this article, form a language that system designers can use like recipes: “if you find this type of problem, then you can apply this type of solution”.

In the domain of applications integration, patterns are particularly helpful. Application integration is complex, it typically involves several different systems, developers with different backgrounds, different organizations, etc. As a result of that complexity, the problems to solve are many. The introduction of Enterprise Integration Patterns (EIP) by Gregor Hohpe and Bobby Woolf [1] is an excellent coverage of patterns for application integration. We will refer extensively to EIP in this article.

Patterns have become so important in application integration that integration products, such as enterprise service buses (ESB), are adopting them, not only in their documentation, but also as building elements. You configure your system by assembling patterns. One such pattern-enabled ESB is Mule [2]. The latest version of Mule offers a sophisticated, Spring-based, configuration system that nicely maps several EIP patterns to configuration constructs.

Mainframe application integration on another hand, is a sub domain of application integration with its own complexity. Here, applications were written a long time ago using languages such as COBOL and transactional systems such as CICS. In this article we examine how some integration patterns from EIP can be implemented when mainframe applications are involved.

Because we are concerned with actual implementations, rather than discussing the patterns themselves, we focus on using a combination of Mule and LegStar. LegStar [3] is an open source mainframe integration solution . It has a module specifically designed for Mule [4]. We use Mule configurations, with LegStar elements, as actual implementations of the patterns discussed.

We will expose a few common use cases in mainframe integration, model the proposed solution using patterns from EIP and then show the Mule/LegStar configuration elements that actually implement the selected patterns.

Use case 1: Data originating outside the mainframe must reliably become available on the mainframe

Typically a situation that would arise if a front end loan broker system sends loan requests to a backend mainframe-based banking system. Communication is asynchronous by nature (response might take several days to come back) and needs to be reliable (we don’t want to lose loan requests). On the mainframe, the approval processing application frequently involves a user interface.

Patterns

Use Case 1 patterns

In this diagram, symbols for patterns are from the EIP book. In the following text we enclose pattern names in double quotes.

A “Document Message” holding the loan request description (a Java object) enters Mule on a client “Message Endpoint” and then goes into a “Message Translator” that converts the message content from Java to mainframe data. The translated message is then presented to a “Guaranteed Delivery” point to point “Message Channel” to send the data over to the mainframe.

On the mainframe, an application consumes messages that come into the loan requests “Message Endpoint”. The mainframe application picks up requests at its own pace. Typically a user starts working in the morning and looks into a queue of requests that might have arrived during the night.

Mule configuration

Let’s now see how this might translate into a Mule configuration (for the sake of brevity we do not include namespaces and connector definitions).

Here we assume clients will be sending their documents as Java objects within JMS messages and connectivity to the mainframe is via WebSphere MQ:

The “Message Endpoint” patterns are mapped to <inbound-endpoint> and <outbound-endpoint> configuration elements.

The “Message Translator” pattern becomes the <legstar:java-to-host-transformer> element from the legstar namespace. This element refers to the com.test.LoanRequestToHostMuleTransformer class. This code is generated using LegStar and performs Java to Mainframe conversion. This is how that works:

The LegStar design time tool parses COBOL source that describes the data format expected by the mainframe application. It then generates the LoanRequestToHostMuleTransformer code which expects a Java object as input and produces mainframe formatted data as output.

The following diagram symbolizes such a transformation process:

Use Case 1 Java to Host

The loan broker client sends a LoanRequest value object to Mule over JMS to initiate the use case described here.

The “Guaranteed delivery” channel is implemented using WebSphere MQ. Here we use the standard Mule JMS capabilities to talk to WebSphere MQ but we could as well use the more advanced WebSphere MQ support that comes with Mule Enterprise Edition.

Use case 2: Data originating on the mainframe must reliably trigger an action in Mule

As an example this could be the loan request approval (or refusal) data that a mainframe application needs to send back to the loan broker system.

Patterns

Use Case 2 patterns

A mainframe application uses a “Guaranteed Delivery” point to point channel to send an “Event Message” to Mule. Upon arriving on the loan replies “Message Endpoint”, the message holding mainframe data goes into a “Message Translator” that converts the message content to a Java object. The translated message is then presented to a “Service Activator”, which in turn runs a Mule “Service”.

Mule configuration

The Loan replies “Message Endpoint” which is now an <inbound-endpoint> from a Mule perspective, maps to a WebSphere MQ queue. Again we use the standard Mule JMS capabilities to listen on that queue.

When a message arrives, it is extracted from its JMS envelope and presented to the <legstar:host-to-java-transformer> “Message Translator”. That transformer refers to the com.test.HostToLoanReplyMuleTransformer class that is generated exactly like its counterpart in use case 1.

The following diagram symbolizes the transformation process:

Use Case 2 Host to Java

The “Service Activator” pattern maps to the Mule <component> configuration element. In this case, the “Service” is a simple POJO called com.test.LoanReplyService. A method of this POJO expects a LoanReply object such as the one produced by HostToLoanReplyMuleTransformer.

Use case 3: Mule service needs asynchronous responses from the mainframe

In this example we assume the mainframe implements a service that is needed as part of a workflow on the Mule side. This could be a credit scoring function for instance that is needed before a loan is authorized. Let us assume a mainframe program takes as input a customer social security number and replies with credit scoring and credit history length.

How can we make that function available to the loan broker application?

Patterns

Use Case 3 patterns

The loan broker request gets into Mule via the client request “Message Endpoint”. The “Document Message” content is then transformed into mainframe format by a “Message Translator”.

The next steps are needed because we intend to use a generic program activator on the mainframe. That activator needs to know which mainframe program to run and where to store the reply.

We first enrich the message with a header that holds the target mainframe program name and other related attributes. We do this via the “Envelope Wrapper” pattern.

The message is further enriched using the “Return Address” pattern so that the program activator knows which “Message Channel” to use for the reply.

At this stage, the message is a “Command Message” since it contains instructions on what program to run on the mainframe. It is ready to be placed on the program activator “Message Channel”.

On the mainframe, the program activator acts as a “Service Activator” and runs the target mainframe program as a “Service”.

Upon return from executing the mainframe program, the reply message will go through a symmetrical number of steps to remove headers and translate data back to Java.

Mule configuration

Since the entire flow is asynchronous, it is implemented as 2 Mule services. One that serves requests and the second one that serves replies. Of course Mule offers other possibilities to handle asynchronous request/reply patterns but we wanted to keep examples in this article as simple as possible.

The various “Message Endpoint” patterns are reflected in the usual endpoint elements in the Mule configuration. The “Message Translator” patterns are again LegStar Transformers which were generated specifically for the target mainframe program input and output structures.

Our target mainframe program is a CICS program. Our mainframe program activator is IBM CICS MQ Bridge [5] (a standard feature of CICS TS 3.2).

The CICS MQ Bridge “Envelope Wrapper” is performed by the <legstar:host-to-mqcih-transformer> element from the LegStar namespace. Note the attributes that designate the target CICS program and the maximum output data length. This same element also implements the “Return Address” pattern by adding a reply to address on the CICS MQ Bridge header.

In the Mule reply service, the <legstar:mqcih-to-host-transformer > element strips the CICS MQ Bridge header from the reply before presenting the message to the mainframe to Java translator.

Finally the client application can pick up replies from the client reply endpoint which maps to a JMS queue within Mule.

Conclusion

Although this is barely an introduction to the topic of mainframe integration patterns and their implementation, we hope that the few examples presented have shown:

  • The patterns potential as a communication tool for mainframe integration projects. Each compact diagram conveys a lot of information on how the system needs to be setup. Java developers, Mule administrator, WebSphere MQ administrator, CICS application developers and CICS system engineers can all share that same representation.
  • The advantage of using tools which configuration elements closely map to patterns. In our case for instance, once the correct patterns are chosen, they can be translated to actual Mule and LegStar configuration elements in a straightforward manner. We could as easily introduce aggregators, routers, etc.. thanks to Mule rich set of configuration elements.

Of course several other topics such as error handling, correlation, security, transactions and monitoring would deserve to be covered both in term of patterns and actual configuration elements. From a mainframe integration perspective, there is relatively few coverage of such topics so far. We hope this article will help start some useful discussions.

References

Authors

Fady MoussallamIs CEO of LegSem and has over 20 years of experience with mainframe applications and integration technologies

Links

[1]Hohpe and Woolf, Enterprise Integration Patterns
[2]MuleSource, Understanding the Mule Architecture
[3]LegStar open source mainframe integration
[4]LegStar for Mule
[5]IBM CICS MQ Bridge

We'd love to hear your opinion on this post


4 Responses to “LegStar for Mule Patterns”

  1. Excellent article Fady. The more mainframe integration documentation you can put out, the better IMHO. The fact of the matter is, the Main Frame is not going anywhere in the enterprises who have existing legacy systems. So the better we can integrate with it, the better our overall solutions will become.

  2. what Craig said. Mainframes are not going away, but they do speak their own language. You give a good explanation of the benefits of a common representation among the specialty skills that combine to build successful bridges.

    Patterns, working models and good examples lead to more robust solutions. Thanks for the LegStar project and for sharing your design with us.

  3. I am relatively new to the industry an by no means an expert but isn’t it true that legacy integration to many vendors is anything that was here before we arrived?
    I was reading you article on the Mule system and have been broadening my view of the industry and I wanted to know what makes integrating the mainframe with mule any better than any other brand/company out there? Are they all pretty much the same with different packaging and price. And do most come with the connectors you need to communicate between the applications and mainframe???

  4. Integrating mainframes with Legs4Mule rather than specialized servers has 2 major advantages in my opinion:
    First is the open source nature of the tools, LegStar for Mule is completely open source, including developers tools. It is free for download by anyone and is licensed under the liberal LGPL. I am not aware of any so called Connectors that are open source.
    Second are the economy of scale that can be achieved by leveraging an ESB core integration capabilities. Transforming XML to CSV is not different from transforming a mainframe data buffer to a java object; so why should we deploy such transformers differently or run them in different complex multi-threaded servers? With LegStar for Mule, developers and administrators have a single platform to learn: the ESB.