Andrew Ferrier

WebSphere ESB includes two distinct technologies, both of which use the name ‘mediation’:

• WebSphere Platform Messaging mediations, introduced in WebSphere Application Server V6.0 (and thus included in WebSphere ESB and Process Server).
• Mediation flows, the primary facility introduced by WebSphere ESB V6.0.1 itself.

Let’s come straight out and say it: yes, this can be a little confusing, given that these two technologies go by similar names. Nevertheless, although both of them can both be used to alter messages on the Platform Messaging bus, they work in quite different ways.

Platform Messaging mediations are specified as a sequential list of ‘mediation handlers’, administratively attached to a destination such as a queue or a topic space. These mediation handlers are programmatic: they are an implementation of the MediationHandler interface. This allows a message to be altered ‘in-place’: the fundamental properties of the message, such as the message ID, are not changed, because it is logically the same message. These mediations can also be used to destroy the message, or route it elsewhere. For more information, see Dan Murphy’s article in the IBM WebSphere Developer Technical Journal.

WebSphere ESB mediation flows, on the other hand, do not have to be a simple sequential list of steps. They are a network of mediations (in the ESB sense of the word). These mediations are generally re-usable components designed to modify a message in a certain way or make some decision based upon its contents (ESB ships with several out-of-the-box). The flows can have several potential paths from start to finish. Generally, when dealing with Platform Messaging messages, JMS exports and imports will be used to bring a message into the flow and output it from the flow. This means that a message will be destroyed when it is brought into the flow, and re-created from scratch when output – it is not the same message. This is partly because ESB mediation flows can also have other types of import and export, such as those which support web services. When a message is travelling through the flow, it is actually represented as a Service Message Object.

These two technologies are therefore quite different. In general, you would use a Platform Messaging mediation when you want mediation to be transparent to an already-existant application (mediation happens after a message is put to a destination but before it is recieved). ESB mediation flows offer a richer and easier-to-use set of functionality when that’s necessary, but don’t preserve message transparency. They also provide the facility to integrate with web services and other data sources.

WebSphere Application Server (WAS) has a variety of ways of defining servers, and their relationships to each other. These are often called WAS topologies. Let’s revisit some of the WAS topology concepts from a WebSphere ESB perspective (although many things you may know or learn about WAS topologies probably apply equally to WebSphere ESB, and vice-versa, because ESB is built on top of WAS).

There is a hierarchy of objects in an ESB topology:

• Cell – This contains one or more nodes. Cells are completely independent of each other from a topology perspective.
• Node – This contains zero or more servers.
• Server – This is a physical server (we’ll be talking primarily about application servers here). There is a 1:1 mapping between a server and an OS process.

An ESB installation can have one or more profiles, which define nodes in a topology (in other words, there is a 1:1 mapping between profiles and nodes). These profiles can have three types:

• Stand-alone
• Deployment manager
• Custom

These profiles and the overall topology are orthogonal to ESB installations. An entire topology (with a variety of profile types) can be run from a single installation, or each profile can be part of a separate installation. To further confuse matters, a physical machine can have one or more ESB installations (although typically it only has one).

A stand-alone profile is easiest to understand. This defines a single node, which exists in a single cell. The single node contains a single default server. If you install ESB using the ‘Complete’ option, you will get a profile created of this type – called ‘default’, containing a server called ‘server1′ (the node and cell name will be some permutation of the hostname of your machine). Administration is done through an administrative console attached to the node.

Alternatively, you can set up a more complex configuration. If you create a deployment manager node, you can use this to manage other nodes. Typically those other nodes start out as custom nodes. When you ‘federate’ them to a deployment manager, they become part of the deployment manager’s cell, and a special type of server called a ‘node agent’ is created on the custom profile. Often this federation is done when the profile for that node is created. This federation allows configuration information to be shared between nodes – the administrative console used is now part of the deployment manager node, and configuration information is synchronised according to a schedule, or on demand. Resources (for example, JDBC connections), can be created at ‘cell’, ‘node’, or ‘server’ scope, and can only be seen at that level. Application servers also need to be manually created for custom nodes – they don’t contain any by default. Cells can only contain one deployment manager.

I plan to write a Part 2 on this topic soon, covering clustering. Watch this space…

So it’s Christmas morning, and you’ve unwrapped the shiny paper, and you see that Santa has left you a copy of WebSphere ESB 6.0.1. You’re keen to get going on developing your first SOA solution, but you’re a little bemused by the ‘Message Service Clients’ inside the jiffy bag, on CD2. Well, here’s what they are:

• IBM Message Service Client for C/C++ (aka C/C++ XMS) – this is available for Windows and Linux.
• IBM Message Service Client for .Net (aka .Net XMS) – this is available for Windows only.
• IBM Web Services Client for C++ (aka WSCC, CWSS) – this is available for Windows and Linux.

These packages allow you to build client code that can be used against a WebSphere ESB system (as well as other messaging systems).

The XMS clients are essentially re-implementations of the JMS API in their respective programming environments: C/C++, and .Net. They are designed to match the JMS API as closely as possible, and are thus useful for client developers who are already familiar with JMS. The clients can work with three different JMS provider families:

• WebSphere MQ
• WebSphere MQ Brokers
• WebSphere Application Server Platform Messaging (included in ESB).

These families use different wire protocols for the JMS messages, so the family being used has to be specified in the client application. Otherwise, though, the API is identical.

The WSCC client is slightly different: it allows one to write a web services client in C++ from an existing WSDL web service specification (such as one produced by ESB when one uses a Web Services Export in a mediation module, for example). Typically this is done using the supplied WSDL2Ws tool to create stubs that can be used to create the client.

The clients are also available as IBM SupportPacs for download, although you’ll want to check the licensing conditions for your support status when using these.

(Note: the ‘Application Client for WebSphere Applications Server’ is also shipped with WebSphere ESB, although, as the name implies, this isn’t specific to ESB – it’s the same client available with WAS).
For more information, see:

• The excellent article on IBM Developerworks – Introducing XMS.
• The comprehensive documentation for the WSCC client – the file wscc.pdf in the docs/ directory where the client is installed.

I still meet a lot of people who are confused by the term ‘WebSphere’. Let’s clarify, as it has two oft-used meanings. The first, which IBM encourages, refers to the WebSphere brand of software. This comprises many products (probably over 100 once all variations are taken into account), most of which are somehow related to middleware or integration. Included in this brand is the product WebSphere Application Server (WAS), upon which several of the other products are based. The second meaning of the word is as a synonym for WAS, and is popularly used outside IBM (and sometimes inside, when IBMers are feeling lazy). If someone asks ‘do you know WebSphere?’, they are almost certainly using the word to refer to WAS specifically.

For more information, see the Wikipedia entry for WebSphere.

I recently had a comment attached to one of my posts regarding ESB suggesting that it needed to be translated into English – so that’s what I’m going to attempt to do here. An ESB is a fairly new and complex concept, and the definition is far from fixed, so don’t be surprised if you see some disagreement. As with many things, though, it’s also a generalisation and aggregation of some other concepts that already exist, so be aware of that too.

An ESB is a fairly abstract architectural idea, but I’d best describe it as ‘a place where messages are stored, routed, modified, created, and destroyed’. It’s probably easiest to describe this with some history (so apologies if you know some of this already), and I’m going to use IBM’s products as examples, simply because they are what I know best.

Enterprise messaging has been around for some time. IBM’s classic product in this area is WebSphere MQ (formerly MQSeries). MQ runs as one or more servers, which contain queues. Messages can be placed on queues by applications (originally using MQ’s own MQI API, now using industry-standard JMS). Other applications can retrieve messages from the queues using the same APIs. This allows data to be easily moved asychronously between platforms, systems, etc. One of the key points about products such as MQ is the way that the messages are treated – typically with ACID properties similar to the way databases treat data. This means you can be confident that once a message has been sent, it will eventually reach its intended destination (or somewhere that it will be noticed). This provides a reliable base for integrating different applications (the original attraction of messaging systems like this). Originally, messages were typically in a proprietary format, but they are often now in XML with an associated Schema.

It was shortly realised that it would be useful to add some form of brokering. IBM’s product in this area is called WebSphere Message Broker. Typically, it is used to build on the capabilities of MQ. It allows you to develop (normally without programming) ‘flows’ that pick messages off queues, transform them, route them, etc., then place them back on other queues. Message Broker also allows one to work with adapters, meaning that flows can input from, or output to, things that aren’t queues. In this respect, Message Broker is essentially a generalised MQ application.

Later on, IBM released a new messaging engine, similar in some respects to MQ, as part of WebSphere Application Server (WAS). It is formally called WebSphere Platform Messaging and is part of the J2EE standard which WAS conforms to. Many other application servers also exist, with their own JMS engines, some open-source.

In recent years, web services have started to come to prominence, particularly as part of creating a Services-Oriented Architecture (SOA) for an organisation. They utilise a slightly-different, but related, model for integrating applications from that of enterprise messaging. Typically, an application makes a synchronous request over HTTP, using a message format called SOAP, to a back-end service, which does some work for it. Many services send back responses. These services typically run on application servers.

WebSphere ESB brings together many of these concepts. It contains all the functionality of WAS (including the messaging engine), but also provides ‘mediation flows’. These are similar to flows in Message Broker in that they allow one to pick up messages from JMS queues and modify them, audit them, etc., before passing them on. However, it also allows mediation of Web Services: for example, one could invoke a web service provided by ESB, which invokes another web service, and ESB then does some processing on the result before returning it to the original invoker.

So now we are in a position to define an ‘ESB’ in the generalised sense – in all these interacting systems, it comprises the messaging engines and the brokering and mediation capabilities. The applications that send and recieve messages, provide and invoke services, are those that sit outside the bus. As such, one doesn’t need any particular product to have an ESB (including WebSphere ESB). ESB is often seen as a pattern rather than a product: just a statement that the bus and the applications are distinct. You could build one with MQ and Message Broker (or even MQ alone if you didn’t want brokering capabilities). WebSphere ESB simply provides extra mediation capabilities, including the ability to mediate web services, without doing a lot of leg-work.

It’s worth clarifying that an ESB doesn’t necessarily have to have anything to do with multicast (although some ESB systems do include the notion of splitting and aggregating messages, which has some similiarities).

This is a large and complex topic, so it’s hard to do it justice in a blog post, and I’ve simplified vastly on some topics above. In particular, I haven’t really discussed any business process management. But I hope I’ve cleared some things up. If you’re interested in learning more, the following might help:

• Chapter 2 of SOA with an Enterprise Service Bus in WebSphere Application Server V6 – descriptions of both SOA and ESB.
• Understanding the ESB
• Message-Oriented Middleware
• Java Technology and Web Services

Alternatively, as always, please post a comment and I’ll do my best to answer it.

People are sometimes confused about the differences between WebSphere ESB (built on top of WebSphere Application Server, and using the inbuilt WebSphere Platform Messaging JMS provider) and WebSphere Message Broker (which uses WebSphere MQ as a messaging engine). IBM sometimes describes the latter as ‘Advanced ESB’, but Message Broker is not a superset of the functionality in ESB. There is a good FAQ on the IBM website which clears up some of the confusion.

In general, Message Broker is designed for working primarily with WebSphere MQ, as well as having a larger set of nodes (or mediation primitives, in WebSphere ESB-speak). WebSphere ESB has a richer set of functionality with respect to SOAP and Web Services (Message Broker treats SOAP as plain XML). ESB is also built on SCA, which allows it work more easily with products such as WebSphere Process Server.

ESB and Message Broker can work together: using the MQLink functionality in WebSphere Platform Messaging, ESB can simulate an MQ server and exchange messages with MQ accordingly. They can be complementary and both form part of an SOA solution.

Advantages of using synchronous behaviour in an ESB (such as SOAP over HTTP):

• You want your clients to be simple to write – with asynchronous behaviour, when a reply comes back to a client, it needs to know how to correlate that with the request it sent out (this is one reason JMS has the concept of message and correlation IDs). This probably requires it to keep some state hanging around. This isn’t necessary with synchronous behaviour, as the client thread blocks waiting for the reply.
• You want your system to be fast (in the general case). In general, you would expect synchronous behaviour to incur less overhead.

Advantages of using asynchronous behaviour in an ESB (such as JMS):

• You want to ‘send and forget’. Asynchronous generally makes more sense for one-way operations than synchronous behaviour, for this reason.
• You want the system to scale more cleanly when there are lots of outstanding requests from various clients.

The above observations are drawn from work with WebSphere ESB, although the general principles probably apply elsewhere.

SOA (Services-Oriented Architecture) is a very general idea; the notion of defining business and computing logic as interconnecting services, typically in a distributed computing environment, where some services depend on (call upon) others. SCA (Service Component Architecture) defines some terminology and standards for this, using the concept of SCA modules – you can think of each module as being a black-box service. SCA modules can import services from other SCA modules (which have corresponding exports, defining the services they provide). They can also expose (export), and call upon (import) services via other protocols.

Inside an SCA module is a similar set of constructs to that of the interconnected modules; in the ‘middle’ there are SCA components, which reference other SCA components (which have interfaces). On the ‘left-hand-side’ of the module (it is indeed drawn on the left-hand-side in WebSphere Integration Developer) are the exports, and on the ‘right-hand-side’ are the imports. The exports typically connect to some components’ interfaces, and imports connect to some components’ references.

In WebSphere Process Server and ESB, imports and exports are one of four types:

• SCA ‘default’ binding (it is perhaps helpful to think of this as the SCA ‘native’ binding). This binding is used to link two SCA modules together. It supports all valid SCA behaviours, and its implementation is system-dependent. SCA bindings can also only link modules within the same address space (in WebSphere Process Server and ESB this means on the same server) [correction: this is not true, please see this update], are easy to set up (just specify the other module’s name on the import), and are typically high-performing.
• SOAP over HTTP binding – A web service is invoked (import) or exposed (export). These are synchronous only, by the nature of the transport.
• SOAP over JMS binding – A web service is invoked over JMS (import) or exposed (export). These are asynchronous only, by the nature of the transport.
• JMS binding – Data is sent to a JMS destination – i.e. a queue or topic (import) or recieved from a destination (export). These are asynchronous only, by the nature of the transport.

In WebSphere Process Server, SCA modules typically contain some of these types of components:

• BPEL
• State Machine
• Business Rules
• Human Task
• Selector
• Interface Maps
• Java

WebSphere ESB adds an additional type of SCA component, a Mediation Flow. This can only be embedded in a special type of SCA module, called a Mediation Module (which can only contain one Mediation Flow). Mediation Modules can be run on WebSphere ESB and Process Server, but ESB is only designed for mediation, and does not support Modules containing the Process Server components above (such as BPEL).

You can find out more about SCA in Process Server by reading this excellent article on IBM Developerworks.

Simply stated:

• Use WebSphere Process Server as much as possible, and WebSphere ESB as little as possible.

Process Server provides a layer to implement a business process – this is (or should be, or could be) ‘interesting’ to a business. ESB simply provides the glue to tie things together – auditing, converting one interface to another, etc. So don’t fall into the trap of faking Process Server functionality in ESB, when Process Server provides it all already.