Message Queue

Introduction to Enterprise messaging (Basics) with Message Broker

Udara Wijeratna Leave a comment

With some industrial experience on Message broker related SOA integrations, thought of discussing about Enterprise messaging. Let’s start with the basics and later in the coming post let’s discuss how we can have a high available messaging architecture.

Messaging

One mechanism which distributed applications can communicate is via message passing. Before we move ahead let’s check what “message” means.  A message is a structured piece of information sent from one agent to another over a communication channel. Message will have an identifier within it and it will tell the receiver the purpose or type of the message.  We can use different protocols and network standards such as HTTP, SSL, TCP/IP to do message parsing between applications.  However when we talked about enterprise messaging with message brokers, we talk about transmission control protocol. Another important fundamental concept we should know in messaging is the message handling methods. Namely Asynchronous and Synchronous. So let’s discuss it.

 

Asynchronous vs. Synchronous message handling

One critical thing we need to consider when planning message parsing or processing is to know whether it needs to be Asynchronous or not. First let’s see the difference between the two types.

  • Synchronous Messages:

    Synchronous messaging involves a client that waits for the server to respond to a message. Messages are able to flow in both directions, to and from. Essentially it means that synchronous messaging is a two way communication. i.e. Sender sends a message to receiver and receiver receives this message and gives reply to the sender. Sender will not send another message until it gets a reply from receiver.

 

  • Asynchronous Messages: Asynchronous messaging involves a client that does not wait for a message from the server. An event is used to trigger a message from a server. So even if the client is down, the messaging will complete successfully. Asynchronous Messaging means that, it is a one way communication and the flow of communication is one way only.

Depending on your business process need you can choose one of the above methods in your communication channel. Now let’s discuss about the message broker which handles all these message flows between different applications.

 

Message Broker

A message broker is a middleware program that translates messages from the formal messaging protocol of the publisher to the formal messaging protocol of the receiver. Message broker decouples the publisher and consumer systems. The beauty of the message broker is that it can receive messages from a publisher and it can transmit it to multiple subscribers. In addition there are key advantages on using a message broker in the middle of a message communication. They are as follows:

  • Ability to store messages when the consumer is not available to consume the messages
  • Guarantee message delivery between two parties
  • Act as a load controller in the middle of an application to ensure the consumer is not overloaded with requests. All the requests will come to the message broker and it will store them until consumer, reads them one by one with the highest rate it can consume.

Now we know what is the message broker and the importance of it. Next we will look in to the two main subscription models in messaging.

 

Queues and Topics

As the message handling model, message subscription model also need to be selected based on the business process requirement.  Let’s see what we need to consider when selecting the model.

Queue Topic
Point-to-point model. Publish/subscribe model.
Only one consumer gets the message. Multiple clients subscribe to the message.
Messages have to be delivered in the order sent. There is no guarantee messages have to be delivered in the order sent.
A JMS queue only guarantees that each message is processed only once. There is no guarantees that each message is processed only once. As this can be sensed from the model.
The Queue knows who the consumer or the JMS client is. The destination is known. The Topic, have multiple subscribers and there is a chance that the topic does not know all the subscribers. The destination is unknown.
The JMS client (the consumer) does not have to be active or connected to the queue all the time to receive or read the message. The subscriber / JMS client needs to the active when the messages are produced by the producer, unless the subscription was a durable subscription.
Every message successfully processed is acknowledged by the consumer. Every message successfully processed is not acknowledged by the consumer/subscriber.

 

Next let’s investigate on how message broker can exist in enterprise SOA.

 

Messaging Topologies

  • Internal Broker:

    J2EE application server contain message broker (embedded with JMS enabled application server webshere / weblogic / jboss ) and external applications call the message broker. Reliability and performance concern since its inbuild in the application server additional overhead to server resources. Messaging required threads and CPU for processing. Application server down then message broker is also down. Worthy when our queues not exposed to external applications.

 

  • External Broker:

    Fully fledge message broker solution. Message broker is an external server. Activemq, Sonicmq, Mulemq are some examples for those external brokers. From those we can achieve clustering, fault tolerance, scalability

 

  • Embedded Broker:

    application with message broker solution in build with it. Source code of your application start the message broker. Message broker part of the applications. It will be a truly localized message broker for an application.

 

Now we have discussed all the aspects related to message broker and enterprise messaging. In the next post we will discuss on how we can setup an External Broker Architecture with high availability and fault tolerance.

 

References

  • http://peoplesofttutorial.com/difference-between-synchronous-and-asynchronous-messaging/
  • Java Distributed Computing by Jim Farley
  • ENTERPRISE MESSAGING WITH JMS by Mark Richards
  • http://www.ebizq.net/topics/eai/features/1590.html
  • JD Edwards EnterpriseOne Transaction Server Reference Guide

Read More

What makes an Enterprise SOA?

Udara Wijeratna Leave a comment

In this post we will identify the key tools or components that work together in an Enterprise Service Oriented Architecture solution.

In Enterprise SOA world there are key components work together to ensure SOA solution work as expected. Let’s identify them and have basic understanding of those components. And In coming posts deep it to those components.

 

As we can see in the above diagram we can list the key components as follows:

Application Server

(Apache Tomcat, Oracle WebLogic, GlassFish, JBoss, IBM WebSphere, Jetty)

In developer world it’s a server which we deploy our enterprise applications or services. However if we further look in to it, then It’s a server program which consist of three tiers.

  • Client Tier : It can be one or more applications APIs or browsers
  • Middle Tier : This is consisted with Web server and a EJB server
  • EIS (Enterprise Information System) Tier : Deployed applications, files and database

As mentioned above in the SOA world we deploy our web services, web applications in those servers. Some services may expose through API manager to public and some will use to develop the composite services which will make a proper SOA solution to the enterprise.

 

Message Broker or Messaging solutions

(RabbitMQ, ActiveMQ, SonicMQ, Kafka)

Message brokers comes in to play when we required reliable messaging communication between applications. The key concepts behind is Queues and Topics.

Queue: Point to Point model so, message will go to only one subscriber

Topic:  With Publisher-Subscriber model each message which publish to the topic will be transmitted   to all the subscribers who are registered to the topic

In addition some key features of Message Broker:

  • Ability to retain (Store) the messages when the receiver is not available / or slow to consume them
  • We can route messages one to many destinations
  • Being able to decouple the message publisher and receiver

 

ESB – Enterprise Service Bus

(Oracle Service Bus, WSO2 ESB, Biztalk, Mule ESB, IBM Websphere ESB)

ESB is one of the main component in ESOA solutions. If someone ask you to give a one term to describe usage of ESB in SOA context, then it should be as a “Mediator”. Let’s find out why we call ESB as a mediator.

The core concept of ESB is to integrate different applications (Services) using an ESB as the middle man (mediator) to each of those applications to communicate with each other. This covers one of the core concept in SOA, with lose coupling of services and ability to interact with each other, Increases organizational agility by reducing time to market for new initiatives.

Key features of ESB are follows

  • Message routing between services by reading the content of the messages
  • Message transformation (ex: SOAP – REST and vise versa)
  • Transport protocol negotiation between multiple formats (such as HTTP, JMS, JDBC)
  • Allow to configure security and monitoring policies for services

 

Registry

(WSO2 Governance Registry, Oracle Registry)

The registry is an information catalog that is constantly updated with information about the different services in a service-oriented architecture project. It helps to govern the SOA solution by storing, cataloging and indexing metadata related to services, so it can be easily manage / governed using this component.

SOA registry support UDDI specification and it is the main component in SOA governance. However with emergence of API Manager, the Registry component is slowly moving out from the SOA component set.

 

API Manager

(WSO2 API Manager, Apigee, Postman, Azure API Management)

In the current SOA component stack, API management component plays a major role. As we discovered in our earlier posts, that now most of all the applications expose services / APIs.  API manager is also capable of covering some registry features as well.

If we define API Management: API management is the process of publishing, documenting and overseeing application programming interfaces (APIs) in a secure, scalable environment.  The goal of API management is to allow an organization that publishes an API to monitor the interface’s lifecycle and make sure the needs of developers and applications using the API are being met.

Key Features:

  • Automate and control connections between an API and the application which use it
  • Monitor traffic of each exposed API
  • Add security policies for APIs
  • One repository to discover APIs in the Enterprise SOA boundary

 

BPEL Process Manager

(Oracle BPEL Process Manager, Apache ODE, ExpressBPEL, jBPM)

The key concept of SOA is construct services using other reusable services. To achieve that goal, we use BEPL Process Manager Component in SOA solutions. It holds a layer to orchestrate and combine multiple services to generate the task services which covers the business processes of the SOA domain. In BPEL manager we use BPEL (Business Process Execution Language) to define those complex orchestration processes. We would say BPEL Manager is the core component in Enterprise SOA solution.

Key Features:

  • It was initially based on XML schema, SOAP and WSDL. However now it support REST and JSON
  • Send and receive messages asynchronously from remote services
  • Manipulate XML, JSON data using XSTL
  • Manage events and exceptions in the process flow
  • Ability to design parallel flows
  • Compensate – Undo portion of processes when exception occur during the flow
  • Version control

 

BAM – Business Activity Monitoring

(Oracle BAM, WSO2 BAM)

BAM describes the processes and technologies that enhance situation awareness and enable analysis of critical business performance indicators based on real-time data. BAM is used to improve the speed and effectiveness of business operations by keeping track of what is happening and making issues visible quickly.

With all the components working together in Enterprise SOA solution, we can ensure a complete SOA solution running in our enterprise.

References

Read More