Enterprise Service Bus & Service Oriented Architecture

What is it?

An enterprise service bus (ESB) and service oriented architecture (SOA) are mainstays of enterprise integration and enterprise integration patterns. As the name indicates, we normally associate the technology with large enterprise implementations of integration processes and services. Historically, they are also technologies and approaches we associate with high costs in terms of licenses/subscriptions and their associated services.

The prevalence of standards around this though has led to a number of open source technologies being developed that drastically reduce the costs associated with licenses and subscriptions. The Apache foundation has a number of projects within its portfolio. There are many other vendor backed and community supported projects out there as well. This has significantly reduced the barriers to entry for smaller organisations wishing to deploy this functionality and technology.

Value proposition of the capability

We generally associate an ESB with classic enterprise integration patterns. Most products/projects are built to be compliant with them. This allows for a standard approach to delivery across all technologies. This should mean that solutions are relatively portable across the technologies. That isn’t always necessarily true in implementation, but it is common for a lot of an implementation’s artefacts to be reusable.

The value of an ESB is in the standards based approach to implementation. This enables the architecture and solution design to be abstracted from the technical implementation. Architects can focus on solution design in terms of a standard set of integration patterns. Service Oriented Architecture is well understood in the market which also means that resources are generally readily available in the market. Given we associate these implementations with large enterprise deployments, we also associate them with highly robust, mission critical and scalable services.

The advent of open source technologies in this space means that smaller organisations can leverage this approach to integration and the associated robustness and scalability of an ESB.

Common uses or use cases

We generally see ESBs employed in organisations managing integrations that are highly mission critical in nature. Organisations connecting to legacy systems, SOAP services, XML data structures, asynchronous processes, managed file transfer or needing guaranteed message delivery often lean on an ESB for these capabilities. Use cases that involve exposing discreet services across an organisation as part of an overall service catalogue benefit greatly from the approach.

We often see the new breed of open source ESB projects whether from Apache Foundation or backed by vendors used in migration projects. Enterprises with high ongoing costs associated with very stable integration services are actively porting their services across to lower cost platforms. With the advent of IPaaS and lower cost ESBs with open source roots, the value proposition for expensive ESB implementations has eroded rapidly.

As a WSO2 partner, we often see the WSO2 ESB paired with WSO2’s market leading API Management product as an alternative to more expensive approaches in the market. We have a number of customers who reap the benefit of this robust and highly scalable approach at significantly lower cost. In many cases we often end-to-end support the entire solution for them and obfuscate the complexity that is inherent in these types of deployment. We term this an Integration as a Service offering.

Implementation Best Practices

In implementation terms there is a very well trodden path with respect to delivery of an ESB. Gathering of requirements, establishing architecture, data architecture/models/schemas, business rules, data mappings etc are common prerequisite activities across all integration projects. One area where we may see a difference in approach is in the roles within a project. It’s common for the solution approach or integration pattern to be deployed to be decided by someone other than the developer delivering the integration. We may also see a segregation in roles when it comes to delivery of some of the integration artefacts e.g. XSLT for data transformation on XML schemas may be done by a dedicated XML developer.

Approach, degree of segregation of roles etc generally depends on the size or scale of the implementation and organisation it is being implemented into. Some organisations, for example, already have a defined XML custodian role or XML developers within the organisation who will define schemas and build XSLT for the integration patterns deployed. They will also manage those artefacts ongoing. In other smaller organisations, the integration developer will deliver the entire solution or process end to end. What a buyer should find is that approach to delivery is fairly consistent across products within the ESB space.

How do technologies differ?

Buyers/adopters of ESB technology should find or look for consistent support for the well defined Enterprise Integration Patterns out there in the products they are evaluating. A failure to support these patterns will generally call into question the value proposition of the technology. Some vendors in the market have done a lot of work in UI terms to obfuscate the complexity of the patterns. Some have even completely changed the way they use the service bus and transitioned their products into more IPaaS style technologies. In which case they are generally leaning on the robustness of the technology as the base of their stack.

User experience, SaaS connector coverage and cloud based technology connectors are key areas of differentiation. Amazon S3 buckets, Azure Blob storage, Amazon SQL, Google Pub/Sub, Apache Kafka are all examples of connector technologies where ESB products may seek to differentiate. In the world of Azure, service bus is just a component of Azure Integration Services. It’s a basis to provide guaranteed message delivery supported by sessions to group related messages. The enterprise integration patterns are not really relevant any more in an Azure based implementation of integration.

User experience and flexibility in deployment options is another key area of focus. WSO2, for instance, has put a lot of effort into being able to deploy ESB functionality as containerised microservices. The same is true of Tibco in the market. This is where we can see vendors adapting to new styles of deployment of integration technology.

Considerations

Adopting an ESB should be rooted in adopting a service oriented approach. If an organisation architecturally has little interest in enterprise integration patterns (EIP) then the choice of this technology will be questionable. The exception will be where the technology has adapted in some way to bring benefits that make EIPs irrelevant in deployment. Organisations adopting an ESB should be looking for the inherent robustness of the approach and willing to adopt the discipline that goes with it or engage an organisation that can provide the discipline like Chakray.

Organisations that are new to integration or have a cloud/SaaS first approach may find an ESB to be an unnecessary overhead in integration approach terms. For these organisations an IPaaS style solution may be a better integration technology to deploy and easier to adopt. For many organisations, dealing with asynchronous or long running processes, the requirement for a message store makes the technology an inherent part of their architectural needs.

The more accessible open source ESB technologies whether engaged open source or where available with a vendor support subscription are now almost the de facto path for ESB deployment. Ignoring this option, is generally considered to be ignoring value in your IT operation.

How Can Chakray Help?

Chakray was originally founded on its expertise with open source integration technologies. If an organisation is looking for ESB technology then we will bring a commercially backed open source or consumption based ESB solution to the table. We consider this core to our values in terms of delivering value to our customers. That doesn’t mean we don’t deploy on top of other proprietary vendor technologies or that we recommend an open source ESB where an IPaaS technology is more appropriate. Our business is built on a tenet of doing the right things for our customers and we help our customers make pragmatic integration choices.