Data integration with monolithic or microservice architectures

In any type of business organization, data is one of the most valuable assets. Using them in the right way makes a difference because it makes it easier to make good business decisions. It also has a direct influence on commercial growth (and very specifically on sales) and improved profitability. All this has made it essential to use data from different sources and to expose them in cases of commercial use. The solution that WSO2 proposes for the perfect integration of data is based on two service architectures that have a wide diffusion, the monolithic or centralized architecture, and the microservices or decentralized architecture. But do you know the differences between monolithic and microservices architectures? We explain what each of them are.

What is a centralized or monolithic architecture?

The monolithic applications are designed to enable different types of companies to carry out their daily business activities with all the necessary components in a single application and in a way that is logical in each case. Two examples of very popular monolithic applications are ERP (Enterprise Resource Planning) and CRM (Customer Relation Management).

The fact that the components of these applications remain interconnected and interdependent means that the agility of such applications can create certain difficulties. The infrastructure associated with such applications can be problematic, as scaling a single component in the monolithic application requires resources for another instance of the entire application.

To solve this type of problem, SOA (Service Oriented Architecture) was created, a type of architecture in which the application can be decoupled into small modules known as services. All these services are integrated using an aggregation layer called a bus. All services communicate through this service bus. Over time, this application can become monolithic, as all services are integrated with the same integration layer.

What is a decentralized or microservices architecture?

Microservices were born to reduce as much as possible the limitations that reside in the SOA architecture, so that a unique application is developed as a combination of specific and independent services. These, known as microservices, are executed and implemented independently.

Its creation is intended to serve only specific business functions, such as inventory management or customer management, among others. In addition, microservices must be independent of each other. This allows developers to develop each microservice using different languages and to use different databases for each microservice. To communicate between microservices, it uses lightweight protocols such as HTTP and REST.

What are the capabilities for achieving seamless data integration?

The fact that companies are highly dependent on data makes it necessary to store it in a way that is very accessible. The applications have a lot to say about it. To do this, data must be exposed as a service, allowing applications and systems to access and use it. This includes migrating from one data source to another, retrieving data from a source and manipulating it within the application.

WSO2 Enterprise Integrator supports several types of data sources including RDBMS, CSV, Excel, ODS, Cassandra, Google Spreadsheets, RDF and any web page. It also offers the freedom to write a custom data source, where users can define their own data source implementation to meet their business logic.

Transactional support. WSO2 data services provide distributed transactions using the Java Transaction API (JTA) that enables global-level transactions on multiple data source resources in the JVM.
Availability as API REST. Because of its flexibility, speed and simplicity, REST is widely used for data integration. Data services can be exposed in WSO2 Enterprise Integrator using SOAP services or REST services. For a RESTful service, a query can be assigned to a web resource via the HTTP method in the data service.
Batch processing. This is an efficient way of processing large volumes of data and is often used in companies for tasks such as paying salaries or calculating and printing invoices. WSO2 Enterprise Integrator provides facilities to invoke multiple operations as a batch.
Data streaming. Processing large volumes of data is challenging, so WSO2 Enterprise Integrator supports transmission capabilities where there is theoretically no limit to the data size of a data service response. This provides high memory efficiency and low response time.
WSO2 Enterprise Integrator identifies and protects against general computer security threats and other threats that are unique to Web services. It adheres to the WS Security, WS-Policy and WS-Security Policy specifications.
Data Transformation and Validation. WSO2 Enterprise Integrator provides capabilities to seamlessly transform output data into different formats through the use of XSLT and JSON and XML schema validation.

Customization and data integration

Enterprise organizations can use different software design architectures, although the most widely used, the aforementioned centralized monolithic architecture and decentralized microservices architecture, are currently widespread. WSO2 Enterprise Integrator allows the integration of all those that originate and arrive by different procedures, doing it in the architecture patterns.

In every organization, efficient and effective data management is an essential part of success. Therefore, steps need to be taken to integrate data in a way that matches the organization’s business needs and the nature of the business.

WSO2 Enterprise Integrator supports data integration capabilities, providing an easy-to-use platform for integrating data warehouses, creating composite data views and hosting them as data services. It supports secure and managed access to data through various types of data sources, data service transactions, and data transformation and validation using an agile, lightweight, and easy-to-use development approach.