How to Orchestrate and Automate Your Enterprise for Efficiency

Naturally, if this structure is scattered, it tends to produce a huge list of support issues, with the most common points of concern being the centralized authentication (most probably via Active Directory), environment monitoring, analytics and reporting, scalability and high-load support, and security. As a result, the business efficiency of an enterprise drops down while operating costs continue to grow from day to day. In short, the major problems caused by an array of platforms and applications are the following:

In light of the above, any enterprise at a certain level of development may find itself facing the same needs - a unified enterprise architecture strategy with a robust technology stack to back it up, the orchestration of the existing business processes and the possibility to quickly add and integrate new ones, as well as streamlined communication between all components of the system.

We, at ELEKS, put an effort into developing the solution that can effectively predict, structure and satisfy these needs.

ELEKS devised the Enterprise Middleware Platforms Suite (EMPS), an enterprise and Cloud -ready, secured, flexible, scalable, vendor agnostic, license-free technological solution that covers different aspects of infrastructure and application-level orchestration, management and integration. All components of the Suite can be easily deployed to the private/public Cloud or on-premise and configured with the automated scripts. The solution covers a wide range of goals that CxOs or Enterprise Architects need to meet:

• Orchestration and management of services, applications and data flow;
• Automation of business processes;
• Establishment of federated security and Single Sign-On;
• System logging and monitoring;
• Data analysis, reporting and predictions;
• Monitoring of mobile and IoT devices;
• Implementation of Big Data solutions;
• Centralized (mobile ready) web solutions for clients and employees;
• Scalability and high-load support;
• SaaS and multi-tenancy support;
• Centralized control of infrastructure management, and more.

You can check out the video to get the high-level understanding of how the Suite works for a business covering all the essential processes and integrating them for effective communication:

With such structure in place, an enterprise can achieve significant TCO cuts due to a number of reasons:

• The ELEKS EMPS can go Cloud or stay on-premise without additional re-configurations needed;
• The infrastructure’s overall portability and compatibility reduces the number of integration issues;
• An enterprise needs minimum dedicated staff to setup, maintain and support the solution;
• The enterprise architecture is properly structured, all of its components are properly placed within the “big picture”;
• The Suite is mobile ready and Big Data ready.

ELEKS Enterprise Middleware Platforms Suite Overview

All components of the ELEKS Enterprise Middleware Platforms Suite are open source and free: the customer enterprise has to pay only for the environment. Moreover, all components of the Suite support horizontal scalability and can be extended with more nodes in case the load grows too high for the existing system.

The ELEKS EMPS consist of several main components that can be deployed and used separately or easily integrated together, depending on the customer’s needs.

The Orchestration Platform is responsible for the applications/services orchestration, business process automation, data/traffic control and authentication/authorization;

The Front-end Platform provides for the UI interaction with clients and employees;

The Mobile Platform gathers data from mobile and IoT devices;

The Big Data/DWH Platform gathers, stores and analyzes the data from mobile devices, telemetry and other data sources;

The Business Intelligence/Data Science Analytics Platform is responsible for online, real-time, descriptive, predictive and prescriptive analytics as well as reporting;

The Log Platform logs and monitors the events in the enterprise system;

The ELEKS Enterprise Platform Suite includes two more platforms that can be considered the sub-parts of the Orchestration Platform:

• the Security Platform that provides for the enterprise-wide authentication/authorization based on federated security and single sign-on approaches, and
• the Delivery/Shipping Platform that is responsible for notifications and communication between systems.

In addition to the existing platforms, ELEKS plans to integrate two more PaaS solutions to its Suite:

• E-Payment Platform, a solution to process online payments via the most popular providers like PayPal, Google Wallet, Payoneer, Skrill etc.
• GIS Map Platform, a solution for creating, updating and rendering the maps of the world (Google Maps, ArcGis etc.) via UI and services.

ELEKS Enterprise Platform Suite Components

Orchestration Platform

ELEKS Orchestration Platform is the core of the enterprise system with all services, communication and integration being processed through it. This platform is based on the open-source iPaas WSO2 Carbon stack that includes several components for message mediation, service orchestration, governance and monitoring as well as business process management.

The Orchestration Platform has the following functions:

• Applications/Services orchestration - control and monitoring of services data and traffic.
  This function is implemented with the help of the WSO2 Enterprise Service Bus (WSO2 ESB) that registers all services and components in the system.

• Business processes automation - Business processes execution and monitoring.
  The foundation of this function is the Business Process Server (WSO2 BPS) that works with BPMN (Business Process Model and Notation) and BPEL (Business Process Execution Language).

• Authentication/Authorization - Storage and control of user roles, rights and permissions in system.
  This function is implemented with help of the WSO2 Identity Server (WSO2 IdS) that is integrated with identity providers like Active Directory and provides authentication/authorization mechanisms (including the Single Sign-On approach).

The ELEKS Orchestration Platform also includes additional components for better business process management:

• WSO2 Governance Registry to store, catalogue, index, manage and govern the enterprise metadata related to any kind of asset;
• WSO2 API Manager to manage API access with the full API governance and analysis.

All the Platform’s components described above have web interfaces for convenient configuration and use.

Front-end Platform

The Frontend Platform provides for the UI interaction with clients and employees. Keeping it simple, this is a reusable and scalable web portal that consists of the following components:

• Content Management System (CMS): Umbraco (free) or Kentico (commercial). These systems support the high-load mode with the help of load balancing, DB replication and advanced caching. Also, they have a lot of standard out-of-the box features such as a blogging platform, social media integration, document management, file distribution, i18n, Cloud/mobile support, graphs and charts and more.
• Single-page application (SPA): a component is based on AngularJS or ReactJS. This UI approach is mobile ready with help of Bootstrap.
• Services for the backend logic:
  • RESTfull services based on the ASP.NET Web API or WCF. These services communicate directly with SPA. They can also be used for communication with the Orchestration Platform.
  • SOAP services based on the WCF. These services communicate directly with the Orchestration Platform.
• Google Analytics for web portal usage monitoring.

Mobile Platform

ELEKS Mobile Platform gathers data from mobile and IoT devices. Basically, it consists of mobile clients that are installed on the target devices (usually via MDM) and a broker that receives and collects the data from the clients and further sends it to the data storage.

This Platform embodies the Fog Computing concept that presupposes data gathering and aggregation without internet access.

The following diagram represents the “big picture” - how the Mobile Platform communicates with the enterprise service bus and the Big Data Platform.

Big Data/DWH Platform

The Big Data/DWH Platform gathers, stores and analyses the data from mobile devices, telemetry and all other data sources. It is also responsible for data structuring and preparation of reports.

The diagram illustrates the location of the Big Data/DWH Platform within the Enterprise interaction model:

This Platform consists of two major components - Big Data and Data Warehouse.

The Big Data component is responsible for data aggregation, analysis and storage. In most cases, it works with the data from mobile and IoT devices and telemetry, which accounts for millions of records or gigabytes of traffic per second and amasses into terabytes of stored data. The component follows the modern Big Data patterns such as lambda or kappa architecture, while all of its units are free and highly scalable.

In general, the Big Data component can be characterised by these features:

• Real-time processing
• In-memory analytics
• Raw data fast writing
• Scalability

Its main clusters include:

• Kafka for building real-time data pipelines and streaming apps. Kafka processes all data from external sources and uniformly streams them to the Cassandra.
• Spark, a fast and general engine for large-scale data processing. The tool analyses and aggregates data in RAM. It is a part of the Cassandra Cluster Analytics.
• Cassandra, a NoSQL data storage. This solution suits best for scalable systems with real-time data analysis.
• Hadoop for long-term storage of historical data and the results of data analysis.

The below diagram illustrates the Fast Data Processing:

Within the ELEKS Big Data/DWH Platform, the DWH component is represented by the Postgres XL (Scalable Open Source PostgreSQL-based Database Cluster) or MS SQL Server

Business Intelligence/Data Science Analytics Platform

This Platform is capable of performing online, real-time, descriptive, predictive and prescriptive analytics as well as reporting. It helps organisations build applications that add transparency and a deeper understanding to their business.

The BI/DS Analytics Platform includes:

• Development tools
• Reporting tools
• Dashboards
• OLAP

The BI/DS Analytics Platform scheme:

An example of a dashboard generated by the Platform:

Follow the link to see the full description of the Platform and more dashboard and report examples.

Log Platform

The Log Platform logs and monitors the events in the entire enterprise system. It consists of two main components, ELK Stack and WSO2 Data Analytics Server, that can substitute each other, depending on the customer’s environment and needs.

Elastic Stack (ELK Stack - Elasticsearch, Logstash and Kibana) is a group of open-source products from Elastic that allows retrieving data from any type of source and in any format with further searching, analysing, and visualising that data in real time.

The diagram below demonstrates the ELK Stack usage example:

WSO2 Data Analytics Server (WSO2 DAS) is a comprehensive enterprise data analytics platform that leverages machine learning to fuse the batch and real-time analytics of any source of data with predictive analytics.

The image below illustrates how this solution supports the demands of a business as well as IoT, mobile and web apps.

The following diagram demonstrates the WSO2 Data Analytics Server usage example:

ELEKS EMPS Deployment

The main goal of ELEKS’ deployment procedures is to deliver the Enterprise Middleware Platforms Suite components to the customer’s environment fast and with minimum configuration efforts. This is achieved through the containerisation and automation of the configuration scripts. The whole process can be handled by one DevOps engineer.

The ELEKS EMPS is delivered in several stages:

• IaaS: the basic layer required to create the environment infrastructure (Cloud). It can be built on public Clouds like Azure/AWS or on a custom Cloud created with OpenStack.
• PaaS: this stage is for automated deployment, scaling, and management of containerized applications across clusters of hosts. In other words, the PaaS layer automatically deploys and configures the applications and relations between them. An open-source system Kubernetes (or other analogs) with the Docker container system can be used for this purpose.
• SaaS: this is the application level, i.e. the level of ELEKS EMPS components. This level supports the multi-tenancy approach.

The following diagram illustrates the architecture of all layers working together:

Finally, the application deployment flow looks like this: