Why platform enablers are the fundamental building blocks of your AIOps solution?

In our earlier blog “6 Essential Questions to ask when evaluating AIOps solution”, we reasoned the questions anyone should be asking when evaluating an AIOps solution. In this post, let us dig deeper on the key platform enablers of any AIOps solution- data discovery, data adapters, data ingestion, correlation engine, MLOps and integrations for AIOps solution. Before that, let us refresh what AIOps is.

AIOps, as the name implies, leverages a range of AI capabilities to enhance IT operations, user experience and customer journey, including knowledge-driven predictive analytics and self-healing. AIOps solutions are built either to automate simple tasks so that IT teams can focus on more strategic work, or to perform tasks that are beyond a human’s capabilities.

It is the AIOps platform’s ability to learn during these processes that creates a huge advantage. For instance, an AIOps solution might understand that performance issues could be the result of saturation caused by cyber-attacks, and that the situation should kick off security processes to mount a defense. For traditional tools, such an incident would be addressed as a simple performance issue, and not recognized as a security threat.

Therefore, the most important foundational capability an AIOps solution is the ability to deal with thousands of data points and make correlations that most humans would not make. For instance, data update errors that lead to a pattern, and then leads to the identification of a bad network connection that would normally take weeks to diagnose.

Now let us look at what makes an AIOps solution. It turns out that there are 4 key platform enablers of an AIOps solution.

1. Data Pipeline: Ingesting data from diverse data sources like log and traces

Modern IT environments include multi-tier applications hosted on-premise, managed cloud, unmanaged cloud with third party integrations, SaaS integrations, legacy applications, big data platforms, dev-ops tools and more.

Despite massive investments in IT tools infrastructure, enterprises are still struggling to keep up with service levels and user demands. In small to large IT organizations, monitoring tool sprawl has become so commonplace that it is not unusual for operators to be monitoring 10-15+ off-the-shelf tools around various groups. Built on old world technology these complex and non-integrated point solutions, meant administrators spend their time jumping across tool consoles.

Typically, they are left with a flood of events when something goes wrong and are unable to get the unified and contextual view to pinpoint infrastructure issues, service degradations or outages.

An AIOps solution needs to sit between these heterogeneous tools and pull data from application and device logs, SNMP data, netflow, text files, xls files, RSS feeds, rest APIs, application transactions; at the same time it should be able to receive data from sources by installing its own agents or third-party agents or without agents. Therefore, an AIOps solution should have plug and play adapters for different types of data sources covering all common devices and systems in enterprise IT environment such as the following:

Network and Network Devices

• SNMP v1/v2c/v3 based monitoring of all popular network vendors and servers with SNMP enabled SNMP traps collection.
• Traffic flow statistics collection using netflow and sflow
• Router logs, Firewall logs, Windows event logs, Linux server syslogs

Compute and Storage

• Servers for all popular Operating Systems hardware systems. Windows, MAC OS, RHEL, Solaris, Sun Sparc, IBM AIX, Ubuntu
• Virtualization hosts and VMs
• Systems irrespective of whether they are present on premise or on cloud
• Storage devices

Application Infra

• Popular web servers including IIS, Apache and Nginx
• Middleware and application containers like Web Logic, Web Sphere, JBOSS, Tomcat
• All popular SQL databases including Oracle, MS SQL, mysql, Postgres
• No SQL databases like Mongo DB, elasticsearch
• Message brokers such as IBM IIB, redis, kafka

2. Correlation Engine: Business journey visibility – correlation of transactions

All the existing tools are also tied to the legacy architectures of traditional SQL databases, with a fixed schema, making it also a single point of failure with scale and performance limitations. This also makes it difficult to give unified and correlated views, which needs analysis and correlation of varied sources of data in the IT infrastructure, across multiple vendors and applications.

Truly competitive organizations need to move away from legacy, silo-based approaches to a broad-based IT infrastructure operational analytics solution. This solution should be rapid to deploy for an environment, provide ability to correlate petabytes of data in real time, not just proactively spot issues but also to discover trends and relationships to support the constant digital transformation of an enterprise.

Therefore, an AIOps solution should have custom adapters for different types of message brokers and Java/Python/.Net applications to correlate business transactions across a customer journey.

3. MLOps: Pre-packaged and Custom ML models

AIOps solutions are all about the data. They store data as the monitor systems and can determine issues that need immediate attention, such as a down storage server. Or, they can deeply analyze historical data to determine trends that may portend a failure or other potential issue. The lifeblood of any AI system is the data needed to train the ML model, and this is the opportunity presented to AIOps tools. Monitored cloud or on-premises systems spin-off gigabytes of data each week, and that data can be fed into analytic systems augmented by AI.

Enterprises that wish to leverage AIOps solutions should be careful to understand the pre-packaged ML models such as User Experience Index, Operations Performance Index. They should also test the AIOps solutions across both pre-packaged and custom ML models tailored to the growing needs of the business.

4. Automation: APIs to enable automation

With every organization working rapidly toward the ultimate goal of digital transformation, there is a natural place for AIOps within service management as a means to automate processes and automatically spot and react to issues in real-time.

The infrastructure itself is also constantly morphing and changing, yet service desks are still expected to resolve requests, incidents, and performance issues in seconds – an impossible task, given the volume of data.

While many enterprises use their ITSM (IT Service Management) tools to process service desk tickets and requests (incident management), a core part of ITSM is also asset management, change management, and problem management. Data from the configuration management database (CMDB) holds IT asset data (or configuration items) and this data is used for process automation of changes, incidents, problems, and more.

Automating these processes helps reduce downtime and cost when done effectively. And the key to automating these processes effectively and efficiently is clean and accurate data.

Your AIOps solution should enable integration with ITSM tools, frameworks, configurations, infrastructure, systems of procedures (SOP) and scripts. Without robust integration with ITSM processes, automation can be a dream.

Conclusion

Do not underestimate the need for your AIOps solution to support new data sources, configure the data sources, understand the data available and create dashboards to get the smarter insights. Understand the 4 key platform enablers while evaluating an AIOps solution for making the right choice.

✍ Srikanth Narasimhan, the author of the article, is a Technical Advisor @ VuNet Systems. He is an Enterprise Architect and has served as a distinguished engineer at Cisco.

VuNet Systems is a deep tech AIOPs startup revolutionizing digital transactions. VuNet’s platform vuSmartMaps™, is a next generation full stack deep observability product built using big data and ML models in innovative ways for monitoring and analytics of business journeys to provide superior customer experience. Monitoring more than 3 billion transactions per month, VuNet’s platform is improving digital payment experience and accelerating digital transformation initiatives across BFSI, FinTechs, Payment Gateways and other verticals.