Let’s understand the types of anomalies

In our previous blogs, we took you through different methods to detect anomalies in your data. But just detecting an anomaly isn’t enough! If there’s an issue in your system, knowing the type of anomaly that’s occurring is crucial to help you get to the root of the problem and analyse it! What is the point of just detecting that something went wrong, if you cannot know what made it go wrong? So to help you better understand this process of root cause analysis, in this blog, we’re going to cover the three most important types of anomalies: global, contextual, and collective anomalies!

Global or Point Anomalies

This is the standard variety of these nuisances. Global, or point anomalies are simply a single data point that are very different from the entirety of your dataset, as can be seen in Figure 1. For example, a sudden drop in payment transactions during the day, or one of your data servers having momentary network connection issues. These are anomalies, no doubt, but perhaps not too important from a business perspective. This is simply because no real-life system is perfect, and trying to fix everything that might have gone wrong is not very practical. That brings us to the second type of anomaly.

Contextual or Conditional Anomalies

This is like a point anomaly, but with a certain context. If there is a momentary drop in transactions during the day, that could perhaps be ignored. But if there is a sudden spike in transactions during the night, that might be something to analyse more deeply, since that’s not only a point anomaly, but also very unusual given its context. You can see this in Figure 2. Basically, knowing the context of a point anomaly helps us in finding out whether it is really important or not. This can be a good method for reducing the detection of false positives; depending on your field you can inculcate contextual information about the data that you’ve collected in order to avoid classifying a super high or low value as an anomaly. 

Collective Anomalies

Remember that momentary network connection issue in your data server we just mentioned? Now, if this happens only in one server, it can be ignored. But if several servers distributed over various places have the same problem at the same time, then it is a cause for concern. It may be a hacker trying to access your entire system! Sometimes, data points will band together and form an anomalous group instead of irritating you as individual anomalous points. What this means is, a single point could be a normal occurrence on its own, but when paired with another point, they make an anomaly together! Let’s take another example. If the payment transaction of one user is taking a really long time, it is alright, and could happen due to an issue at the user’s end as well, but when you have many users, whose payments are taking much longer than they should, this is a good reason to examine your system and see what could be causing this problem! 

Detecting these different kinds of anomalies well in time is very important for enterprises to prevent business loss. And given the hundreds and thousands of time series data enterprises have to monitor continuously, it is impossible to do this anomaly detection manually. The algorithms developed by the AI/ML team at VuNet enable various enterprises in automatically detecting all these three kinds of anomalies at just the click of a button.

In the next blog in this series, we will stitch together all the concepts we have shared in this blog series so far and more, like Singular Spectrum Analysis, Saliency Maps, and Dynamic Time Wrapping. We will see how all this comes together in creating more robust anomaly detection models which take good care of the vagaries of time series data.

VuNet Systems is a deep tech AIOPs startup revolutionizing digital transactions. VuNet’s platform vuSmartMaps™, is a next generation full stack deep observability product built using big data and ML models in innovative ways for monitoring and analytics of business journeys to provide superior customer experience. Monitoring more than 3 billion transactions per month, VuNet’s platform is improving digital payment experience and accelerating digital transformation initiatives across BFSI, FinTechs, Payment Gateways and other verticals.