Intro
Machine Learning will empower the user by monitoring, identifying, flagging and deducing the reasons behind changes in data states. It goes beyond the automation of data processing, applying powerful algorithms across systems to evaluate data and initiate data analysis. But how far down the rabbit hole can we go? What functions does Machine Learning lend itself to best? Alvin Chan considers how AI is improving BI.
This is the 1st of a 2 part blog series on Augmented BI.
Part 1: Anomaly Detection
A machine can’t solve the problem for you but it can tell you where that problem is if you design the system right. At AVORA, we wanted to create a solution that did the heavy lifting by crunching the data at source, avoiding multiple copies, and which could calculate cause/effect models, freeing people up to use their intuition and ask the right questions.
Achieving that optimal blend between human and machine is what we call Augmented Intelligence. Utilising the best of both, Augmented Intelligence is an efficient form of data processing that is fast, informed and places the data the user needs at their fingertips so that they can make the right call.
The obvious place to start when applying this concept was data anomalies. You’d think anomaly detection would be straight forward given that it involves the monitoring of data metrics for extremes that fall outside the tolerable boundaries set by the business. But previous attempts in the industry to automate it could be considered failures largely due to the quality of data input.
There are broadly three approaches to anomaly detection:
Supervised: Data that is unusually high/low is manually labelled by the user and the system accepts and applies these thresholds. It is a time intensive, difficult and costly process to maintain, and the practice can kill databases.
Unsupervised: The system makes an informed judgement based on very limited user inputs i.e. low/medium/high alerts. It then can determine the range of usual behaviour and anything outside of that is considered anomalous. But the system is only as smart as the user input it receives.
Augmented: Machine initiated, this picks up unusual behaviour with as little human intervention as possible while allowing the user to adjust and refine those parameters. These corrections allow the system to adjust the sensitivity level and improve over time.
Machine learning exploits the sweet spot of semi-autonomous anomaly detection and has the flexibility to adjust and accommodate changes to the metrics being monitored. This makes the system adaptive overtime and ensures that alerts are only triggered when the variance exceeds specified tolerance thresholds.
At AVORA we automatically detect what is unusual – just pick an alert level – and our bespoke algorithms pick up trends, patterns and automatically surface any unusual behaviour, alert you so that you can take action. Our anomaly detection system, Smart Alerts, allows the user to set alerts on specific KPIs relevant to individual users, teams or the business at large so that these alerts have meaning and are actioned. In time, users will be able to contribute to tailoring the system to their specific preferences, by interacting with these alerts.
Take for example a digital media agency, they can use Smart Alerts to stay ahead of their clients’ questions. With real-time alerts on unusual behaviour of conversions, click-through rates and CPC (cost-per-click) they can act quickly and better understand how to service their clients. Users end up spending less time worrying, and more time tackling key business issues.
With the detection of anomalies, businesses can focus more on running their business and less time dealing with data quality, data connection and data accuracy issues. But knowing there is an anomaly is only part of Augmented BI. In the next blog in the series we will look at understanding the drivers of Smart Alerts using machine learning and Root Cause Analysis.
For our next blog in the series “Augmented BI” – we will focus on Root Cause Analysis – read the blog here.
