I received this comment on my last blog: “Is this meant to be a humorous piece?” Or does the author not realize that all data, even real time data, is after the fact?” – Robert T.
Yes. A bit humorous. But meant to be more ironic.
You are correct, Robert, that all data is “after the fact.” But it’s important to consider “how long after the fact” and what you think you can do with aging data.
The flaw in IBM’s thinking is that they presuppose that the environment, on which they base their analytics, will not change. They are looking back in time – hours, days, weeks and months. A lot changes over those periods of time.
The thing that kills the accuracy of IBM’s analytics is not the single, root cause effect pattern prediction but the quicksand that the pattern prediction is based upon. Couple that with the combinations of different things that show up in that specific instance. The cause and effect conditions change.
While things may not change for days or weeks in Legacy infrastructures, DevOps changes everything. DevOps aims for continuous delivery – you are constantly changing or adding new applications. Elastic compute (cloud, SDN, NFV, and even more traditional elements like load balancers and MPLS networks) presumes failures will happen and so offers dynamic change.
Backward-looking statistical analytics relies on data that can be baselined. But minute-by-minute or even second-by-second change makes it pretty hard to establish a statistically significant baseline.
Right now, most of the world tends to manage incident by incident. Hey, it’s what we are taught. ITIL. You build a model of your infrastructure and you manage according to that model. But in a virtualized, dynamic world, things are changing all the time. Today, models are outdated as soon as they are created.
Maybe it’s time to rethink things that do not scale or cannot adapt with the speed of change.
So what I’m talking about here is opening our eyes to this fact – we are in the 21st century.
Let’s talk about analyzing data that is seconds old, not days or weeks or months old. Capture telemetry that matters most – what is newest – process it in real time (or very near real time as it were) and continually add in new data. Stop relying solely on the old landfill of static, out-of-date data, models and rules.
Machine Learning and natural language processing give us the power to do this today. Moogsoft has patented sets of algorithms that give us the power to discern anomalous behavior from real time data without pre-describing an anomaly.
This means we can indicate the existence of an anomaly earlier. By contextualizing the real time data contained in the anomaly, Incident.MOOG helps support teams diagnose the cause and impact of the Situation much more quickly. Layer social collaboration on top of that (think virtual war room – like a Facebook wall) and you have the kind of solution that is adaptable and collaborative to harmonize the support of both the DevOps world and the underlying elastic infrastructure.
Bottom line, processing and acting upon streaming data without a model is referred to as ‘real-time’ in the computer industry. Robert, you and others may regard it has near real time, but there is a distinction between data that is hours, days, weeks old. Incident.MOOG works in “near” real-time.
Think about it as acting upon impulse. That’s what the next generation of IT operations analytics is all about.
