Data Analytics-The Wheat And The Chaff

Data analytics has become a hot area , and everyone wants to know how they can use machine learning (ML) and artificial intelligence (AI) to make better decisions. We are drowning in data, and want to know how we can make use of it intelligently. How do you transmute Big Data into knowledge and then convert that into actionable insights? This is the promise which data analytics offers.

Data analytics falls into four buckets. We can analyse incoming data and this is called descriptive analytics. This is fairly straightforward, and you need to create dashboards so it’s easier to visualise the trends – after all, real time graphics are worth a thousand words.

You can then apply an intelligence engine to the data, to run diagnostic analytics. These help with correlation which can lead to causation, so you can understand why certain trends occurred in the past.

This then leads on to predictive analytics, which is what we’re interested in. How can you extrapolate from the past into the future, so you can follow a path which has a higher probability of leading to success.

Finally, when your model is mature, you can use prescriptive analytics, which actually tell the business owner what to do next. This is the Holy Grail of data analytics, and we are slowly but surely getting there.

Part of the problem is that because big data, ML and AI have become buzzwords, lots of companies which talk about providing insights through data analytics often confuse correlation and causation. Mindless data mining leads to over-fitting of data, and this can lead you down the wrong track. This is a big risk when you place too much reliance on the data and stop using your common sense.

Also, they often tell you things which you already knew – stuff which is so obvious that it is of no use because it doesn’t changed your decisions or actions. Often, this ends up being pointless academic information which just reconfirms your biases, and this is of no use to the business owner.

The key question every data analytics company needs to answer is – what actionable information are we providing to the business owner which he would otherwise would have overlooked or misinterpreted? How will our insights help him to become more successful? If they can answer this question intelligently, they’re far more likely to succeed.

[The author of this post is Dr Aniruddha Malpani, Director and founder, Malpani Ventures.]