The best analytics tool

The purpose of data analytics, using big or small data, is to provide business insight. It is about taking sets of information, organizing them in a meaningful way and then combining them with other bits of information in order to empower intelligent decisions. 

We all use the best analytics tool, every day, practically all the time: our brains!

I am sitting here looking out the window at a set of colors and shapes. That is really meaningless information, as what I described is too vague. If I provide you with more details you will be able to match the patterns in your analytical tool (your brain), compare it to things you already know and will soon be able to comprehend what I am looking at. Let’s fire-up your analytics engine: I am looking at Green and brown patterns. The brown patterns are rectangle-like in vertical orientation and considerably thick. The green ones are mainly on the upper portion of my view, in oval-like shapes, randomly oriented, and have chainsaw-like edges. They are mixed with long, thin and curved brown patterns which link to the bigger brown patterns I already mentioned earlier. I can stop. You probably worked-out that I am looking at some trees outside my window.

You may be thinking now: well obviously our brains are analytical tools. We always talk about people’s analytical skills. That is true, but the property I am after here is the effectiveness of the tool. Can we build analytical technologies which are better than our own brains? Faster? Definitely. Bigger capacity? Of course. Better ability to filter information, reason and draw insight? I doubt it!

If now I tell you that the wind is blowing. You will start asking: well what does this mean? Well, that is related to reason I am looking out the window in the first place. Let’s assume I am planning to go on a picnic. Because I know it is windy, you might conclude that this is likely to degrade the quality of the picnic. So the “business” dilemma here is whether to postpone or cancel the picnic. This now demands further insight. Can I go at any other time? What is at stake? Are there any other parties involved? What if I now told you now it is 2 degrees Celsius outside?  Or that the planned picnic’s location is far away from where I am now, or even that the scheduled time is only tomorrow afternoon?

Your brain is continuously integrating the information I am giving you. It is analyzing each scenario and drawing insight. You can barely even notice that your brain is doing all this work for you.

The point I am trying to make is that our brain does not change its capacity or processing speed yet we generally improve our analytical skills every day through learning. So forget distributed data stores, in-memory computing, and processors’ speed. The best analytical tool obviously needs all the right information and knowledge base to draw conclusions, but the real “magic” lies in its ability to filter and integrate information, even (or especially) when there are gaps in the information that is available to us. Performance will always come second to clear business requirements and well-designed algorithms for reasoning and decision making. It seems to me that we are always chasing the rainbow of bigger and better data stores with stronger processing power, perhaps conveniently forgetting to check our pockets for wisdom, common sense and simplicity.

Therefore, I leave you with this: Ask not what you can do with the data, but ask what can the data do for you.