Friday, September 28, 2012
The two sides of data In-e-quality
Not all data is born the same. Its intrinsic characteristics differ based on its genetics, while its quality and value differ based on its environment.
In short - never judge data by its raw statistical profile appearance.
Bad looking data may be economic and relevant to the business, while beautiful, complete data might actually carry high maintenance costs, and dare I say - redundancy...
But lets focus on the opening statement. I stand on giants shoulders, when I talk about the DNA and eco-system of data. Data can be measured in various dimensions from volume to change velocity, verity and extent of construct. Data is only as valuable as its fitness for use. Oh, and of course, different people use the same data in different ways - so what is really good data?
We say a person is good or bad, useful to have a around or a burden. We choose friends differently than we choose team mates - or do we? This is perhaps the key for why people struggle so much with data quality. Maybe it's not just about how well the data fits its purpose. Maybe its really how comfortable we are with it.
Believability you say? Nonsense! You like the data because you are either familiar with it, or it chimes with your view of the world! That is why people change data sources. This is why you will never find an absolute single version of the truth.
People use data, and people are subjective. It is part of what makes us risky and successful. So have we lost the war for data quality already?
The answer is no. We might hold an individual view on data quality, but in the same way society manages to create law and order to give each person an equal opportunity, so can data governance create data quality across the enterprise. I like to call this "data in enterprise quality", or data in-e-quality for short. Not only because data is different, and needs to be handled differently, and not only because its quality is seen differently by different people. All data is born equal with the right and opportunity to be in-e-quality.