Good Design or Fast Results? The Truth? Learn to Manage Expectations

I write about data management, but this topic is deliberately none data specific. It actually applies to many aspects of one's work and life - but since I am a data geek - the content and examples will focus on data. You can extrapolate and draw behavioral conclusions on other domains of productivity.

Let's get started. Some people would say that a good design is imperative for success, and sufficient time should be spent on that phase of the project. They are of course - right. Other people say you need to deliver results, respond to market changes and generate revenue and contain costs, otherwise the business does not survive. These people are of course - also right. So how can both groups be right? The answer is balance.

One of the most valuable lessons you will learn (if you have not done so already) is that balance is as important as substance. What I mean by this is that there is an optimal point, where over-engineering becomes a hindrance, while lack of design can lead to unacceptable results. We will get to data in a minute, but my favorite example on this life-lesson is from civil engineering: suppose you are tasked to build a bridge across a river. You need it long enough, but not too long. If you measure your materials to the millimeter - chances are you are over-engineering the process and will spend too much time and money getting the exact "right" measure for your bridge. On the other hand, if you just grab a few planks of wood and start building, not only do you have a risk of bridge instability, you might actually have too little wood, or too much left over. In all cases, you will probably need to extend the time and effort to complete the bridge in order to compensate for these shortcomings. What you need is a balance: make a "good-enough" plan quickly enough, and then build a bridge correctly with optimal time and effort.

Now let's talk data management. One of the typical problems in today's information chaos age is the need to build or upgrade the train bridge while the train is already speeding along the tracks. Another well-known metaphor is building a plane while it is already in flight. The overwhelming criteria in this type of operation is stability. As long as we do not lose business or cause harm. This often leads to patches and workarounds and compromises in good design which eventually leads to issues in costs and strategic alignment. Raise your hand if you felt frustrated on not being able to implement a good design, because of time pressure or concerns about systemic risk. For example, not only is it expensive to change a data model (migration costs will make your executives cringe), but there often seems to be limited foresight as to the strategic benefits.

This is when the conversation needs to shift gears, and the balance needs to be struck by managing expectations. Understand where the business is going, and agree what is best from a technology and design perspective. Then start creating a balance by managing changes such that new parts are created ahead of time, and then brought to the bridge when opportunity allows it. Justify the "extra" efforts by identifying and quantifying the architectural benefits (in other words what you stand to gain or lose in the longer term). You may conclude that your data model should not change right now, but every time it changes it does so in a deliberate direction. It will also help you to understand its limitations and sensitivities better which helps with systemic risk management. The notion of managing expectations and creating a common architectural vision will also funnel independent views of resources in the business to work together under a joint effort, and will allow you to harness many minds to improve progressing in a common and agreed direction.

Now tell me, are you working towards implementing a strategic data management plan, or just doing localized data fire-fighting?

Motivating Data

What motivates people? A belief, a hope, a goal. The only thing that makes you read this is the belief or hope of finding something useful here. Something you can learn from, quench your curiosity or help you reach a certain goal in learning or performing.

What is motivating data? Well, data makes no decisions, and cannot apply any resources to a particular action. So, the statement is an absolute absurdity. Or not ... While data cannot change the way it interacts with the world - people certainly can affect this. And that is the point.

When I first started writing about data management, yes the notion of people affecting how data is applied seemed very trivial. Of course people manage data, and of course we affect how it is being applied. But what I am realizing more and more is how pervasive the subjective motivation of data really is. Every day, all of us, make dozens of decisions to disseminate or block information from flowing. Whether it is of personal nature, such as protecting loved ones from anxiety or pain, or professionally, where we use our professional judgment to improve the outcome of our efforts and those in our teams.

On a larger scale, companies and organizations make conscientious decisions as to what information to expose, when and to whom. This is really part of doing business, or interacting with the world. There are many strategies one can apply to affect these type of results. You can provide too much information with the intent to overwhelm, or create an impression of sharing everything, while carefully omitting certain bits of information. Whether it is right or wrong depends on the situation and the parties involved.

On the other side of the coin, we are information seekers. We look for information that can help us satisfy our beliefs, hopes and goals. We subscribe to channels of information in hope to receive the information we seek. We continuously fine-tune these subscriptions, replace or change the optionality’s on those channels. But in reference to what I said above - this is a tall order, as without understanding your information provider's intention - you are subject to their information filters.

Strategies to combat disinformation of that sort, include evaluating consistency and patterns related to the details of information you receive (sounds complicated, but we do this all the time naturally). A second strategy involves subscribing to multiple channels in hope of verification, or ability to have a more comprehensive perspective.

As a small bolt in the global information system, we can originate or terminate information flows by applying some of the strategies I noted above. We also need to motivate other people to behave the same in order to reach a level of meaningful influence.

My question to you is how are you motivating "your" data, and more importantly - what is your intention?

The Data Ocean

Are we ever going to get tired of comparing data to water? We heard of trickling data, data waterfalls, now people are talking about data lakes and guess what – I am going to talk about a data ocean. Yes, I am referring to the notion of the largest bodies of data in the world, comprising of a multitude of sources and consumers across many, many data domains.  But what I really want to focus on is what these oceans currently look like, what they will look like in the future, and how we should prepare ourselves to maximize their value.
 

What makes water so powerful is its combined force, its chemistry and its consistent and predictable behavior. Our current way of handling data is more like trying to mix water with oil, mud, rocks, milk, sand and lots of other stuff. That is far from the elegant nature of water.
 

While each entity in the world feels the need to derive its own chemistry of data, the truth is that the nature of data is as pure as water. We perceive data as murky and hence treat it as such. Therefore it molds, by our own actions, and becomes difficult to manage.
 

What am I saying? We have no common (agreed) perceptions, models or governance on data. As our data management models mature, we will see more harmony in data.  The essence of the simplicity in data has always been there and the notion of a data lake, and a data ocean, never caught me by surprise.
 

To truly “see” these bodies of data, we need to ensure we are able to view them as such. True “global” harmonization of perceptions on datasets is key to drive governance, management and hence data chemistry.
 

The sooner and broader you can tune in your organization and your business partners to mature and harmonize data perceptions – the better prepared you will be for Teneo Vulgo.
 

I predict a world where ALL data flows through a central data delivery framework, probably centered around a few major providers. In this world entities which have prepared and invested in orchestration power over data will hold the advantage. This is not a world where data is fair (when has it ever been fair), but rather a world where data is managed to serve those who have the strongest ability to influence and exploit it.
 

Think about data tsunamis, data storms as well as data seaside holiday homes, data sanitation systems and data feeding into our daily life. The power of information is only starting to emerge. For me, the famous saying comes to mind: may you live interesting times…