5 Core Skills to Harness Data and Make Better Decisions

Data as crude oil
The Economist magazine in 2017 likened the modern application of data in business to ‘the new oil’. In some ways the analogy has limitations—oil is a finite resource, whereas data is near infinite, for example—but it is still a useful analogy. Crude oil, extracted directly out of the ground, does not have an immediate value as such. It is only after that oil has been refined—to aviation fuel or petrol, for example—that it gains in value.

The same is true with data: the requirement for refinement. Raw data, in and of itself, is of limited value. Only once we have effectively processed that data—analysed it and made sense of it—does it truly become valuable. We convert data into information, and ultimately into insights that we can use to inform decision making.

Increasing our probability of success
We are forever making decisions under uncertainty. As a means of quantifying uncertainty, we need to apply probabilities. This is where data can support us.

Taking a data analytic approach to decision making does not guarantee success—we cannot eliminate uncertainty. However, we can at least seek to quantify it, in order to increase our probability of success—such that we are right more often than we are wrong.

Data storytelling
“If you can’t explain it simply, you don’t understand it well enough.”—Albert Einstein

Think of data science akin to a scientist conducting experiments in a laboratory to gain a better understanding of a subject. Data scientists, armed with data and computers, seek to make new discoveries—then data visualisation allows them to communicate the results of those data-driven experiments clearly to an appropriate audience. This is a critical skill.

In a business, when you are analysing data, you may not necessarily be the key decision-maker yourself. Part of your role may be to persuade and convince a decision-maker—guided and informed by the data—with a powerful and effective story based on data fact not opiniated fiction.

Next time you are viewing a chart, think with a critical eye and ask yourself:

• How effective is it at communicating the story it is claiming to tell? Indeed, what is the story being told?
• What sort of scale is being used—linear or logarithmic?
• Which datasets have they chosen to display? What is their source?
• Have they chosen to control for population size—if we are looking at demographics? A matter of aggregates versus averages!

Slight tweaks to the setting in a visualisation can potentially convey very different messages.

With this subtlety in mind, we can see that there is scope for abuse in data and statistics.

When consuming data visualisations, we should do so with a healthy degree of scepticism. Look closely at how charts and data are presented—make sure there were no ulterior motives behind them. Even in the absence of sinister motives, poor choices in visualisation design can nonetheless be misleading.

Closing the perception/reality gap

A great benefit of data is the potential to reduce the perception/reality gap which almost always exists in life.

When we make decisions, we often tend to do it based on gut instinct or our personal worldview—and worldviews will deviate substantially, subject to perception. Decisions based on reality rather than perception, will of course always be better-informed decisions.

Executed well, and with good quality data—there is tremendous potential for us to reduce this perception/reality gap, and make better, data-informed decisions in any arena.

Five core skills
The demand for people who can make sense of data currently far exceeds the supply, with this skills deficit only likely to widen in future. Expanding your analytical skills is a sensible and valuable endeavour for any professional. Here are the key areas of skill to focus on:

1. Decision making under uncertainty
This is really a mindset change to a position of acceptance, awareness, and appreciation of uncertainty.

2. Data visualisation and descriptive statistics
The power to communicate potentially vast quantities of data, quickly and clearly.

3. Quantifying risk through probability
The ability to use the latest available data to update our understanding and (probabilistic) beliefs. As new data comes to light, our view of the world should be updated with that most recent information. Things change, trends change. “When the facts change, I change my mind”—John Maynard Keynes.

4. Data integrity
Can you trust your data? Political opinion polling routinely shows there is always potential for biases in data collected from individuals. Being aware and conscious of biases which may exist in data is a starting point. Then, perhaps easier said than done, try and remove them, adjust for them, or allow for them when analysing the results.

5. Evidence-based decisions
With the widespread availability of data and the accessibility of tools such as data visualisation, I am hopeful we will see more data-driven decision-makers based on evidence, whether in the private sector or public sector—more decisions driven by reality, and reliable data, than by perception.

Developing and honing these skills will take some investment of time and effort, and of course there are always opportunity costs. Each of us faces the same dilemma: we have 24 hours in a day, there is a limit to what we can do, and we must decide where to allocate our personal resources. I would say the benefit of you being able to analyse data, make sense of it, and assist with better decision-making will typically far outweigh the negligible cost of acquiring those skills!

……………………………………………………………………………………………………….

James Abdey is Assistant Professorial Lecturer in the Department of Statistics at LSE. He is Course Convenor of LSE’s online certificate course Data Analysis for Management.