The Big Data Revolution

With decision-making being at the heart of effective leadership, it’s not surprising to see a variety of research studies into the factors affecting it. After all, leaders themselves want to know how best to make decisions that can ultimately decide the fate of their business. Now, they may not have to worry so much about the pressure on them alone to get it right; according to Professors at MIT Sloan School of Management, there is a shift taking place from relying on a leader’s “gut instincts,” to embracing a new method that involves data-based analytics.

 A recent article in the NY Times highlighted the “data flood” that’s sweeping through academia, business and government, with the data itself being seen as a new class of economic asset, like currency or gold. This is “the Age of Big Data” they call it. Similarly, an article in McKinsey Quarterly also suggested that this explosion in the volume of data available to executives may quickly lead to it becoming a new type of corporate asset that will cut across business units, and function much as a powerful brand does, representing a key basis for competition.

“This ‘Big Data’ revolution,” says MIT Sloan’s Professor Erik Brynjolfsson, “is occurring mainly because technology enables firms to gather extremely detailed information from, and propagate knowledge to, their consumers, suppliers, alliance partners, and competitors.”

Then why is it that despite the increasing availability of real-time digital data, many companies still aren't taking advantage of this tool to measure activities and customer relationships, and are instead relying continuing to rely on intuition and guesswork?

MIT Sloan is one of the first schools to realize this gap between the availability of Big Data and its effective utilization by executives. To help better understand the uses and applications for it, the School is offering a new executive education program called “Big Data: Making Complex Things Simpler” next month.

Taught by Brynjolfsson alongside Professor Sandy Pentland, the program will explore how Big Data changes the way IT interacts with the rest of the organization, and what has made the Big Data revolution possible. Last year, Brynjolfsson took part in research that analyzed 179 large publicly-traded firms, finding that the ones that adopted data driven decision-making were approximately 5 per cent more productive and profitable than their competitors. “Leading-edge firms don’t just passively collect this data,” says Brynjolfsson. “Instead, they actively conduct experiments to develop and test new products. Companies such as Amazon, eBay, and Google are good examples of successful firms that rely heavily on field experiments as part of a system of rapid innovation.”

Pentland shares similar views, emphasising that companies should think of data roughly the way they think of money. “It’s something you own, it’s something you can loan to people, but you want to get something back in exchange for it, such as a more useful search engine or real time traffic information.”

So the evidence in favour of Big Data is there; what seems to be more ambiguous is how to control and use it in the most effective way, and as Pentland describes, get something back for it. Also crucial is avoiding common pitfalls along the way, some highlighted in the NY Times as including “false discoveries” and biased fact-finding excursions.

As such, if MIT Sloan’s program proves to be popular in assisting leaders understand how to do so, we may well see more research and executive education focusing on Big Data strategies in the near future.

Further Information: 

"The Age of Big Data” NY Times

“Are You Ready for the Era of Big Data?” McKinsey Quarterly

“Strength in Numbers: How Does Data-Driven Decision-making Affect Firm Performance?” Research by Professor Erik Brynjolfsson, et al.


Professor Erik Brynjolfsson’s profile at MIT Sloan

Professor Sandy Pentland’s profile at MIT Sloan