There’s a fascinating report in the Feb27 issue of The Economist about “the data deluge,” which everyone should read.

Photo courtesty of The Economist

Photo courtesty of The Economist

Of course, the article is weeks old by now, and no one is likely to go back to find it.

The topic, though, is fascinating because it’s a snapshot of something that’s been happening for a long time and that’s still in play: As a race, humans are creating more and more data, and we are at a point when we can actually see some of the results of this trend with the naked eye.

In fact, data will only become more important to each of us—individually and as a global community—in more and more practical, structured ways in the coming months and years.

Some economists opine, “Data are becoming the new raw material of business,” and we–teachers, students, buyers, sellers, gardeners, technologists, healthcare workers, bankers, governments, all of us–would be remiss if we don’t take time to understand what this might mean.

Here are a few quotes/highlights from the article, which will spawn several upcoming shorter blogs about these trends in data and policy that are shaping human daily life.


“Torture the data long enough and they will confess to anything.”


“A new kind of professional has emerged, the data scientist, who combines the skills of software programmer, statistician and storyteller/artist to extract the nuggets of gold hidden under the mountains of data.”


“Just as the microscope transformed biology by exposing germs, and the electron microscope changed physics, all these data are turning the social sciences upside down…Researchers are now able to understand human behavior at the population level rather than the individual level.”


“What we are seeing is the ability to have economies form around the data—and that is a big change at a societal and even macroeconomic level.”


“Data are becoming the new raw material of business: an economic input almost on par with capital and labor.”


“The amount of reading people do, previously in decline because of television, has almost tripled since 1980, thanks to all the text on the Internet.”


The data-centered economy is just nascent…You can see the outlines of it, but the technical, infrastructural and even business model implications are not well-understood right now.”


“Best Buy, a retailer, found that 7% of its customers accounted for 43% of its sales.”


“Wal-Mart is a good example. The retailer operates 8,400 stores worldwide, has more than 2M employees and handles over 200M customer transactions each week.”


“Visa, a credit-card company, in a recent trial with Hadoop [a new technology]  crunched two years of test records, or 73 billion transactions…The processing time fell from one month with traditional methods to a mere 13 minutes.”

© 2009 John Dila