For those of you that missed it, the February 27th-March 5th edition of The Economist featured a special report on the over abundance of data, and where we’re heading with it. This may seem like a boring topic at first, but if you start reading , you’ll quickly realize why I continue to be so interested in social media, especially the broader social-cultural and global implications of  all the content/data that we are feeding into “the cloud“.
Rather than providing you with a summary of the article (which hopefully you will read in full), I am providing some excerpts which IÂ highlighted myself while reading:
- There are 4.6 billion mobile-phone subscriptions worldwide (though many people have more than one)
- By 2012 the amount of traffic flowing over the Internet annually will reach 667 exabytes
- Farecast can advise customers whether to buy an airline ticket now or wait for the price to come down by examining 225 billion flight and price records.
- As the world is becoming increasingly digital, aggregating and analyzing data is likely to bring huge benefits in other fields as well. For example, Craig Mundie and Eric Schmidt sit on a presidential task force to reform American health care (Personal note: Take a look at Google Flu Trends)
- Digital records should make life easier for doctors, bring down costs for providers and patients and improve the quality of care. But in aggregate the data can also be mined to spot unwanted drug interactions, identify the most effective treatments and predict the onset of disease before symptoms emerge. Â In a world of big data the correlations surface almost by themselves (Personal note: Take a look at Patients Like Me).
- The flood of data from sensors, computers, research labs, cameras, phones and the like surpassed the capacity of storage technologies back in 2007
- The amount of reading people do, previously in decline because of television, has almost tripled since 1980, thanks to all that text on the Internet.
- In 2004, Wal-Mart peered into its mammoth databases and noticed that before a hurricane struck, there was a run on flashlights and batteries, as might be expected; but also on Pop Tarts, a sugary American breakfast snack (Personal note: One can only imagine the millions in increased revenue once they were put front and centre)
- Best Buy, a retailer , found that 7% of its customers accounted for 43% of its sales, so it reorganized its stores to concentrate on those cusomters’ needs.
- Airline yield management improved because analytical techniques uncovered the best predictor that a passenger would actually catch a flight he had booked: that he had ordered a vegetarian meal.
- A free programming language called “R“; lets companies examine and present big data sets, and free software called Hadoop now allows ordinary PCs to analyze huge quantities of data the previously required a supercomputer.
- Data are the coin of the realm. Companies that get this concept have a big lead over others.
- If users knew how the data were used, they would probably be more impressed than alarmed.
- Creating new economic value from unthinkably large amounts of information is Google’s lifeblood. That helps explain why, on inspection, the market capitalisation of the 11-year old firm , or around $170 billion, is not so outlandish.
- Zynga, an online games company, tracks its 100m unique players each month to improve its games.
- Cleanscores puts restaurants’ health-inspection scores online; other sites list children’s activities or help people find parking spaces.
- One obstacle is that most countries lack America’s open-government ethos, nurtured over decades by laws on ethics in government, transparency rules and the Freedom of Information Act.
- Public access to government figures is certain to release economic value and encourage entrepreneurship. That has already happened with weather data and America’s GPS satellite-navigation system that was opened for full commercial use a decade ago.
- The web project We Feel Fine by Jonathan Harris and Sep Kamvar, captured every instance of the words “feel”; or “feeling”; on Twitter, and matched it to time, location, age, sex and even the weather.
- If a picture is worth a  thousand words, an infographic is worth an awful lot of data points.
- Jeffrey Heer of Standford University helped develop sense.us, a website that gives people access to American census data going back more than a century. (Personal note: Be sure to take a look at data.gov and apps.gov as well)
- Tableau Software, co-founded in 2003 by Pat Hanrahan of Stanford University, does for visualizing data what word-processing did for text, allowing anyone to manipulate information creatively.
- The software Gapminder elegantly displays four dynamic variables at once
- In recent years there have been big advances in displaying massive amounts of data to make them easily accessible. This is emerging as a vibrant and creative filed in melding the skills of computer science, statistics, artistic design and storytelling.
The shocking thing about all of this is that many public sector organizations are sitting on a ton of data and yet are still making decisions based on hunches, personal preference or political pressure. I can think of plenty of departments that spend a great deal of money on web analytics software, email databases, valuable primary research , etc.. but don’t spend a dime on hiring someone to provide them with ongoing “actionable insight” based on that data. As a result, senior leaders are left with increasingly prettier 3D graphs and colourful out of context pie-charts created from standard reports, which aren’t tied to any objectives.