In a long, fascinating piece in the May/June issue of Foreign Affairs, Kenneth Neil Cukier and Viktor Mayer Schoenberger look at the emergence of “big data” and how it’s changing the world. “Using great volumes of information,” they write,
requires three profound changes in how we approach data. The first is to collect and use a lot of data rather than settle for small amounts or samples, as statisticians have done for well over a century. The second is to shed our preference for highly curated and pristine data and instead accept messiness: in an increasing number of situations, a bit of inaccuracy can be tolerated, because the benefits of using vastly more data of variable quality outweigh the costs of using smaller amounts of very exact data. Third, in many instances, we will need to give up our quest to discover the cause of things, in return for accepting correlations. With big data, instead of trying to understand precisely why an engine breaks down or why a drug’s side effect disappears, researchers can instead collect and analyze massive quantities of information about such events and everything that is associated with them, looking for patterns that might help predict future occurrences. Big data helps answer what, not why, and often that’s good enough….
The article is a great primer for the upcoming edition of GMNsight: Advancing Grantmaking journal, which is all about big data.