Today, we have more information than ever. But the importance of all that information extends beyond simply being able to do more, or know more, than we already do. The quantitative shift leads to a qualitative shift. Having more data allows us to do new things that weren’t possible before. In other words: More is not just more. More is new. More is better. More is different.
Of course, there are still limits on what we can obtain from or do with data. But most of our assumptions about the cost of collecting and the difficulty of processing data need to be overhauled. No area of human endeavor or industrial sector will be immune from the incredible shakeup that’s about to occur as big data plows through society, politics, and business. People shape their tools—and their tools shape them.
This new world of data, and how companies can harness it, bumps up against two areas of public policy and regulation. The first is employment. Big data and associated algorithms challenge white-collar knowledge workers in the 21st century in the same way that factory automation and the assembly line eroded blue-collar labor in the 19th and 20th centuries. But there are benefits: Big data will bring about great things in society. We like to think that technology leads to job creation, even if it comes after a temporary period of disruption. That was certainly true during the Industrial Revolution. To be sure, it was a devastating time of dislocation, but it eventually led to better livelihoods. Yet this optimistic outlook ignores the fact that some industries simply never recover from change. When tractors and automobiles replaced horse-drawn plows and carriages, the need for horses in the economy basically ended.
The second policy area is privacy. Of course, privacy was an issue even with “small data,” but it’s a much greater challenge in the big data era. Here, too, more is different. The nature of securing personal information changes when potential privacy threats happen not daily or hourly, but 1,000 times per second. It also changes when the act of collecting data happens invisibly and passively, as the byproduct of other services, rather than overtly and actively. It’s hard to imagine how classic privacy law will work in that world, or how a person whose privacy has been violated would take action—or even be aware of the situation.
It gets worse. A basis of privacy law around the world is the principle, enshrined by the Organization for Economic Cooperation & Development privacy guidelines, that an entity should discard data once its primary purpose has been fulfilled. But the whole point of big data is that we ought to save data forever because we can’t know today all the valuable ways it might be used tomorrow.
Big data will change business, and business will change society. The hope, of course, is that the benefits will outweigh the drawbacks, but that is mostly a hope. The big-data world is still very new, and, as a society, we’re not very good at handling all the data that we can collect now. We also can’t foresee the future. Technology will continue to surprise us, just as it would an ancient man with an abacus looking upon an iPhone. What is certain is that more will not be more: It will be different.