Big data: Global good or zero-sum arms race?

It remains to be seen if big data will catalyze exponential growth.

Last month, Netezza CEO Jim Baum gave a talk at the GigaOM big data event. If I’m honest, I was checking my email and missed most of it, but I do remember tuning in just in time to hear him say something like “big data is going to have a huge economic impact.”

I spend most of my days considering how the component pieces of this big data transformation will impact the corporate enterprise. Baum’s comment got me thinking, though, about a more meta question: Is “big data” a key to some kind of industrial revolution reboot? Or, is it just going to be expensive table stakes for previously simple-to-understand businesses?

For 200-plus years the industrial revolution* has been a kind of Moore’s law of human productivity. Over that period our economic output per person has been growing like clockwork, and whatever you think of the various political -isms that sprung from industrialization, this march of productivity has pulled a lot of people out of poverty and is cause for the first sustained increase in wealth across human history.

But like Moore’s law in a single core, our industrial revolution in advanced economies is kinda playing out. Our economy has been shifting for some time toward services that are proving to be impervious to order-of-magnitude productivity gains. The thousand-fold increases in productivity we saw on the farm and in the factory just don’t seem likely to happen in health care and other service intensive sectors.

Of course our economy continues to grow, but at a rate that is staying just a skosh ahead of population growth. And since the top 1% are taking all of that (and perhaps more), for the first time in American history parents are worrying that their kids won’t have opportunities better than their own. Voila! There stems the populist anger that feeds the Tea Party.

That’s the U.S.-centric view. Of course on a global basis there is tremendous growth as late-stage industrial revolution innovations are applied with vigor to developing economies. The 8% growth rates many countries are achieving will double their population’s wealth every 10 years. But for the U.S., achieving growth requires parallelism. Of course, in this context we call it globalism and it means if we can’t be more productive in one place, we have to take advantage of modern communications to do it in a bunch of other cheaper places. The problem is, after a few decades of those 8% overseas growth rates, there will be less comparative advantage for us to take advantage of and if we want continued economic growth, we really will need to find ways to be more productive.

So, that’s why Baum’s comment stuck in my head.

At the risk of way over generalizing, so far “big data” has mostly been about behavioral analysis to better target ads. Is that what Baum meant? That more effectively matching producer and consumer long tails through precision ad placement is going to fundamentally change the economy? That type of matching can promote economic activity, which is good, but I don’t see the link to fundamentally improved productivity. If this kind of innovation pulls another tranche of the bell curve out of poverty it will do it by putting more people to work doing the same stuff, not by making our economy fundamentally more efficient.

When I heard “huge impact on the economy,” my first thought was maybe it’s just a throw-away comment. Maybe he just meant the economy as seen through the narrow lens of his company revenues. But then I tried to think about this on a deeper level: What’s here that I haven’t considered? Could he somehow be saying that this is a catalyst for the next big phase of productivity growth in our 200-year-old industrial revolution? Is this the industrial revolution equivalent of nanometer chip design, which starts the next decade of doubling? Is it the thing that gets the middle class growing again and eases all this populist anger?

Yeah, that might sound kind of absurd, but that’s how my head works — a daily stream of ADHD-fueled big dreams immediately dashed on the rocks of reality.

(As an aside, half way through writing this I came across a prediction of the “wine and roses” we’ll all experience with this “New Information Age.” Don’t sweat the death of privacy, the surveillance state is highly unlikely.)

So, back to the question: Is big data an economic driver or just a must-have to be in the game?

As early as the 1950s it was obvious that robotic automation was going to fundamentally change manufacturing. As automobiles increasingly were built by robotic labor, the industry saw incredible productivity gains. The hours of human labor per automobile dropped by orders of magnitude over the next 30 years. Naturally, cars didn’t just get cheaper, they also got more complex and feature-rich. But anyone could understand the return on capital of installing robotic lines. What’s the return on capital look like for a Hadoop cluster?

It’s worth noting that robots didn’t just increase productivity, they also reshaped labor’s relationship with management. If you’re labor, competing with a robot sucks. This was presciently described by Norbert Weiner in his classic “The Human Use of Human Beings, Cybernetics and Society.” Of course we don’t need history’s warning to know that big data might have a dark side, too. If you don’t see it now, you will when you download a new car stereo software version and it resets all your radio station presets based on Toyota’s notion of people like you. Of course, for a car company to be as obnoxious as your software and search bar providers have long been, they have to learn as much about you as those software guys do, and that’s weird. We aren’t really used to the idea of a manufacturer knowing where we go and who we go there with.

There obviously are places where large-scale data and analysis will improve efficiencies and productivity. Particularly in areas like smart grid, where it will reduce the investment necessary in power plant construction, or financial services, where it promises to help fight fraudulent transactions. What else? Are there big opportunities for order-of-magnitude productivity gains out there that come to mind? Or is most of the value created by this “new information age” going to be in some mushy upper region of Maslow’s hierarchy? A kind of middle class feel-good machine that remains completely irrelevant to the working poor dreaming of their first homes?

Norbert Weiner was concerned that automation-based productivity gains would disrupt the working man and woman’s living. He held that concern in the face of the obvious and compelling productivity gains that were sure to flow through to GDP as wealth.

As we enter the big data era of the information age and give up what’s left of our privacy, I’d like to think that it will be for more than a zero-sum game of musical chairs to decide the next winners.

* For the purposes of this post, I’m treating the industrial age and the information age as two parts of one continuum.

tags: , ,
  • Canuck

    What’s happening in the world economy isn’t so much Zero Sum as Regression To The Mean.

    Now that more countries share the benefits of industrialization and the information economy, and there hasn’t been a world war for a while, huge-but-temporary income disparities should lessen, with poor countries (like China or India) will becoming relatively richer, and rich countries (like the US or Canada) becoming relatively poorer, even though we’re all growing in absolute terms.

    I don’t think there’s any magic rabbit — data or otherwise — that the OECD countries can pull out of a hat to restore the old world income disparities. Nor should there be.

  • Jim Stogdill

    Hey Canuck, I hope it wasn’t your take away from this that I’m arguing for income disparity. I’m simply arguing that the industrial revolution created GDP growth at a rate greater than population growth, significantly greater. That is still happening in developing economies, not as much in ours.

    If things like “big data” do in fact become big economic drivers, they can either do it by influencing our fundamental ability to drive wealth creation, or they they can do it by rearranging the deck chairs. I’m hoping their is more of the former.

  • http://drcoddwasright.blogspot.com Robert Young

    – There stems the populist anger that feeds the Tea Party.

    Unfortunately, the Tea Bag movement is funded by the 1% (actually, more like the .1%), while the troops are from the lower/stupider classes. The Great Recession *was not* caused by the targets of the .1% funders and the Tea Baggers; the Tea Baggers are just pawns. Stupid people are just easier to Bait and Switch. Go figure.

    As to productivity: physical production has been the source of increased productivity (but, without a theory of distribution, not to the advantage of the mass of population) in ways that simply can’t happen in “service economies”. There were studies done in the early days of Windoze which showed that Office (and the like) actually resulted in decreased office (lower case O) productivity, since it leads to the promotion of style (eye candy) substance (content).

    Moreover, what matters is the production of physical goods. Most folks, normal consumers, don’t consume the services produced by the service sector. It can be argued, and I am among those who do, the emergence of this service sector is a backdoor way to employ those tossed out of physical production.

    As to Canuck, no, it isn’t regression to the mean. Far from it. What’s happened is exactly the opposite. In 1980 the 1% took 8% of national income, while in 2007 they took 24%. Over that time the mean increased, but the median (a more equitable measure of central tendency of income) didn’t move. The USofA looks more like Haiti today than it’s ever done; ever. This is progress?

  • http://urbanodelacruz.tumblr.com Benjamin de la Pena

    “Are there big opportunities for order-of-magnitude productivity gains out there that come to mind?”

    Think #bigurbandata. Think urban logistics. Think of how the data about urban infra and systems will help us tweak and facilitate the productivity of the economic engines of the worlds economy: cities.

  • http://tim.oreilly.com Tim O'Reilly

    Ben,

    I love your insight that there are huge gains to be made in urban logistics. That’s a thought worth some deep minding and data mining.

    That being said, one of Jim’s key points remains: there is a litmus test for “impact on the economy.” If it doesn’t make life better for most people, it’s a technology with a negative impact, not a positive one.

  • Anon

    It’s important to know that the Tea Party is just a manufactured illusion; it’s not a truly genuine grass-roots social movement. It’s not bottom-up but top-down. The Koch brothers, the Walton family, and other billionaires are the masterminds behind this. They pay for bus tours around the country to setup faux protests. They find and use people stupid enough, usually Republicans, to advocate for the 1%’s interests such as lowering tax rates for millionaires and billionaires so that they can “create jobs”, which happens to be overseas. So the Tea Party is not real populist anger. The incident in Wisconsin is real populist anger. Hope you will correct the article.

  • Ralph

    Smart and better advertising could really affect the economy.for example a price comparison engine , or search enginene that display ads , that shows you advertisiments on disruptive innovations that relevant to your query could really increase productivity.

  • http://Http://globalgoodgroup.com Josh Bois

    These massive arrays of storage, whether from EMC or Netezza, are the future. One way or another the world’s data is being collected whether it is for government or private use. At this point there is nothing we can really do to stop it and we must atleast protect what data is collected and more importantly protect the security of the data that is being stored.

    Stricter standards need to be placed on the companies that are storing data even the smaller players which may only hold millions of records about people versus the billions and trillions of transactions that some big players hold. Even firms like InfoUSA and others make large amounts of data available to anybody with a checkbook for the most part and we as a world must be careful to place federal and state level laws on these firms holding the data to protect from the bad guys who already have enough access to this data. Our site GlobalGoodGroup.com is trying to bring about news for the Global Good whether that is about protecting data security or finding ways to feed those starving in Africa and I am glad that there are people writing about the large amounts of data out there, I just wanted to add a twist that opens peoples eyes to the needs we have to protect and secure this data.

    Our world is growing and so are the threats, let us unite whether as companies, organizations, individuals, or however to make sure the steps we do take forward ( legally, technologically, and morally ) are all aligned and on the right path. Just my 2 cents.