The Myth of Macroinnovation

Zachary Spencer [2009-05-26 05:27 AM]

I completely agree. Micro innovation is happening now more than ever, especially in the realm of web applications. The tools and technology in place that previously required hundreds of thousands of dollars are now available on Google App Engine.

Artists now have the tools and technology to create videos that speak to the soul with equipment that costs less than a car.

Musicians can record their own music simply by turning on their computer and running Audacity.

Micro Innovation is now alive more than ever!

hymanroth [2009-05-26 06:16 AM]

This post ties in with Eric Ries' views on lean start-ups and agility.

Whilst appreciating micro-innovation can be very effective in many contexts, there are some problems which, I believe, are just too big to be solved by anything other than a large and dedicated team of experts working in a cooperative rather than competitive manner.

The structure of the web itself is a good example. Most engineers would admit that if we could do it again, we would do it differently, do it better.

Small companies, under intense pressure from their investors to generate, if not revenues, at least some semblance of traction, haven't got the time or the energy think about the big picture.

This is how shanty towns build-up, all higgledy-piggledy.

I'll take urban planning any day.

Jacem Yorob [2009-05-26 06:45 AM]

The linked NYT article gets so many things wrong:

Solutions won’t come from the next new gadget or clever software, though such innovations will help. Instead, they must plug into a larger network of change shaped by economics, regulation and policy.

This is a false dichotomy. To be broadly useful clever gadgets and software must fit into these larger systems, and that's exactly what they'll do. With common interfaces, platforms for mashups, and near free distribution mechanisms, it's trivial for small shops to plug into and massively scale their innovations.

Progress, experts say, will depend on people in a wide range of disciplines, and collaboration across the public and private sectors.

Who will find it easier to overcome traditional organizational boundaries, large bureaucratic corporations, or small one to a few person operations which never had those boundaries in the first place?

These systems are animated by inexpensive sensors and ever-increasing computing power but also require the skills to analyze, model and optimize complex networks, factoring in things as diverse as weather patterns and human behavior.

I can't say how it's done everywhere, but in most shops I've seen, this kind of modelling is done by a small group, if not a single individual, with expertise in both mathematical modelling and the particular domain. I believe it's actually pretty difficult to divide these kinds of tasks across several workers, because you basically need to weigh all factors simultaneously when making design and implementation decisions. As such, small shops and individual consultants will have just as much capability as larger organizations, especially when you factor in how many modelling problems are now tractable with cheaply purchased or rented compute resources. I'm stretching a bit outside my area of expertise here, so I'd love to get input on this from others who've worked in complex system modelling at organizations large and small.

To be competitive in Internet search and some other Web services, which cater to hundreds of millions of users worldwide, a company must build data centers of gargantuan size,

While the resource requirement bar is set pretty high for a new search engine, the availability of cheap to free, dynamically scalable application hosting from the likes of google and amazon make it easier, not harder, to deploy a new service and have it scale automatically with demand.

In summary, the NYT article fails to take into account many of the technology trends this blog has publicized.

Joe Buhler [2009-05-26 07:21 AM]

I recently read the article by Malcom Gladwell in The New Yorker in which he talked about the David vs. Goliath principle - how underdogs break the rules.

Innovation disrupting and challenging the status quo has for the most part been introduced by individuals or small start-ups, the Davids of the business world and not the Goliaths. They either have been defeated, moved on to other areas or acquired the disruptor, which often has led to those innovations being able to scale.

The premise of this post is in the same vein and I support the viewpoint that it is at the micro not the macro level where innovation happens.

Scott Berkun [2009-05-26 09:22 AM]

Anytime anyone says "it's the end of" or "it's the beginning of" regarding innovation means mostly they have no sense of innovation in history. There have been and always will be large and small scale innovations is plentiful supply. The NYT Piece is simply an empty piece.

Scaling can be innovation - much of what Edison did was figure out how to scale power across an entire city. Others had made light bulbs and figured out much of how electricity could work. But he figured out how to make a scalable business. You could make similiar arguments for Ford and assembly lines. The entry of an new idea that changes a field should qualify as an innovation regardless of the dimension the idea sits on.

Governments and large organizations do innovate, as NASA (Moon landing), The Pentagon (100 years of weapons research), The U.S. Senate (Civil Rights), and on it goes. Is it hard and rare? Sure. But so are most innovations done by lone inventors and creators too. Many of the ideas big companies eventually get credit for come from individuals and small companies (dig up the history of the I-pod, the Apple-II, DOS, etc. you'll find both big companies and small shops)

Trend spotting on the history of innovation is nearly always bogus and distracting. It has *always* been difficult to conceive, develop, and market good ideas and always will be. The particulars of big/small/private/public/autocratic/distributed are interesting but mostly overstated differences. The particulars of a given idea, in a given field, in a given time determine a potential innovation's fate more than the macro trends described in Wired or the NYT.

Pas B [2009-05-26 09:45 AM]

I've worked in some very large companies. The only innovation I've ever seen has come from the self-motivation of individuals. Where the rubber meets the road is the extent to which they are or are not networked (which, ultimately, means as much or more so socially as anything else) into the larger organization, which determines their effectiveness at gaining support for those ideas and innovations.

I learned to value the individuals. At best, the corporation made their lives sufficiently comfortable to enable them to pursue their passions. All the rest coming out of the corporation boiled down to "newspeak".

John Blossom [2009-05-26 10:27 AM]

Interesting angle to the innovation story. As a research and advisory company we hear a lot about how to foster innovation as a goal of strategy setting in major organizations. But as you point out, middle management, which is risk-averse oftentimes, is rarely offered the incentives to fail frequently enough to allow the lessons that foster innovation.

Google's famous 20 percent "do other things" method is one way to foster innovation that leads to breakthroughs, and certainly the old Bell Laboratories was an interesting example of how to fund breakthroughs that could be channeled into marketable products and services. Innovation only happens when you have the ability to commit to fostering its long-term benefits in the face of short-term disappointments and perceived inefficiencies.

With that said, as I outline in my book "Content Nation" I do believe that social media is enabling highly scalable systems which foster both communications and innovation that will benefit both small and local economies as well as the global economy. We no longer need centralized organizations as much as we used to in order to foster innovation with global impact.

bowerbird [2009-05-26 12:04 PM]

it might help innovation if we remind ourselves of our problems.

the earth problem: how can the planet survive our pollution of it?
the animals problem: what happens when the animals go extinct?
the food problem: why don't we feed all of the world's people?
the overpopulation problem: how many people are too many?
the energy problem: where will the energy we need come from?
the transportation problem: how will we move ourselves around?
the violence problem: when will we stop all the hurting and killing?
the weapons problem: how do we beat them into plowshares?
the greed problem: how do we curb our insane desire for "more"?
the hatred problem: what do we change our treatment of "others"?
the communication problem: can we eliminate misunderstanding?

-bowerbird

gnat [2009-05-26 01:46 PM]

@hymanroth: "The structure of the web itself is a good example. Most engineers would admit that if we could do it again, we would do it differently, do it better." Sure, but that doesn't invalidate my point that you *can't* do it again differently. It's bloody hard to make "a large and dedicated team of experts working in a cooperative rather than competitive manner". It's bloody hard to change the world. Put the two together and you've got the hardest task in the pile.

@Jacem: "This is a false dichotomy. To be broadly useful clever gadgets and software must fit into these larger systems". So true! It's amusing that an article making sweet love to "complex systems" failed to realize that the existence of large projects doesn't mean the end of small ones. It's an ecosystem: the presence of dinosaurs doesn't mean the mammals have all gone.

@Pas: "The only innovation I've ever seen has come from the self-motivation of individuals." So true! The trick of building a successful company appears to be creating space and roughly steering those self-motivated individuals. You could argue that's been Google's whole management philosophy.

Thomas Lord [2009-05-26 04:50 PM]

"Macroinvention" emphasis is more a reflection of the distribution of capital and its socio-political implications than it is about efficient allocation of capital. Big Pharma is a good example, so is the biofuel effort from BP, UCB, Lawrence Berkeley, and Indiana, and frankly, so is Web 2.0: all are the industrialization of "innovation" and erection of high barriers to entry to protect an already illegitimate distribution of capital.

-t

Falafulu Fisi [2009-05-26 05:48 PM]

Nat said...

...but scaling is not innovating.

Agreed with you on that point Nat.

hymanroth [2009-05-27 02:15 AM]

@gnat -I'm not disagreeing with you, I'm just saying that many of the real game changers did not come out of a garage, for example:

c / unix - BELL
mouse-based GUI - XEROX
http protocol - CERN
tcp/ip - DARPA
SQL - IBM
Postscript - ADOBE

Thomas Lord [2009-05-27 08:53 AM]

hymanroth,

Well, hmm.

Multics was an example of "macroinvention" - lots and lots of people working on various parts. C and unix happened when a couple of guys were able to score a mini-computer and go off on their own for a couple of months to get it started.

The mouse-based GUI at Xerox came initially and in smalltalk form from small teams.

Ditto for http, as far as I know.

I'm not familiar enough with the others to say.

Those things didn't come "from a garage" in the sense of a funded-by-the-inventor kind of side project, that's true. They all came from lucky bastards who to some extent got "paid to think". But they weren't "macroinvention".

Two examples of macroinvention:

One is the drug discovery process where you, big pharma guy, might hire a hundred workers to systematically test N+1 thousand compounds to identify a few dozen or hundred candidate compounds and so forth. Think of the army of brooms in the Sorcerer's Apprentice.

Another is the Berkeley Energy Biofuels Institute working with BP. Crudely speaking (because I don't know that much about the details) the notion there is that you have smallish (call 'em, "nonmacro") teams of biologists who have lots of ideas about cellulosic conversion and whatnot - too many ideas - and the challenge is to select which ones to work on. So at EBI the notion is to winnow those choices down by consulting with dozens or hundreds of folks from the industrial side - folks who, if a scientific breakthrough is found, would have to figure out how to implement an industrial scale facility to crank out fuel in high volumes.

In both the drug discovery case and the biofuel case there is no solitary inventor or small group forming a vision and working it through - it's more like a few hundred people running a search algorithm like a chess playing program does but by hand.

Google is (by report) a kind of hybrid: there's the "20%" thing and the formation of product teams by consent, but atop that there's the internal market for resources and the internal "filtering" process of projects - so it's kind of a big search algorithm but uses "micro-" invention to generate new ideas. (I'd put them more on the macroinvention side, if we're going to polarize, just because the "micro-" that they do host is completely contextualized in a "macro-" environment.

The "micro/macro" dichotomy is a little contrived anyway but isn't vacuous. It's never quite "either/or". What was Thomas Edison at Menlo Park: (a) micro or (b) macro? (Correct answer: (c) "effective" :-)

-t