How do we measure innovation?

In response to the IEEE’s report on Patent Power, which lists the top companies ranked by number of patents, Ari Shahdadi and Brad Burnham made trenchant comments in email that I thought were worth sharing (with their permission):

Ari wrote:

The main article is sad to read, with choice quotes like this: “Clearly, the global recession seriously hampered innovation in the United States.” If I’d like to do anything, it’s end the use of patenting statistics as a metric for innovative activity, especially by groups like the IEEE.

Brad responded:

Amen – R&D spending is also a bad indicator because so much is wasted in big companies. The methodology should have something to do with end user utility. Facebook has had a bigger impact on more lives than IBM and they don’t spend a fraction of what IBM spends on R&D or on patents.

I totally agree with both Ari and Brad, but just wishing that people would use another metric won’t make it happen. How might we construct a metric that would reflect the transformative power of the web (no patents), Google (nowhere near as many as their innovations), Facebook (ditto), Amazon (ditto, despite the 1-click flap), Craigslist, Wikipedia, not to mention free software such as Linux, Apache, MySQL and friends, as well the upwelling of innovation in media, maker culture, robotics… you name it: all the areas where small companies create new value and don’t have time, money or inclination to divert effort from innovation to patents?

I’ve long been mindful of the power of synthetic indexes. How many people who religiously check the Dow or the Nasdaq know which companies it actually represents?

It seems to me that there ought to be a way to measure the introduction of new products, and rank them by novelty and by widespread acceptance, in some way that reflects a more substantial measure of innovation and its impact on the economy.

I’d love your thoughts about what could go into such a measure.

tags: ,
  • lesa mitchell

    Tim,
    We have been pushing the notion of new metrics for some time and this discussion desparately needs the voices of real entrepreneurs – thank you thank you thank you

  • gregorylent

    the deep answer .. the brain-wave coherence and breath-rate of employees …

    innovation is inherent in the nature of an undisturbed consciousness …

    ask any yogi

  • Mau

    What always bothers me about innovation indexes is the use of “Number of patent applications at USPTO”, that’s not representative at all because USPTO’s definition of what is patentable or not is heavily criticized and limits non-US companies/residents who can’t afford such process.

    I’d prefer instead:

    -Number of PCT applications
    -Number of patent applications by residents at local patent office/1.000.000 residents

  • Christian Gray

    I agree there should and could be better ways to evaluate innovation.

    I work daily with companies that have large IP portfolios and very large R&D budgets, here are some additional thoughts on innovation metrics.

    RE: Buzz and high profile
    Some combination of standard news plus social media mentions, frequency and trending (similar to PR & communications tracking)

    RE: Impact factor
    So must of us know about Page Rank/Google, some know about the Science, Technical and Medical (STM) Publishing world and the importance of an impact factor. A number of STM publishers and related data providers track and measure the number of times a peer reviewed scientific article is cited by other researchers, leading to come conclusions about the value of that research

    RE: Adoption
    Clearly this could be key, trouble is getting reliable data from non-public companies, and what to do about non-like comparisons. Measuring number of users for an installed software like MS Office v. Open Source Linux

    RE: Ripple effect
    New technologies created based off previous discoveries and innovations

    RE: Impact over time
    How about looking at the lifespan of a technology – phone v. fax v. cellular phone v. IM, etc

    Just a few quick thoughts, I’m interested to see more on the subject.

  • Bill Seitz

    Revenue from products launched in past n years.

    Obvious problems:
    * even public companies don’t report revenue at this level of detail
    * distinguishing truly-new from zombie-new (tweaked clothes detergents) is rather subjective.

  • Bill Seitz

    Meta-question: what’s the point?
    * to compare/rate different companies
    * to say “hey, that public policy change last year has killed innovation, undo it”
    * to give marginally-employed journalists something to write about
    * other???

    The purpose of the metric should drive its design, since any metric is a filter on reality.

  • mikem

    I’ve wondered about this myself, primarily because I think a better innovation metric will be required in the task of reforming the patent system itself.

    The trickiest bit about coming up with such a metric is finding a way to prevent gaming of the system. Even things like ‘widespread acceptance’ can be gamed if there is benefit (such as differential funding/press based on scores) for doing so.

    Afraid I don’t have anything to offer right now in terms of specific metrics, though I hope to post back here later after I give it some thought. One refinement of the question I can think of is to distinguish between pre-release and post-release measures of innovation; i.e. measures that can be applied to an innovation before it is released to the market, and measures that are only possible after it can be released. These two types might have rather different uses.

  • Michael F. Martin

    One crude approximation: look at how turnover in revenue increases upon intro of new features/methods. Many innovations are incremental.

    People don’t look at turnover rates because they’re hard to back out from public financial statements. But managers can see this easily if they want to.

  • Sergio Majluf

    What is innovation?

    I believe there are pending efforts towards defining this (buzz)word/term/concept, before being able to measure it in a consistent, sustainable and long-term way.

    Also, and recognizing that indeed US patents and the USPTO is the reference everybody looks into for “recent innovation”, we live indeed in a globalized culture, where “innovators” are to be found, certainly, in places other than the US.

  • Clark Ritchie

    It seems pretty short-sighted to claim that Facebook has had a bigger impact on more lives than IBM. IIRC IBM basically invented the desktop PC market, not to mention the impact their mainframe business has had on industries worldwide, such as finance and healthcare.

  • Steve Gordon

    There has been some published academic research addressing this question. See the following:

    Anne Martensen, Jens J. Dahlgaard, Su Mi Park-Dahlgaard and Lars Grønholdt, Measuring and diagnosing innovation excellence – simple contra advanced approaches: a Danish study, Measuring Business Excellence, Vol 11, No 4, 2007, 51-65.

    Jean O. Lanjouw and Mark Schankerman, PATENT QUALITY AND RESEARCH PRODUCTIVITY: MEASURING INNOVATION WITH MULTIPLE INDICATORS, The Economic Journal, 114 (April), 441–465.

    Marian Fitzgibbon, Of shadow and substance: The dilemma of measuring innovation, IBAR, Journal of the Irish Academy of Management, Vol 21, No. 2, 2000.

  • Jeff Ubois

    Seems like measurement is going to have to start with choosing useful definitions of innovation. Piero Bassetti (http://www.fondazionebassetti.org/) suggests innovation is ‘achievement of the improbable;’ Drucker said it was ‘change that creates a new dimension of performance,’ Altshuller said it’s a ‘reconciliation of contradictions,’ and others have said it’s the application of capital to scientific discovery. Proxies for some aspects of all that might include investment, publications, citations, changes in spending patterns as old systems are abandoned, training time (especially on the job), job loss…but I wonder if some numeric measurement might just give us the illusion we know what we’re talking about…

  • Lindsay Watt

    Bill’s got the best metric, but as he says, it’s pretty difficult to get and easy to game.

    Also, there’s probably no single metric, rather an index of a bunch of different metrics that roll up to an index. The index has some leading and lagging indicators that tell us what’s driving innovation and how it is trending.

    Here are a bunch:
    * Number of new companies started
    * Employment at those companies
    * Funding at those companies
    * Revenue of those companies at year x
    * Rate of technology transfer from universities to companies
    * Rate of visa applications by foreign engineers/scientists
    * New fundraising by venture capitalists
    * % of GDP coming from companies that are less than x years old
    * Growth rates of public companies

  • ePonti

    It would require substantially more resources to switch to a more social approach to measure innovation. Impact and diffusion. You will need a combination of qualitative and quantitave analysis. Check this rough idea: an expert panel to list major innovations in their domain, assess their transformational potential, and a survey to check diffusion and re-asses transfomational impact.

  • Michael F. Martin

    To be more precise, I meant track inventory turnover ratios before and after new products or processes are introduced by a company.

    I’ll probably do a short post about this on my blog brokensymmetry.typepad.com

    Clicking through on the “Accounting” category brings up some posts I have done on this subject in the past, which is of great interest to me (I’m a patent lawyer by vocation and an investor by avocation).

  • Mauricio Longo

    While I agree with desire to find other ways to measure innovation, I find Brad’s example a bit short sighted.

    IBM introduced the PC to the world and effectively made possible the very existence of a Facebook in the future. How do we know what IBM is cooking up in its labs while Facebook is putting to use lessons learned from folks such as LinkedIn, Google (Orkut) and others and recooking it.

    This is obviously tough to measure with any sort of precision. I feel confident, though, that there is quite a bit of innovation happening in the US. I feel there should be more worrying about all the manufacturing capacity that has been transferred elsewhere.

  • Tim O'Reilly

    Clark, Mauricio -

    I don’t think Brad was comparing IBM’s entire history of innovation with Facebook’s. Just the impact of Facebook in the past couple of years.

    But you may well be right that even in that time period the comparison is flawed.

    But Brad’s point is still a good one. Measuring the relative rate of innovation by the two companies based on the number of patents filed is a poor measure of the real impact.

    And taking the example of the IBM PC even at face value – was the value of that innovation by IBM expressed in what was patented? It was in fact that it was published and easily copied by others that made the PC such a substantial innovation.

  • Tim O'Reilly

    For those who are deeply interested in this topic, there are additional comments over on the buzz thread that I also started:

    http://www.google.com/buzz/timoreilly/TUZ2kg8FTip/I-posted-this-on-my-blog-at-http-radar-oreilly-com

  • Lanie Tankard
  • Chris Spurgeon

    Maybe some sort of annual poll with questions along the lines of “name as many new “X” things you’ve started using this year as you can” where “X” can be “computer tools”, “appliances”, “websites”, “TV shows”, you name it. The rise and fall of the numbers of things people can think of from year to year could be a metric of innovation.

  • ian

    depends on what you are trying to measure. my problem with the economist conf we went to this week was nobody made a reasonable definition despite non-stop talk about innovation. one could easily argue patents granted are the opposite kind of measure, but patents that are associated with revenues, incorporated into products, etc…create real value. think about patent trolls v. IBM’s $1b+ annually in free cash flows as a result of licensing

    if one knows what one wants to measure, there’s a better chance of being successful at doing so.

  • Michael F. Martin

    Here’s a short post explaining how product and process innovations can increase turnover ratios (and hence return to equity) WITHOUT affecting profit margins: LINK.

  • M. Edward (Ed) Borasky

    Like it or not, the way we measure economic time series is in terms of *dollars* and *time*. I was just re-reading Kamien and Schwartz (Market Structure and Innovation, http://meb.tw/cF1nw6) a few days ago and they seem to have uncovered what few answers there are on this subject. In any event, “How we *measure* innovation” is clearly described therein.

  • Michael F. Martin

    Ed Borasky,

    That’s a great observation. Thanks for the link, which looks interesting.

    A fundamental problem with the design of financial statements and accounting rules is one that every undergraduate electrical engineer would spot: Time and frequency are a Fourier pair. If you sample a time-series only once a quarter, you spot trends only twice a year!

  • David

    A lot of measuring innovation is about measuring inputs (dollars spent, patents obtained, visas applied for, etc.) What if we measured outputs? Chris Spurgeon mentioned counting new devices, but I would advocate for something like the consumer price index. We choose a list of tasks, and track the cost and time of completing them (e.g., verifying a math proof). Whether to include externalities, like the environment is probably worth considering as well.

    David

  • Jim Stogdill

    I think maybe our well deserved distaste of software patents has created an over reaction against the patent system in general. It seems to me that patents still have some gross correlation with innovation across the economy as a whole. For example, I think this guy should still be able to get patents and that they are very much correlated with innovation: http://archive.sciencewatch.com/jan-feb2000/sw_jan-feb2000_page3.htm

    The idea that Facebook has had a greater impact that IBM is kind of sticking in my craw though. Zuckerburg’s “epiphany innovation” couldn’t exist without the likes of IBM’s grind-it-out science-into-technology innovations. Facebook may have made more “apparent” impact in the last few years than IBM, but if revenue can be considered some reasonable surrogate of value, then IBM is handily beating facebook in real impact below the waterline. The value is just buried in the stack a bit so it’s hard to see it when slaying zombies.

  • Jeff Alexander

    I’m with Bill Seitz. Professionals who deal with metrics know well the adage that “you make what you measure.” Everyone seems to think that “innovation” is inherently good, and therefore we need to measure it so that we can produce more of it. Why should we make that assumption? You can easily argue that there are certain innovations that are simply destructive, rather than creatively destructive.

    It’s better to step back and think about what distinguishes “constructive” innovation from just plain innovation, and derive metrics which measure THAT type of activity.

    And R&D spending, # of R&D FTEs, and patent counts are waaaay too early in the innovation life-cycle to predict with any accuracy which firms PRODUCE more innovation. Apple is another great example of a firm with low R&D spending and extremely high return on innovation.

  • Tim Kastelle

    The Community Innovation Survey is actually a pretty useful tool. It is used within the OECD, particularly in the EU countries. It does pick up on process innovations, and several of the other more subtle forms of innovating. There is some information about it, results, and a link to the survey form here:

    http://www.bis.gov.uk/policies/science/science-innovation-analysis/cis

  • James Love

    In the area of medicine, one can measure the benefits of innovation through clinical evidence regarding the impact of an innovation on health care outcomes. These metrics are not perfect by any means, but they are rationale, even with noise and imperfect information. The issue isn’t how many patents does Pfizer or Amgen hold, but what is their productivity in terms of improvements in health outcomes (measured against the state of the art before a new product is introduced), over the period measured. More on this here:
    http://keionline.org/prizes

  • Lanie Tankard

    Why not move out of the realm of technology and business, and even the sciences, to consider innovation in a broader, almost philosophical sense? What drives innovation in the arts and humanities? Can you measure what Rem Koolhaas is doing? Twyla Tharp? How large a role does the Zeitgeist play in all this? Consider biomimicry, and cases of innovation such as Velcro. Look at measurements that reflect what ePonti was suggesting above, such as: http://science.thomsonreuters.com/globalprofilesproject/

  • Eur van Andel

    Revenue from new product is a good measure for companies. Also: the speed with which they react to new opportunities. Apple didn’t want externall apps on the iPhone. There was no SDK. They made one, and now the 150k apps are the major selling point.
    Seeing that, they quickly made a computer with the same OS.

  • Carlos

    Well, there is no need to have only one ratio to measure innovation, mainly in the world we are creating.

    I would suggest to create an innovation observatory panel that collect and gathers information of innovative products, services, processes… and come up with different ratios like “value created/money invested”, “people reached”, “number of patents”, “people perception” … And then someone can come up with a formula to create a general ratio on innovation of every company.
    Not so easy.

  • Peter

    If you measure innovation by the number of people whose lives are affected by it either directly or indirectly then you are probably looking at some kind of pagerank measure. You’d need to define a ‘link’ so that derivative works passed on some value to their source of inspiration.

    A highly page ranked site on google will have content that is unique and delivers high value to many users.

    All this patent rubbish is a bit like trying to do game google. Cross licensing of patents is just cynical link swapping and link farming.

    If you’re interested I’ve done a bit more thinking in this area, so drop me a line :-)

    cheers

    peter

  • Lori E. Mazzola

    Innovation is the introduction of something new, a new idea, method, or device.

  • Hamranhansenhansen

    I measure innovation by how many things you make reality that used to be only science fiction.

    For example, in 1996 I read Neal Stephenson’s “The Diamond Age” which featured a computerized book that could be any book, and which was also a wireless Web browser. I thought the idea of a wireless Web was pure science fiction, decades away. There are many other examples of similar devices in science fiction.

    Then in 1999 I bought an iBook and an AirPort Base Station from Apple. In 2007, an iPhone. Now in 2010, an iPad.

    Now, reading “The Diamond Age” is an entirely different experience. That’s innovation to me. I can’t read that book the way I once did. My perception has been changed.

    So it isn’t just about changing the technology, it’s about changing us, changing culture. I can’t go back to when I thought the wireless Web was fanciful, or when I thought an interactive everybook was decades away. These things are part of the culture now.

  • Michael

    In the paper “Discovery of factors influencing patent value based on machine learning in patents in the field of nanotechnology” by Scott D. Bass Æ Lukasz A. Kurgan Scientometrics (2010) 82:217–241 innovation was indirectly measured by measuring the relative value of patents.

    Patents reflect technological change by representing inventive output in a manner that is standardised in different fields, countries and time periods. This enables the use of patent analytical techniques to reveal the following:
    (i) inventors approach(es) in light of the economic policies at the time;
    (ii) extent of commercialisation of academic pursuits (e.g. was the patent assigned/licensed); and
    (iii) outcome of the technology in terms of patent value.

    This study sampled the field of nanotechnology patents and identified some interesting trends (e.g. common features of high value patents) that could be used to predict future value of nanotechnology patents.

    Of interest, patents with a higher probability of performing well were:
    (i) patents with a greater number of outgoing citations revealing their scientific sources; and
    (ii) lodged by inventors who:
    a. previously had valuable patents in place; and
    b. had a longer history of inventing.

    Patents require novelty, then combine citation indexes (as used in academic literature etc)and licencing/assignment and you are likely to have a good index of innovation.

    However, it does not follow that research institutions such as universities have high innovation indexes: there are no universities or research institutions in the Top 100 PCT Applicant list in 2009.

    (I wrote a similar review on my blog at http://senseof1place.wordpress.com/2010/03/27/can-you-determine-future-success-of-your-inventive-pursuit/)

  • bustem

    Facebook? Wikipedia? Seriously? Let’s not confuse popularity with innovation, although Perez Hilton’s blog sure is changing people’s lives.

    All Facebook does is waste time, and it’s only a matter of time before it fizzles away. And Wikipedia is worse. It’s downright dangerous–more subjective than the New York Times.

    I mean, look at this crap from the Wikipedia entry on “innovation”:

    “product innovations develop customer support however at the risk of costly R&D that can erode shareholder return. Innovation can be described as the result of some amount of time and effort into researching an idea, plus some larger amount of time and effort into developing this idea, plus some very large amount of time and effort into commercializing this idea into a market place with customers.”

    Unbelievable.

  • Ruy de Queiroz

    @Michael: “there are no universities or research institutions in the Top 100 PCT Applicant list in 2009″. You may have missed the University of Califórnia, ranked 40th:

    40 -6 THE REGENTS OF THE UNIVERSITY OF CALIFORNIA US 321 -26

  • Tim O'Reilly

    Bustem -

    Citing a badly written Wikipedia entry to deny that Wikipedia is an innovation is like citing Fox News to show that television had no impact.

    This is a technology that replaced an entire industry – and I’m not just talking about the old print encyclopedias, but new digital encyclopedias from companies like Microsoft.

    Major disruptor of an established market, used daily by millions of people, and incredibly useful to anyone who doesn’t have a chip on his or her shoulder.

    And BTW, you might remember that Nature (the respected science magazine) did a study that compared the accuracy of Wikipedia and Brittanica, and found them largely on par.

    As to Facebook being a time-waster, it is that. But so is television. So is radio. So is the telephone. But they are also major innovations that reshaped our markets and our culture.

  • Adam

    Let me contribute by a comment and question.

    Although there is ongoing discussion among innovation scholars on what is innovation, in a business context it can be approximated as succesful commercial introduction of a new idea. Or in blunt words, certain level of revenue stream from new product. Therefore, aforementioned “revenue from products launched in past n years” is quite correct. However, it gets trickier when applied to service innovation because it can consists just in incremental improvements without introducing brand new one (eg new design of a website). It is even more difficult for public sector as revenues are not measured purely in financial terms.

    Seen from this perspective, amount of RD spending is irrelevant, albeit being often used. Moreover, there is emerging discussion about further relevance of patents for innovation in academia.

    And the question: Why do we need to measure innovation anyway?

    The only reasonable answer to me seems ‘to design policy measures that will increase it’. However, this is more about understanding it than measuring it. And here is where the article corresponds with current attempts to better understand the nature of service sectors. See eg work of Ian Miles http://www.slideshare.net/IanMiles

    Adam, student at Manchester Institute of Innovation Research, MBS

  • Steve Wigney

    “Facebook has had a bigger impact on more lives than IBM”.
    Really?

  • Michael

    @Ruy de Queiroz Thanks Ruy – My humble apologies, you are right that the University of Califórnia ranked 40th.

    I was amazed not to see many US universities in the top rankings since licencing fees for universities are a huge revenue source.

    So internationally there is only one university – making up 1% of the top applicants.

    Further, University of Califórnia is not just in but well in, so it IP strategy out does every other university and most big research companies!

    Lets wait for the future to take place to see if this places the University of Califórnia in a better position than it’s peers.

  • michael.davisburchat

    The comment about being blinded by input seems insightful. Innovation, as I have come to think of it would regard the diffusion of ‘a different way of doing/experiencing something’. So there would seem to be some importance to measuring either the rate or breadth of adoption.

    Often the words new or better get used as well to inform ‘innovation’ planing. However those modifiers lead to what one comment-er is calling ‘zombie’ innovation, like ‘new tide’ for example. By starting with the imperative, to perform something differently, innovators are inclined to reframe a perspective before proposing an (innovative) solution.

    But, getting back to the topic of metrics..

    I humbly suggest two:
    1) the new sources of revenue the innovation attracts from a consuming public.
    2) the amount of creative destruction it wields in a marketplace – again measured in revenue.

  • Andrew Hargadon

    A clearly good and provocative original question. Several quotes come to mind, the first unattributed:

    “innovation is what you do when you don’t know what to do”

    It’s hard measure those things nobody has yet decided are worth measuring. By the time an innovation fits into a definable category, it’s no longer an innovation. Now it’s a matter to be decided by lawyers and historians. Reminding me of the second quote, by J.H. Fabre

    “History records the names of royal bastards, but cannot tell us the origin of wheat.”

    Comparing facebook to IBM is a bit of a red herring—who invented wheat, or the internet for that matter? Most innovations depend on the social/technical infrastructures of their time, making it dangerous to extricate and measure them.

    Not a useful comment, necessarily, but my two bits…

  • Suzanne Lainson

    Some good points being made in this discussion.

    1. What is innovation?
    2. For what purpose do we want to measure it?
    3. Often we only recognize an innovation in hindsight and by the time it has truly impacted our lives, it is no longer perceived as innovative.

  • Henrik Ingo

    “Clothesline paradox”: if I were to buy an electric clothes-drier, that purchase would show up on a statistic somewhere, from which someone would eventually infer that electricity consumption had increased in Finland. If, however, I get rid of my clothes-drier and let my clothes dry on a clothesline, there’s no statistic that says Finland is now using more solar energy. But that doesn’t mean a clothesline isn’t a good way to dry clothes. And there we have the clothesline paradox.
    (http://openlife.cc/onlinebook/barn-raising-and-clothesline-paradox-debian sorry for the self plug)

    Another similarly misguided “index” are the economists who infer that the price of a product (like a software license) is a measure of the _value_ a product provides the customer. Whereas in reality it usually is a function of competition vs lack of competition, negotiation ability, etc…

  • Dan Keldsen

    Innovation needs to be defined within context. Innovation for a government agency is much difference from innovation in a publicly held company, vs. a non-profit, vs. a 10-person software startup vs. a hospital, etc..

    Comparing innovation levels across disparate contexts is, well, ridiculous. There is no “innovation thermometer” that allows for direct comparisons. Innovation is in the context of whatever you decide it to be – product-oriented, cost reduction-oriented, service-oriented, revenue-oriented, etc.. Raw patent #s don’t mean much of anything – look how many patents are filed that don’t actually result in a real impact in the world. Defensive patents aside, many patents are just ideas that made it to the patent office first, not necessarily good nor “innovative” per se.

    For those who don’t believe innovation can or should be measured, it’s safe to say that you aren’t doing innovation, no matter how innovation might be defined. In my experience (having covered the idea management and innovation management space for 6 years), you need to measure and encourage both small i innovations (improvements, of the like that Toyoyta [normally] are famous for), and BIG I INNOVATIONS (what most people would call innovation – or disruptive innovations for Clay Christensen fans) and put SOME stake in the ground to see what level of innovation you had when you started caring about innovation, if those measures change from a raw #s perspective, but, importantly, what the outcomes of those innovation experiments DO for your business.

    If you simply want “more innovation” – you’ll either get nothing, or a lot of undifferentiated STUFF, with the odd useful idea/innovation mixed in. If you want PRODUCTIVE innovation, you need to put some parameters on that sucker, or the results are going to be a mess.

    We did a short research project on this last fall, view the webinar discussing the findings at:
    http://www.informationarchitected.com/resources/webinars/innovation-management-webinar-perception-outpaces-reality-archive/

  • Alex Tolley

    I would echo those who have asked, “Does innovation matter?”

    Innovation feeds into the productivity of te economy, and it is the results of that economy for individuals that matter.

    Consider an analogy. What if Tim asked, “How do we measure developments in art?”. Just looking around, it seems obvious that there is plenty of art and experimentation happening, in all the arts.
    More importantly perhaps, there doesn’t seem to be some shortage of art, quite the contrary it seems.

    My question would be, “is the level innovation a limiting factor in the growth of the economy?”. It isn’t clear to me that it is. Innovation (and invention) are products of the network effects between the stock of existing ideas and technologies. This stock is increasing exponentially. The internet has even improved the network effects by allowing innovations to spread more rapidly, reducing the patchiness of their distribution.

    Bruce Sterling once wrote an essay about technology in 2050, suggesting that people had become almost saturated with change. We may be seeing already signs of that today.

    In other words, innovation may no longer be a limiting factor in the growth of the economy.

    So while I think that intellectually, measuring innovation is interesting (although it may be as elusive as defining ‘art’), I do question its utility for the output of the economy.

  • Michael

    Our Brains, Our Limits & Our Innovation:

    We have a limited number of “learning tools” to perceive our environment. These tools evolved:
    1) by exploiting neuronal plasticity to “recycle” brain area(s) to optimize our perception for survival;
    2) to enable our understanding of information by putting it into a form that is perceptible to our brain – that is, to enable learning; and
    3) in the form of reading, mathematics, tool use, music and religious systems according to Cook (2010).

    Consequently, our culture has evolved by what was able to be understood by our brain. Our learning tools have evolved to filter information to what is, in evolutionary terms, necessary for survival.

    Are these learning tools, by filtering information, now limiting our innovation flow?
    That is, what learning was required for survival may be very different to what we now require in terms of innovation?

    Computer learning: Are our evolutionary next steps now unhindered?

    Using computers we can now remove the limitations of our human:
    a) learning tool filtration; and
    b) culture perceptions?

    See:
    1. Gareth Cook (2010) Scientific American Mind; Mar/Apr2010, Vol. 21 Issue 1, p62-65, 4p, 4
    2. http://radar.oreilly.com/2010/03/how-do-we-measure-innovation.html

    Post previously on my blog:
    http://1place.com.au/wptest.php

  • Pete Forde

    Innovation can only be measured in the past tense. There is no way to predict whether an invention will come to be seen as innovative — or forgotten without consequence.

    This post motivated a longer reply on our Rethink blog:

    http://rethink.unspace.ca/2010/4/2/stop-trying-to-measure-innovation-and-be-a-hero

  • Richard Weddle

    Metric
    The metric for evaluation of innovation is often subjective and foolish.

    Silly
    Saying Facebook is more important or innovative than IBM is silly. That just shows the writer knows nothing about what IBM is doing.

    Trial
    How about we value those who try things and reward their process with a bonus for success.

    Tolerate the Outsider
    The Incas never got the wheel, because my guess is their ‘Thomas Edison’ was killed in one of the human sacrifices.

    Why not work with the inventive folks and see what you can make real?

    Consider.

  • Kate Eyler-Werve

    A functional definition of innovation might be a product or service that changes the way people interact with the world.

    For example, Facebook has changed the way people interact with family, friends, companies and causes in complicated and interesting ways that simply can’t be measured by totting up the number of users. You can get much more detailed information on these changes using observational research methods and surveys. As an added bonus, using these methods in addition to user and sales numbers helps you beat the “clothesline paradox.”

    The reason a company would want to have information about how people have changed their behavior is to figure out new niches they can fill with their products and services. This is certainly a case where measuring innovation in the past can help spur future innovations.

  • Jim Rait

    I have always kept these definitions to remind me that innovation is about users/consumers not me the designer/developer:
    Innovation is the successful exploitation of new ideas. It is the process that carries them through to new products, new services, new ways of running the business or even new ways of doing business. – Cox Review
    “‘Innovation’ isn’t what innovators do….it’s what customers and clients adopt.”
    – Michael Schrage, author of the book Serious Play
    “Innovation is something that changes the life of the customer. It changes the life of the customer in some way, or the world in which the customer experiences things. That’s innovation.”
    -Kevin Roberts wrote in the book Lovemarks
    “Can’t innovate. Don’t innovate. Go nowhere. ” Sir George Cox.
    So the innovation measure has to do with market share over time and contribution to turnover over time too. one app in the marketplace has 100% share:as competitors come into the market soour share will drop and so on

  • moladi

    I agree with desire to find other ways to measure innovation

    I believe that there is a relationship between the success and the volume of people impacted.

    An innovation that would impact or improve Maslow’s hierarchy of needs, BOP or Physiological needs would have a greater impact than an innovation for the top of the pyramid.

    Even better if the innovation can create wealth for more than just the inventor.

    http://www.moladi.net

  • Andy Wyckoff

    The measurement of innovation broadly (beyond R&D and patents) has been underway at the OECD since 1997 — when a methodological manual (The “Oslo” Manual) was produced. It has been revised twice since then and is used by most OECD countries for years. The US (NSF) has just gone into the field with a questionnaire that draws on it. For a compilation of new (and old) innovation metrics, see http://www.oecd.org/innovation/strategy/measuring and the report: “Measuring Innovation: A New Perspective.”

  • Irene Rodríguez

    Very interesting this conversation.
    In fact at some companies, we consider not only producing patents, which helps in the intellectual propriety not only of your company, but also producing new products and services in terms of entrepreurship and benefit the customer with innovations to provoke new uses.

    It´s definitively a mix of criteria, we souldn’t skip market responses to the innovation.

  • Perla Ni

    This is a great post and discussion. What about nonprofit innovation? There’s even fewer good barometers for that. There’s few patents and very rarely any formal R&D expenditures (maybe there should be!).

    In my observation, there’s very little devoted to either spurring innovation or measuring it in the nonprofit sector. This is a shame because the nonprofit sector deals with some of the biggest and most difficult challenges of our time. Nonprofits deal with everything ranging from teaching kids how to read, to providing hospice care for the dying, to helping human rights workers safely document their findings. Of course, many of these issues require more than just technology innovation. But some, can be.

    Anyone who wants to talk more about this topic, which is one of my personal interests, can contact me. perlani(at)greatnonprofits.org