Previous  |  Next



Tim O'Reilly

Tim O'Reilly

More on Techmeme and Financial Markets

My article about herding behavior on social networks, stock markets, and blogs, Facebook, the Quant Fund Meltdown, and the Techmeme Leaderboard, provoked a lot of comment, itself appeared on Techmeme, and has provoked some excellent comments, both on the entry itself and on other blogs.

Gabe Rivera, the creator of Techmeme, noted in the comments:

“I think there is a self-reinforcing effect with Techmeme, but I believe it’s (1.) overstated and (2.) as often good as bad.”

Fundamentally, I agree with Gabe, and I (slightly) regret bringing Techmeme into the discussion, although it has been a psychoactive reference that brought a lot of attention. Techmeme was, however, a grace note in the discussion, and I do want to bring attention back to the deeper thread that I'm following. But first, to the "good or bad?" question.

In an excellent follow-on piece, The Techmeme Pile-On: Good or Bad?, (which has led to yet another Techmeme thread) Matthew Ingram wrote:

Do pile-ons occur? Obviously, they do. And do many of those posts essentially consist of a re-posted excerpt and a “what he said” kind of comment? Definitely. Every system has a certain amount of noise. But I think on balance the posts that do add something to the conversation — and there are many of them on the average day — bring enough value to make it worthwhile.

Take Tim’s post itself (more meta): the sub-links [turned up by Techmeme] included Bob Warfield’s post, which I thought had lots of value, as well as an excellent one from Alexander van Elsas, who I hadn’t come across before. And in many cases — as Alexander points out in his post — I find even further interesting blogs and points of view in the comments section of the blogs that appear as sub-links.

Those kinds of value are very difficult to quantify, but they do exist. It’s a chaotic system, in some ways, like biology or the stock market. But on balance, systems like Techmeme help to bring value to the surface, if you are prepared to look for it.

He's totally right. Bob Warfield, Alexander van Elsas, and Matthew Ingram himself are now higher on my radar as a result of the cross-commentary Techmeme unearthed. So yes, we're all a bit smarter as a result.

But my point remains: there are some downsides. And a lot of what I try to do on Radar is to think about the future -- often reasoning by analogy, either from history or from parallel markets. And the connection I was trying to make between what happened recently in the stock market, and what is happening now with Facebook and blogging, is extremely important. Connected systems are smarter -- until they aren't. There are feedback effects in what we pay attention to, so that certain things get valued more and more highly.

Think about it for a moment: Techcrunch is the #1 site on the Techmeme leaderboard, yet most of what it covers will be forgotten not merely in years but in months, and have proven to be completely unimportant: the froth of me-too company creation around ideas and trends that as yet are quite immature and poorly understood. (Michael Arrington himself told me that most of the companies he's covered since starting Techcrunch "have just faded away".) This is the stuff of a bubble.

And that brings me to another important point: precisely because once everyone "knows" the same stuff, there is no competitive advantage to be had from learning it, eventually that over-known area loses its importance. Even in cases where there is no bubble, one real source of competitive advantage is knowing something others don't. Web 2.0 has been about unlocking value by sharing what we know, but that doesn't mean that keeping information private has lost its value as a technique. It's just temporarily out of favor.

And here is one of the really interesting things I'm learning from my study of financial markets as predictors of the future of Web 2.0: after decades of exploring the potential of the efficient market, where everyone has access to the same information, the current rage on Wall Street is for "dark pools of liquidity," where trading patterns are invisible even to the participants.

What does this tell us about the future of Web 2.0? It tells us that the pendulum is going to swing from public data to private. I'm not sure when this will happen, but I'm quite confident in the trend. (If you see signs of areas where this is already happening, be sure to let us know.) There's still a LONG way to go in making all the world's information accessible, but maybe one sign of the maturity of the Web 2.0 trend will be when more companies are started on the premise of keeping information hidden than on building publicly accessible repositories that grow through user contribution.

(Paul Kedrosky and I will be doing a high order bit, Web 2.0 and Wall Street, at the Web 2.0 Summit on Friday, and we've got a whole conference brewing on the subject in New York in February: Money:Tech.)

tags: markets, money:tech, techmeme, web_2.0  | comments: 12   | Sphere It


0 TrackBacks

TrackBack URL for this entry:

Comments: 12

Deepak   [10.16.07 11:55 AM]

Are you saying that we are moving to a world where data is going to get locked up in dark silos again? That would be unfortunate. If one agrees with the assumption (I do), that data by itself has limited value and what one does with the data is where the value lies, then why would a company, limit itself to the data collected within the organization

Meta-data on the other hand does have intrinsic value and it makes complete sense for companies to want to keep that private. Knowledge extraction methods, the information gleaned, etc are the real value drivers and are likely how companies are going to differentiate themselves.

I hope there will be video of your discussion with Paul. That's must see TV

Deva Hazarika   [10.16.07 12:18 PM]


I agree w/ your basic premise and think we'll be seeing more startups like Mint that attempt to add value around very private finance information by aggregating information found across a broad group of users contributing their information. A natural segmentation of contacts and information occurs based on how important people view the information and how open the system is. More on that in this blog post:

JP   [10.16.07 01:58 PM]

I was reading the other day where we are in a bull market, and we are around 60-70% through the typical length of markets in this cycle. This could coincide with a bursting of the bubble in around 2009-2010 (but ad revenue is going to continue powering the internet ad market growth until 2011, so I dont know if that might prolong the bull market. The tail end of 2010 might show companies start to get skiddish over the impending slowing of ad dollars, so... ) or so.

What I see might be happening, is that we get the open social graph going somewhat, in the next 18 months, social data becomes significantly more open (yet decentralized), and a lot more bubble companies spring up, which ends up tipping the cart over (magnified by the slowing of ad revenue growth). This might correlate to Tim's assertion that the pendulum is going to swing back towards private data, as during a bust public perception of tech tends to undervalue new tech (as opposed to overvaluing it right now in a boom). I have a feeling that it might play out such that after we get over the bust of the this current cycle, and open data can recover and operate under the duress of "hard times" which tends to make companies more lean and mean, we will see the true power of "web 3.0" which I think will be powered by the semantic web, and see the cycle start again. A very nice graph of this projection from the w3c:

I think the open social graph has the potential to be the catalyst that drives us into the semantic web age, as it looks to be following the same pattern of the early web, and currently, the semantic web is being employed at the research level:

gnat   [10.16.07 02:08 PM]

I finally realized what has lead Techmeme to be so unsatisfactory over the last three months: it's increasingly hard to find the cool stuff that people are doing. There's plenty on the stuff that companies are doing: Google launches this, Facebook pilots that, AOL closes the other. But it's rare and unusual to see a hacker doing something cool make it onto Techmeme. For this reason Techmeme's value has declined for me as it has turned into Google News's technology section.

Chris Spinchange   [10.16.07 02:31 PM]

I see 2 early signs of this informational "dark pooling"

-The rise of “Invite-only” networks/applications/web services

-'Desktop' OS software robust, secure, and most importantly, easy enough (say, for our parents) to run a personal web/media servers on.

I know "the cloud" is still all the rage, but I think "personal clouds" are where it's at.

isb   [10.16.07 06:30 PM]

Facebook is a prime example of increasing privacy controls in the social networking area.

The Business of Software   [10.16.07 09:46 PM]

I don't think the analogy stands. Efficiency markets hypothesis is about the extent to which prices reflect information.

Stockmarkets are supposed to be "semi-strong form efficient" which means all public information is immediately factored into prices as a result of arbitrage. It is only "inside" information that is not factored into prices.

In Web 2.0, if you're talking about innovators and followers and being able to profit from knowhow, it should be said that knowledge and information alone cannot guarantee success, even if you have a few months head start. I've seen a lot of sub-par products outsell brilliant ones simply because of a bigger marketing warchest.

Whereas in the stockmarket, any person with information that everyone else doesn't have can profit from it within seconds simply by executing a few trades. This is public information of course, because using private information to trade would constitute insider trading. ie, This is why companies have to announce trading halts as soon as price sensitive information is about to come out -- so that it is fair for everyone.

Also it should be said that in Web 2.0, information naturally leaks out. The more (real) users you have, the more likely it will spread.

I think the Web 2.0 and EMH analogy works only from the point of view of venture capitalists. Should money be put here or there? What's in, these days? Etc.

The fact that they are all following the leader suggests that not many of them will end up doing well!

Nick Lothian   [10.16.07 11:43 PM]

Data privacy is absolutely a hot topic right now. In first generation web 2.0 apps very few people tackled this at the API level (FlickR being the exception). Most APIs required you to make you data publicly accessible to everyone (eg, the API, etc).

Now we are beginning to see some attempts to address this - see the some of the work around social network portability, and also the Open Authentication initiative.

John Furrier   [10.17.07 12:06 AM]

These are great posts and not only are they provocative but telling. The big thing to add to your points is collaboration. Collaboration dies when the noice level is too your Techmeme example and Techcrunch reference. The keys to the web besides the data is the collaboration effect. I am focused on this piece. Fun and interactive content is great but the real action is the collaboration area.. real productivity... Keep this thread cuts across all your conferences and authors.

John Furrier   [10.17.07 12:07 AM]

These are great posts and not only are they provocative but telling. The big thing to add to your points is collaboration. Collaboration dies when the noice level is too your Techmeme example and Techcrunch reference. The keys to the web besides the data is the collaboration effect. I am focused on this piece. Fun and interactive content is great but the real action is the collaboration area.. real productivity... Keep this thread cuts across all your conferences and authors.

JamesPage   [10.17.07 03:12 AM]

I have a story that may interest you. The story is about Andrew Black who started Betfair. This tale is about how taking an idea from one industry to another can bring about new ideas.

Andrew and me used to work in the finance industry together. While working together we played around with using Monte Carlo theory to value complex financial instruments. This was in the early 90's. The model would work for as long as nobody else was using the same theory to value the same type of instruments.. As soon as the crowd discovered your area there was no longer any opportunity.

Anyway Andrew left the firm we where working together in, and he came across that the betting industry in the UK only set the odds for soccer matches once a week. He started using Monte Carlo to correctly price the odds of the games. This brought him a reasonable income, but not a great one. He tried selling the tool to all the major players in the UK market, but they did not want or believe in any fancy tool.

There was still some risks in his betting strategy in that he could only bet one way. I.e. for a team to win. So he started betfair, which is now a very successful player in the UK betting industry.

akaplan9   [10.31.07 02:51 PM]

To paraphrase a Hermetic maxim, the snake head eats the end of its opposite side. Every phenomenon is the victim of its own success, as when fire consumes everything available it smolders.

Let's use the phenomenon of the shortcut as another example.

The shortcut is only effective in so far as it is a quicker, shorter route to follow.

As soon as the shortcut becomes longer in time (most likely caused by it’s overuse, traffic, crowds, etc.) people will stop using the shortcut and other routes will be explored (constrained by and affecting the availability of routes of course).

A secret is also only effective as it is kept quiet. Although like a shortcut, if it is never used at all it becomes obsolete. For example, an old path that is never used becomes overgrown with trees and such. Grass and erosion will quickly consume even the most contemporary of roads. Thus, supply/demand ratio is a key function of the shortcut and secret. The key is balance. Without balance every entity risks turning into it’s opposite.

The fire must be tended and poked. The Secret kept within a close circle can leverage power, perhaps even more so due to the power of close circle than the content of a secret itself. Such that even a useless or fabricated fact veiled in secrecy may wield power. Thus, the allure of the whisper and fade away...

Post A Comment:

 (please be patient, comments may take awhile to post)

Remember Me?

Subscribe to this Site

Radar RSS feed