Search Startups Are Dead, Long Live Search Startups

Echoing Debra Chrapaty’s comment that “in the future, being on someone’s platform will mean being hosted on their infrastructure“, Bill Burnham has an interesting entry on why search is dead as a category for VCs:

A couple of months ago I had the pleasure of moderating a panel at TIECon on the Search Industry. Peter Norvig, Google’s Director of Research, made one comment in particular that stood out in my mind at the time. In response to a question about the prospects for the myriad of search start-ups looking for funding Peter basically said, and I am paraphrasing somewhat, that search start-ups, in the vein of Google, Yahoo Ask, etc. are dead. Not because search isn’t a great place to be or because they can’t create innovative technologies, but because the investment required to build and operate an Internet-scale, high performance crawling, indexing, and query serving farm were now so great that only the largest Internet companies had a chance of competing….

 

So where does this leave search start-ups? For many of them it will undoubtedly leave them in the Web 2.0 Deadpool, but some of them may thrive if they adapt to the new reality. This new reality is that search innovation will increasingly be about applications and not about the core infrastructure. In fact, there’s a good chance that much of the core infrastructure will be available as a service to search-based applications. Amazon is pioneering this “search as a service” with its opening up of the Alexa crawling and indexing APIs.

I came to that same conclusion several years ago, when I was on the board of Nutch, the open source search engine based on Lucene. Great software, but without the funding to take the operations to scale, it could never become more than a research platform.

In my talks on Web 2.0, I always end with the point that “a platform beats an application every time.” We’re entering the platform phase of Web 2.0, in which first generation applications are going to turn into platforms, followed by a stage in which the leaders use that platform strength to outperform their application rivals, eventually closing them out of the market. And that platform is not enforced by control over proprietary APIs, as it was in the Windows era, but by the operational infrastructure, and perhaps even more importantly, by the massive databases (with network effects creating increasing returns for the database leaders) that are at the heart of Web 2.0 platforms.

But as Bill notes, this doesn’t mean the end of the application category. It just means that developers need to move up the stack, adding value on top of the new platform, rather than competing to become it.

tags: