Why Wikia will change search

This morning Jimbo Wales’ Wikia launched their search effort, Wikia Search. Wales & co. have been getting a lot of heat for this launch, most notably from Mike Arrington at Techcrunch who calls Wikia Search “an inexcusable waste of time” and “a complete letdown”.

I have to be honest that my first reaction to Wikia Search was lukewarm, and I fully support Arrington in his assessment that Wikia Search is no viable replacement for Google, Yahoo!, Ask.com, or any of the other established search players. But Wales takes out the air of Arrington’s choler by commenting that “it’s a project to *build* a search engine, not a search engine”, recalling that

When I launched Wikipedia, I wrote at the top of the first page “Wikipedia, the free encyclopedia”. On that day, anyone reviewing it would have laughed. What’s this? There’s nothing here! This is not an encyclopedia, it is an empty website with some funny editing syntax!

Were Wales goes wrong is that Wikia has released a search engine on a limited index with no opportunity for users to contribute. In terms of Wikipedia, it would be the equivalent of launching an out-of-copyright dictionary but giving users no ability to edit the erroneous articles or adding new. Wikia should not have launched without, and that is a mistake.

Yet Wikia will change search. They may very well be run overend by Google, Yahoo!, or Facebook (who eventually will turn towards search like their fraternal predecessor AOL) in the process of doing so, but they’ll change a few rules of the game.

First, values
AOL’s search log blunder was notorious, and generally people are slowly starting to question the potential privacy invasion from our online data trails.
A Wikia employee told me today that people were already asking what the most popular search terms were. He said there was no way of finding out as no logs are kept.

Second, they’ll open search. Really.
Wikia claims that they’ll make their index freely available. If they haven’t already, we’ll almost certainly see Wikia’s index in Amazon’s S3 (Amazon is a major investor), making it effortless to create custom search engines using a couple EC2 instances. Think vertical search engines with custom algorithms for anything from gaming to Japanese manga cartoons. Talk about giving Google’s 16K employee brute-force machine competition.

Go play with Wikia Search. Then come back here and read the above again. Tell me what you see: a bluff or a ripple of change?

Disclosure: Fellow Radar blogger Artur Bergman is Director of Engineering at Wikia, and I’m posting this just a few hundred meters down the road from their Poznan (Poland) office.

  • http://mndoci.com Deepak

    I am not so sure. Wales himself has been talking up Wikia, and what was released today was overwhelming. It it had been presented as a open project to build a search engine, it was a different matter. Fact remains that Google was good right from the start.

    Does Wikia have a place. I am convinced it does, but it will never ever challenge Google in its current incarnation. The scale will just not be there, plus at that scale, the algorithms are going to win anyway. To me, projects like Lucene and Hadoop are where the change will come, cause search is all about infrastructure and quality, and those are two areas where at least at a general level, its going to be hard to top Google

  • http://paul.annesley.cc/ Paul Annesley

    Just pointing out a small markup error – your second link, “Wikia Search”, is missing the equal sign between href and the double quotes.

  • http://web.doodlekit.com/ Internet

    The more spectrum of quality information resources for researchers, the better.

    Not everything can be a panacea. Not everything can have an instant ‘WOW’ factor. Not everything should have a pre release buzz.

    This is but one more piece of the puzzle in the art of information retrieval.

  • http://forum.kahuki.com Manuel

    I would love to see Wiki Search to succeed. I tried the alpha which looks lousy, but I think that project will get better soon!

  • http://sugarpharm.com Carla

    “a limited index with no opportunity for users to contribute.”

    I don’t understand the rationale for releasing the product in this state. it baffles me

  • http://www.bgoncalves.com Bruno Goncalves

    If wikia is meant as the “wikipedia of search”, where is the tool to let me play with the algorithms? Where can I download the index and a sample implementation? Basically, where is the source code and data? And even if they do eventually release all of this, how can it be “for profit” (.com and not .org) and ask people to donate their time and skills for free?

  • http://www.ajeetkhurana.com Ajeet Khurana

    I would say that I am underwhelmed by what currently exists on the site. Despite that, I agree with “Yet Wikia will change search.” This is because I see it as the first of many such efforts. And that is a good thing for searchers. An open index is a refreshing approach.

  • http://www.cpeterso.com/ Chris Peterson

    Wikia is a joke. Try searching for George Bush or even Wikipedia:

    http://re.search.wikia.com/search#george%20bush

    http://re.search.wikia.com/search#wikipedia

    Search for “wikipedia” and none of the first 100 search results include any link to en.wikipedia.org?! I gave up looking any beyond 100 results.

  • http://www.knetsch-partner.de Brandau

    My first impression wasn´t that good either, but it has enough media attention to get big. We will see if a lot of people like the idea and will invest some time to help it grow.

  • did

    One data point about index size: i searched for ‘”norbert wiener” krishnamurti’: wikia found 0 matches and google 323 matches, many if not most relevant.
    It’s just a data point, not a conclusion.

  • http://www.fridaynite.org/ Black Friday

    I dont think, that this new search engine will be a burner. The design is nice, but the search results are very very bad. The competition is Google.

  • http://www.chiletour.org ChileGuy

    Well, the search results are still bad but I really like the idea and when the human aspect will play his influence search results will get better.

    One guy finds a nice project and just a few hours later the wohle community will know which should push up the project in the SERPs. Google is too much about trust, link-age etc…

  • free sms

    Interesting article I wait from what the project is …

  • http://www.mallorcaexperten.de Eddy

    Well I don´t like the way google creates its search results. By creating my own website I learned that it is not so much the content that counts. There are too many ways to manipulate this search engine. Although it claims to constantly get better, I don´t believe that there will be an appropriate way to do it the best way.
    That´s why I don´t really believe that this new approach is going to get better. Everything is just too much about making money in the web.

  • http://uri-geller.speyburn.de the next uri geller

    I hope wikia could be an alternative to google, cause everyone is optimizing on one search engine. that’s not good. if google kicks you out of the index, your site will never be found…

  • http://www.tvfacts.de Manuel

    At the first view Wikia looks quite reasonable, but the results are very bad. There is still much to be done. I stick with Google!

  • http://makarevitch.org/rant/google_strategy.html Nat

    Google may benefit from tracking activity on Wikipedia, and Wikipedia needs hosting. Seems a good fit?!

    Nope.

    Wales announced Wikia Search, which will in fact be a competitor for Google.

    Then Google announced Knol, a competitor for Wikipedia.

  • http://www.mobilefacts.de/shop/surf-stick/mobook-surf-stick/ Mobook

    i think the results from wiki search are really bad.
    thanks for the interesting article.

  • http://freeconnect.de/flatrate-tarife-surf-stick/ Dateflatrate

    Gotta admit to Mobook. Great Article.
    THX