When was the last time you mined your site's search data?

Lou Rosenfeld on the benefits of parsing and refining site search.

Site search boxShow of hands: If you run a website, how often do you consult your own search logs? I’m not talking about search engine optimization or inbound lead generation, here. I’m referring to the data that’s generated by your site’s own engine.


Lou Rosenfeld (@louisrosenfeld), author of “Search Analytics for Your Site,” believes site search deserves a place in every site owner’s toolset. These engines reveal the content that’s working, the queries that go unanswered, and the things audiences need most — and all that information is tucked into datasets you already own.

Rosenfeld expands on these points in the following interview. He’ll also explore many of these same topics during his upcoming session at Web 2.0 Expo New York.

Is site search often overlooked by site owners?

LouRosenfeld.jpgLou Rosenfeld: It’s not necessarily overlooked by users, but definitely by site owners who assume it’s a simple application that gets set up and left alone. But the search engine is only one piece of a much larger puzzle that includes the design of the search interface and the results themselves, as well as content and tagging. So search requires ongoing testing and tuning to ensure that it will actually work.

Is site search data an untapped resource?

Lou Rosenfeld: If your site has substantial search traffic, then you already own scads of useful data that describe what your users want from your site — in their own words. Why would you not want to take advantage of this resource?

In my book, I cover different ways to improve many aspects of a site — not just search performance, but content quality, navigation, metadata, and overall performance. There’s certainly enough there to keep a full-timer occupied. But if you don’t have such resources in your hands, at least spend an hour per month reviewing two common reports:

  1. Your site’s most common queries — to see what’s of greatest interest to your users and how those interests change over time.
  2. Your site’s most commonly failing queries — either queries that induce immediate site exits or those that retrieve zero results.

Dedicating just a little bit of time with these two simple reports will go a long way toward identifying and diagnosing problems with your site, many of which will be surprisingly simple to fix.

What are the best tools for analyzing site search data?

Lou Rosenfeld: Most common web analytics tools are lacking when it comes to helping analyze site search data. While you can get basic reports out of the big name tools, I always suggest downloading what you can into your favorite spreadsheet or database and playing with the data yourself. Your users, your site, and your organization are different than everyone else’s, so you’ll really want to get beyond your web analytics app’s generic reports and interrogate the data as best suits your needs.

Search Analytics for Your Site — Any organization that has a searchable website is sitting on top of a valuable and under-exploited dataset: your search logs. This book shows how search data can improve your website and help you better serve your audience. See this title and others from Rosenfeld Media.

How do you act on the conclusions you reach from the data? Create more content for popular queries? Something else?

Lou Rosenfeld: Missing content is only one problem. You might find that the content actually does exist, but it’s poorly structured, tagged, or titled. Or the content isn’t prominent enough. Or your individual search engine results are poorly designed or unhelpfully sorted. Or that your deep contextual navigation is broken. Or that your search box isn’t wide enough. These are just a few of the conclusions you might reach, but remember: analytics data can only tell you what might be wrong. You still need to use qualitative research methods to establish why.

Is there a particular site-specific search engine you recommend?

Lou Rosenfeld: Yes. The one you’re using right now. Crappy search performance has less to do with your search engine selection process and more to do with not bothering to learn about users’ needs and configuring your search engine accordingly. So before your CIO spends six figures on a new search engine license, at least make sure you’re getting the most out of what you already have.

What’s the difference between site search analytics (SSA) and search engine optimization (SEO)? Is SSA really just SEO for your own site?

Lou Rosenfeld: Pretty much. Large organizations have sites so jam-packed with content that searching them is as overwhelming as searching the web. In such environments, a decent search system is the user’s only chance of finding what they’re looking for. So site owners need to play God for their sites just like Larry Page and Sergey Brin have for the web — by ensuring reasonable search performance.

Does SSA reveal user intent better than other forms of analytics?

Lou Rosenfeld: I think so, as the data is far more semantically rich. While you might learn something about users’ information needs by analyzing their navigational paths, you’d be guessing far less if you studied what they’d actually searched for. Again, site search data is the best example of users telling us what they want in their own words. Site search analytics is a great tool for closing this feedback loop. Without it, the dialog between our users and ourselves — via our sites — is broken.

This interview was edited and condensed.

Strata Conference New York 2011, being held Sept. 22-23, covers the latest and best tools and technologies for data science — from gathering, cleaning, analyzing, and storing data to communicating data intelligence effectively. Save 30% on registration with the code STN11RAD.


tags: , , , , ,