May 24

Nat Torkington

Nat Torkington

Rewarding Users for Contributing Data

Four months ago I spoke to a real estate guy with a great mobile platform who wanted to gather user data about places (similar to Platial) and resell the data via a mobile app. He was planning to pay the people who contributed geographic information based on who chose to buy their map layer. I thought that wasn't a winner: he was simply paying people a little bit so he could take a lot. Since then I've been pondering incentives and rewards. Marc's blank faces post (give your users reasons to contribute their data) rang a bell, so I wrote up my conclusions on the subject. Here they are.

Users should contribute data for a reason other than "Nat's business model is predicated on collecting user-generated data". The best reasons give a reward that's related to the data. BitTorrent gives you fast downloads if you are in turn offering fast uploads; you're rewarded in bandwidth for offering bandwidth.

Community is a good reason: my theory is that Amazon reviewers and list makers are passionate book-lovers contributing data because it builds their place in this community. I'm not particularly enthusiastic about Yahoo! Local Reviews because it's not a site that has a community and so I have trouble finding incentives, rewards, or reasons for contributing reviews other than you feel particularly passionate about a particular restaurant. End result: I predict Yahoo! will get only extreme reviews from a minority of the population. A quick check of restaurants in my area show extremes in ratings, despite there being a lot of mediocre restaurants around.

Self-interest is a good motivator: eBay feedback began as a way to weed out the fraudsters making life miserable for marketplace participants, and by contributing information on whether they were good traders you were helping to promote good trading. Now the feedback's more of a ritual ("AAAA+++++!!!!! BEst sellerr evaH! !!!").

The worst kind of reward is money, I think. There was talk of MSN search paying people to use them. That's completely disconnecting the reward from the behaviour. The best reward for searching is relevant results. And guess what? The site with the most relevant results is also the top search site.

Money, and any other type of reward that substitutes for money such as vouchers or access to powerful site features (e.g., features that would otherwise be subscription-limited) encourages deceit and gaming. If you pay me to search, I'll search when I don't need to (just to make money), advertisers will be getting impressions that aren't useful to them, and it'll drive the price of impressions down. Because you can't reward people for the quality of the data they contribute, you have to settle for the act of contributing something. And once the reward is worth money, you'll get people contributing crap just to get the rewards. Your reward system is now paying people to piss in your data pool. That hasn't happened yet for Chris Sells's offer to share royalties in return for Amazon reviews, but that's only because of the scale he's operating at. Needless to say, the rules are also different for intranet applications rather than public Internet applications.

Leaderboards are similarly problematic. Competitions as motivation for contributing data are workable, but only if you can validate the data cheaply. Otherwise you'll have people submitting bogus data just to get a higher position on the leaderboard. See any Orkut profile with >300 "friends" for an example of this in action.

So if you're asking people to contribute movie reviews then you should have a site that's serving a movie-loving community (people with passionate opinions who will want to express them), or it should improve their site experience (you can now recommend other movies they'd like and other users whose tastes they share). You shouldn't send them movie vouchers, you shouldn't give the review contributors access to more detailed information about the movies, and you absolutely shouldn't send them a quarterly check for 25c/movie reviewed.

This is related to, but different from, Napster-style contribute-by-default behaviour. You reward when you have to elicit data that the people don't already have. Napster shared the files people had already downloaded. If you had to get people to rip their CDs, you'd turn to rewards and reasons for contributing data.

tags: web 2.0  | comments: 6   | Sphere It

Previous  |  Next

0 TrackBacks

TrackBack URL for this entry:

Comments: 6

  Tom Carden [05.24.06 04:30 AM]

We talk a lot about this at OpenStreetMap.

In the case of mapping, I think the reward is the satisfaction of creating something that people can get use out of straight away. There's an incentive to go out and survey your local area, because OpenStreetMap then gives you the tools to solidify your experience and create a nice map - a good incentive for those who like maps, and everyone likes maps, right?

As the tool chain over at OpenStreetMap matures, I think that the delay between effort and reward will narrow to the point of instant gratification. Creating a map of the whole world will be a by product of the fact that for lots of people drawing a map of an area you intimately know is actually fun!

  Chris Campbell [05.24.06 05:39 AM]

Community is the most powerful motivator to participate and it works especially well when you don't even realize it it happening. With Flickr the comments and people adding photos as favorites encourage the uploading of more photos and commenting yourself. On 43 Things, 43 Places, and All Consuming the Robot Co-op has designed a workflow that gradually eases you into the community. You can cheer entries or goals that you like. When I receive a cheer I'll see what the cheerer has consumed or what they have as goals and I'll usually cheer them as well. But the very clever part of the design is that when you cheer someone a comment box opens. You don't have to fill it in and leave a comment, but it pushes you towards making a comment. It's a great way to build community in a casual and optional way.

  Pete Cashmore [05.24.06 07:49 AM]

I think you're right that community is a much better motivator than money, and that financial rewards can lead to abuse. However, I think there are instances where paying users *is* the best route - when the content is being sold directly, for instance (stock photos, Cafepress etc). If a sale has to be made before revenue is shared, there's no gaming. So Flickr shouldn't pay me for contributing my photos, but the stock photo sites should - so long as they manage to sell them.

  Alex Tolley [05.24.06 08:50 AM]

This issue is well addressed by Yochai Benkler's book, The Wealth of Networks: How Social Production Transforms Markets and Freedom.

  Paul Dixon [05.24.06 10:27 AM]

Geograph is a site aiming to collect a geographic photo for every square kilometre of the British Isles, and I think we've use a leaderboard to good effect, but it's definitely not the whole story

When we started the site, we figured we'd be doing well if we could get 50 images contributed a day, but at the moment it's 500 a day.

The reason for this is that I think we've created an environment where people don't think there is a secret plan to rip them off. This lets people feel good about contributing, turning the green bits of a map red, and helping to build a really useful free photographic archive.

We call it the "warm glow" - if you can make contributing its own reward, just try and *stop* people contributing!

  Vishi Gondi [05.26.06 12:20 AM]

Netflix: Rate my movies to get better recomendations.

Less gaming of results.

Post A Comment:

 (please be patient, comments may take awhile to post)

Type the characters you see in the picture above.