Podcast: ratings, rankings, and the advantage of being born lucky

A conversation with Sean Taylor, Hilary Mason, and John Myles White about how ratings affect our thinking

Outcomes following random exogenous upvotes and downvotes on message board posts. Image via Sean Taylor.

Researchers randomly upvoted some posts and downvoted others on a popular message board. The upvoted posts became substantially more popular over the long run. Image via Sean Taylor.

Is popularity just a matter of simple luck–of some early advantage compounded by human preference for things that are already popular? A paper published today in Science offers some insight into the way that popularity emerges in online ratings. Lev Muchnik, Sinan Aral, and Sean Taylor were able to set up a randomized experiment on a popular Reddit-like message board in which they gave some posts a one-point upvote on publication and others a one-point downvote. Posts that were “born lucky” ended up with 25% higher scores on average than those without modification.

In our latest podcast, Renee DiResta and I are joined by Sean Taylor, Hilary Mason and John Myles White to talk about Sean’s findings and about ratings, rankings and reviews in general. Bits and pieces that come up in the podcast:

Subscribe to the O’Reilly Radar Podcast through iTunesSoundCloud, or directly through our podcast’s RSS feed.

tags: , , ,

Get the O’Reilly Data Newsletter

Stay informed. Receive weekly insight from industry insiders.

  • Ivan

    How Amazon reviews are both satirical and uninformed? There are pretty good reviews there and usually it is easy to discern poor from good (not to mention that the poor reviews often get pwnd in the comments bellow them). If a person can not discern good from bad arguments it is his own problem. That kind of person end up believing that the world was created 5000 years ago or that the President is being advised by aliens, living in the basement of the White House, that came from their base on the dark side of the Moon.

    • Jon Bruner

      I was referring to two different types of entertaining Amazon reviews through the separate links under that bullet point: some are entertaining because they’re deliberately satirical; others are inadvertently entertaining because the reviewer is evaluating some unconventional aspect of the product. On the whole I think Amazon reviews are extremely helpful (though, as we discuss in this podcast, they can introduce some interesting biases).