"reviews" entries

Podcast: ratings, rankings, and the advantage of being born lucky

A conversation with Sean Taylor, Hilary Mason, and John Myles White about how ratings affect our thinking

Outcomes following random exogenous upvotes and downvotes on message board posts. Image via Sean Taylor.

Researchers randomly upvoted some posts and downvoted others on a popular message board. The upvoted posts became substantially more popular over the long run. Image via Sean Taylor.

Is popularity just a matter of simple luck–of some early advantage compounded by human preference for things that are already popular? A paper published today in Science offers some insight into the way that popularity emerges in online ratings. Lev Muchnik, Sinan Aral, and Sean Taylor were able to set up a randomized experiment on a popular Reddit-like message board in which they gave some posts a one-point upvote on publication and others a one-point downvote. Posts that were “born lucky” ended up with 25% higher scores on average than those without modification.

In our latest podcast, Renee DiResta and I are joined by Sean Taylor, Hilary Mason and John Myles White to talk about Sean’s findings and about ratings, rankings and reviews in general. Bits and pieces that come up in the podcast:


Subscribe to the O’Reilly Radar Podcast through iTunesSoundCloud, or directly through our podcast’s RSS feed.

The hidden language and "wonderful experience" of product reviews

Panagiotis Ipeirotis on the phrases and formatting of effective product reviews.

How much is an Amazon review — good or bad — worth? Computer scientist and NYU professor Panagiotis Ipeirotis analyzed the text in thousands of Amazon reviews to find out.

Four short links: 9 June 2010

Four short links: 9 June 2010

DIY Games, Code Review, Open Oil Data, Crowd Sourced Science Success

  1. Game Dev 101 lessons with WarioWare DIYNintendo’s long-running and (at its debut) groundbreaking WarioWare franchise has always been predicated on discrete games played for 5-10 seconds at a time, in rapid succession, and it’s precisely that stripped-bare approach that makes it an ideal launchpad for re-wiring the way aspiring designers think about what makes games fun. With its own bespoke image and music editor, a graphical scripting language not altogether (so I’m told) that different from the tools available in popular PC package GameMaker, and — crucially, if a bit over-long for those more familiar with game dev proper — hours worth of mandatory tutorials that leisurely stroll you through Your First Animated Sprite or Your First Logic Gate. (via BoingBoing)
  2. What Should Mozilla Look For In an Automated Review SystemMondrian’s review comment system really seemed to encourage a style where there was a one-way flow of instructions from the reviewer to the reviewee: “Do this. Do this. Do this.” and the reviewee replies with “Done. Done. Done.” Sometimes this is appropriate, but oftentimes it isn’t. (Mondrian is Google’s internal tool for this) (via Marc Hedlund)
  3. DOE Releases BP Oil Spill DataAs part of the Obama Administration’s ongoing commitment to transparency surrounding the response to the BP oil spill, the Department of Energy is providing online access to schematics, pressure tests, diagnostic results and other data about the malfunctioning blowout preventer. (via EllnMllr on Twitter)
  4. The Rise of Crowd Science — fascinating account of the life work of Alex Szalay, who has turned astronomy into a data-sharing discipline embracing crowdsourcing. I loved this story: More than 270,000 people have signed up to classify galaxies so far [on Galaxy Zoo]. One of them is Hanny van Arkel, a schoolteacher in Holland, who found out about the site after her favorite musician, Brian May, guitarist for the rock group Queen, wrote about it on his blog. After clicking around on Galaxy Zoo for a while one summer, she landed on an image with what she describes as a “very bright blue spot” on it. “I read the tutorial and there was nothing about a blue spot,” she says, so she posted a note to the site’s forums. “I was just really wondering, What is this?” Her curiosity paid off. Scientists now believe the spot is a highly unusual gas cloud that could help explain the life cycle of quasars. The Hubble telescope was recently pointed at the object, now nicknamed “Hanny’s Voorwerp,” the Dutch word for object. Astronomers have published papers about the discovery, listing Ms. van Arkel as a co-author. “Don’t ask me to explain them to you, but I am a co-author of them,” she says with a laugh. Szalay will be at Science Foo Camp this year, and I can’t wait to meet him. (via Penny Carnaby)

Three Paradoxes of the Internet Age – Part One

In the circles that I travel the Internet is often breathlessly embraced as the herald of all things good; the bringer of increased choice, personal empowerment, social harmony… and the list goes on. And yet, as with any powerful technology, the truth of its consequences eludes such a singular and happy narrative. More access to information doesn’t bring people together, often it isolates us.

Maintaining a Web Community is as Hard as Building One

Notoriety tools provide one way to keep a Web community vibrant and engaged.