An unusual Distributed
Distributed Work Meetup was held last night in four different
cities simultaneously, arranged through many hours of hard work by Lukas
Biewald and his colleagues at distributed work provider CrowdFlower.
With all the sharing of experiences and the on-the-spot analyses
taking place, I didn’t find an occasion to ask my most pressing
question, so I’ll put it here and ask my readers for comments:
How can you set up crowdsourcing where most people work for free but
some are paid, and present it to participants in a way that makes it
seem fair?
This situation arises all the time, with paid participants such as
application developers and community managers, but there’s a lot of
scary literature about “crowding out” and other dangers. One basic
challenge is choosing what work to reward monetarily. I can think of
several dividing lines, each with potential problems:
-
Pay for professional skills and ask for amateur contributions on a
volunteer basis.The problem with that approach is that so-called amateurs are invading
the turf of professionals all the time, and their deft ability to do
so has been proven over and over at crowdsourcing sites such as InnoCentive for inventors and
SpringLeap or 99 Designs for designers. Still,
most people can understand the need to pay credentialed professionals
such as lawyers and accountants. -
Pay for extraordinary skill and accept more modest contributions on a
volunteer basis.This principle usually reduces to the previous one, because there’s no
bright line dividing the extraordinary from the ordinary. Companies
adopting this strategy could be embarrassed when a volunteer turns in
work whose quality matches the professional hires, and MySQL AB in
particular was known for hiring such volunteers. But if it turns out
that a large number of volunteers have professional skills, the whole
principle comes into doubt. -
Pay for tasks that aren’t fun.
The problem is that it’s amazing what some people consider fun. On the
other hand, at any particular moment when you need some input, you
might be unable to find people who find it fun enough to do it for
you. This principle still holds some water; for instance, I heard
Linus Torvalds say that proprietary software was a reasonable solution
for programming tasks that nobody would want to do for personal
satisfaction. -
Pay for critical tasks that need attention on an ongoing basis.
This can justify paying people to monitor sites for spam and
obscenity, keep computer servers from going down, etc. The problem
with this is that no human being can be on call constantly. If you’re
going to divide a task among multiple people, you’ll find that a
healthy community tends to be more vigilant and responsive than
designated individuals.
I think there are guidelines for mixing pay with volunteer work, and
I’d like to hear (without payment) ideas from the crowd.
Now I’ll talk a bit about the meetup.
Venue and setup
I just have to start with the Boston-area venue. I had come to many
events at the MIT Media Lab and had always entered Building E14 on the
southwest side. The Lab struck me as a musty, slightly undermaintained
littered with odd jetsam and parts of unfinished projects; a place you
could hardly find your way around but that almost dragged creativity
from you into the open. The Lab took up a new building in 2009 but to
my memory the impact is still similar–it’s inherent to the mission
and style of the researchers.
For the first time last night, I came to the building’s northeast
entrance, maintained by the MIT School of Architecture. It is Ariel to
the Media Lab’s Caliban: an airy set of spacious white-walled forums
sparsely occupied by chairs and occasional art displays. In a very
different way, this space also promotes open thoughts and broad
initiatives.
The ambitious agenda called for the four host cities (Boston, New
York, San Francisco, and Seattle) to share speakers over
videoconferencing equipment. Despite extensive preparation, we all had
audio, video, and connectivity problems at the last minute (in fact,
the Boston organizers crowdsourced the appeal for a laptop and I
surrendered mine for the video feed). Finally in Boston we
disconnected and had an old-fashioned presentation/discusser with an
expert speaker.
In regard to the MIT Media Lab and Architecture School, I think it’s
amusing to report that Foursquare didn’t recognize either one when I
asked for my current location. Instead, Foursquare offered a variety
of sites across the river, plus the nearby subway, the bike path, and
a few other oddities.
We were lucky to have Jeff Howe, the
WIRED contributor who invented the term Crowdsourcing and wrote a
popular book
on it. He is currently a Nieman Fellow at Harvard. His talk was wildly
informal (he took an urgent call from a baby sitter in the middle) but
full of interesting observations and audience interactions.
He asked us to promote his current big project with WIRED,
One Book, One Twitter. His goal is to reproduce globally the
literacy projects carried out in many cities (one happens every year
in my town, Arlington, Mass.) where a push to get everyone to read a
book is accompanied by community activities and meetups. Through a
popular vote on WIRED, the book American Gods by
Neil Gaiman was chosen, and
people are tweeting away at #1b1t and related tags.
Discussion
With the sponsorship by CrowdFlower, our evening focused on
crowdsourcing for money. We had a few interesting observations about
the differences between free Wikipedia-style participation and
work-for-pay, but what was most interesting is that basic human
processes like community-building go in both places.
Among Howe’s personal revelations was his encounter with the fear of
crowdsourcing. Everyone panics when they first see what crowdsourcing
is doing to his or her personal profession. For instance, when Howe
talked about the graphic design sites mentioned earlier, professional
designers descended on him in a frenzy. He played the sage, lecturing
them that the current system for outsourcing design excludes lots of
creative young talent, etc.
But even Howe, when approached by an outfit that is trying to
outsource professional writing, felt the sting of competition and
refused to help them. But he offered respect for Helium, which encourages self-chosen
authors to sign up and compete for freelance assignments.
Howe is covering citizen journalism, though, a subject that Dan
Gillmor wrote about in a book that O’Reilly published We the
Media, and that he continues to pursue at his Mediactive site and a new book.
Job protection can also play a role in opposition to crowdsourcing,
because it makes it easier for people around the world to work on
local projects. (Over half the workers on Mechanical Turk now
live in India. Biewald said one can’t trust what workers say on their
profiles; IP addresses reveal the truth.) But this doesn’t seem to
have attracted the attention of the xenophobes who oppose any support
for job creation in other countries, perhaps because it’s hard to get
riled up about “jobs” that have the granularity of a couple seconds.
Crowdsourcing is known to occur, as Howe put it, in “situations of
high social capital,” simply meaning that people care about each other
and want to earn each other’s favor. It’s often reinforced by explicit
rating systems, but even more powerful is the sense of sharing and
knowing that someone else is working alongside you. In a blog
I wrote a couple years ago, I noted that competition site TopCoder maintained a thriving
community among programmers who ultimately were competing with each
other.
Similarly, the successful call center LiveOps provides forums for operators
to talk about their experiences and share tips. This has become not
just a source of crowdsourced help, and not even a way to boost morale
by building community, but an impetus for quality. Operators actually
discipline each other and urge each other to greater heights of
productivity. LiveOps pays its workers more per hour than outsourcing
calls to India normally costs to clients, yet LiveOps is successful
because of its reputation for high quality.
We asked why communities of paid workers tended to reinforce quality
rather than go in the other direction and band together to cheat the
system. I think the answer is obvious: workers know that if they are
successful at cheating, clients will stop using the system and it will
go away, along with their source of income.
Biewald also explained that CrowdFlower has found it fairly easy to
catch and chase away cheaters. It seeds its jobs with simple questions
to which it knows the right answers, and warns the worker right away
if the questions are answered incorrectly. After a couple alerts, the
bad worker usually drops out.
We had a brief discussion afterward about the potential dark side of
crowdsourcing, which law professor Jonathan Zittrain covered in a talk
called Minds
for Sale. One of Zittrain’s complaints is that malicious actors
can disguise their evil goals behind seemingly innocuous tasks farmed
out to hundreds of unknowing volunteers. But someone who used to work
for Amazon.com’s Mechanical Turk said people are both smarter and more
ethical than they get credit for, and that participants on that
service quickly noted any task that looked unsavory and warned each
other away.
As the name Mechanical Turk (which of course had a completely
unrelated historical origin) suggests, many tasks parceled out by
crowdsourcing firms are fairly mechanical ones that we just haven’t
figured out how to fully mechanize yet: transcribing spoken words,
recognizing photos, etc. Biewald said that his firm still has a big
job persuading potential clients that they can trust key parts of the
company supply chain to anonymous, self-chosen workers. I think it may
be easier when the company realizes that a task is truly mechanical
and that they keep full control over the design of the project. But
crowdsourcing is moving up in the world fast; not only production but
control and choice are moving into the crowd.
Howe highlighted Fox News, which runs a UReport site for
volunteers. The stories on Fox News’ web site, according to Howe, are
not only written by volunteers but chosen through volunteer ratings,
somewhat in Slashdot style.
Musing on the sociological and economic implications of crowdsourcing,
as we did last night, can be exciting. Even though Mechanical Turk
doesn’t seem to be profitable, its clients capture many cost savings,
and other crowdsourcing firms have made headway in particular
fields. Howe hails crowdsourcing as the first form of production that
really reflects the strengths of the Internet, instead of trying to
“force old industrial-age crap” into an online format. But beyond the
philosophical rhetoric, crowdsourcing is an area where a lot of
companies are making money.