Recent conversations about online documentation

Computer documentation has long been like government funding: nobody wants to contribute to it, but everybody wants it to be there when they need it. In considering the free, online documentation produced by amateurs for open source projects, there are other somewhat whimsical ways it can be compared to government funding:

• It's allocated haphazardly, reflecting the interests of "whoever shows up at the table." (In documentation, each volunteer writes about what he or she finds interesting.)

• The prerequisites needed to make it effective are sometimes missing. (Advanced topics and fussy details may get into the documentation without background material.)

• Efforts are not sufficiently coordinated to achieve the results they merit. (Authors don't check for overlaps, gaps, and consistency.)

Enough strained metaphors--I am now happy to say that attitudes seem to be shifting in online documentation. Major projects are hiring authors or coordinators for amateur contributions. They're undertaking large tutorial works and trying to organize the documentation along more rational lines, usually through wikis or content management systems.

My own goals are to find technologies and practices that promote user-contributed documentation, and to find a role in this space for my work as editor and O'Reilly's work as a publisher and information disseminator. Recently I've discussed the topic and the results of my research at a number of conferences and other public events, and had a chance a chance to exchange email with a number of interesting people working in the area.

Motivation through a reputation system

A survey answered by 350 contributors to online documentation showed that building reputation could be a useful motivation for getting people to contribute, although reputation interests them less than community-building. As I point out in the article, enhancing chances for building reputation may help to attract the people who tend to provide high-quality documentation: trainers, consultants, and others trying to fashion a career around a technology.

I was thus happy to be introduced by open source activist William Hurley to a simple, flexible reputation system for a mailing list or forum. His company, BMC Software, has implemented this reputation system, which is a part of the Jive Forums product, on their developer network. As described, the system lets participants label their postings as "questions," and then rate each answer as "helpful" and "correct." The people who earn the most points through helpful and correct answers are posted on the site, thus furnishing them with a reputation on which they can build a business, while helping new members of the forums learn whose postings are particularly worth reading.

Streamlining contributions

I had several discussions with Adam Hyde of FLOSS Manuals, which provides tools to facilitate the creation of documentation for free software. Hyde has built a formatting layer on top of the popular TWiki software so that documents can be presented as more attractive web pages.

FLOSS Manuals has also recently started printing and selling books; the first one (on the audio editing utility Audacity) is available at their\ own site or Lulu. Whereas most publishers (including O'Reilly) struggle to automate the conversion of documents between online and print formats--a process that never goes smoothly except for very simply laid-out documents--Adam just bytes the bullet and pays someone to manually put the text into InDesign. The process takes a couple days per book. They're moving to Scribus for print-ready layout, and plan to develop plug-ins that make conversion from wiki format to Scribus format more automatic.

The provision of different formats for the same documents leads to a two-stage system. FLOSS manuals have a development version that can be viewed in the standard TWiki format, and a stable version that gets put up in the attractive format and printed.

FLOSS Manuals also offers a facility for mixing sections from different documents. Their "live manual" function lets people serve a remixed manual on a new website by just pasting in five lines of HTML.

Who writes, and why?

I exchanged some very interesting mail with Lance Smith, who believes that people who write documentation have an artistic bent and are chronically unsatisfied with the sloppiness that accompanies most of the work they see around them. The urge to intervene and tell people how to use their computers is aesthetic. The kind of person who contributes to mailing lists and web pages just finds it demoralizing to see bad practices carried on by generation after generation of new learners.

While this is a hard hypothesis to test, it fits nicely with the skills I see as necessary for writing good documentation: a sense of when a passage looks and sounds right, combined with the determination to revise and rework the passage until it reaches the quality that the author can live with.