If it’s true that books in the iBookstore can include video, for example, what kind of video? Adobe promotes Flash video for use in EPUB, but of course there’s no Flash in the whole iPad/iPhone ecosystem. So yes, EPUB is a standard, but the kind of content that would really showcase a tablet versus an E-Ink reader — like video — seems to have already fragmented before the device even hits the marketplace. That kind of fragmentation is going to deter publishers from doing much more than what they’re already doing: making text-only EPUB for wide distribution, and doing the occasional book-as-app experiment (at a premium price point).

We’re already seeing the tension between Apple’s own interests and their control of the App Store with the announcement that many apps (notably the Kindle) won’t be available in the App Store on release of the iPad because they haven’t been able to test on a real device. And then … how long until they release? That’s totally up to the Apple approval process.

In our development of an HTML5-based web app, we’ve found that hardware profiles are key to making web apps really compete. Ibis Reader isn’t nearly as compelling on an older iPod Touch as it is on an iPhone 3GS. By providing what is apparently a really sexy tablet that’s capable of running video and games fluidly, Apple also widens the playing field to let web apps shine too. As the hardware increases in capacity, web apps become increasingly attractive as an end product, not just as a way for web developers to play in the app space.

Thanks to everyone who attended and especially to those who asked so many excellent questions.

Upcoming TOC Webcasts

Not so fast. Experienced computer crackers will find weaknesses in any encryption scheme, but regular folks with basic computer
skills can exploit the one weakness found in all DRM’ed media: the analog hole.

What is the Analog Hole?

The “analog hole” reflects a basic principle of physics: before humans
can consume any digital media, the ones and zeroes that computers
understand must be converted into an analog format
that our senses can perceive. For music, it’s sound waves; for
video and for digital books, it’s patterns of light.

If you’ve ever visited a major metropolitan city you’ve probably
seen the analog hole in action: street vendors selling pirated
copies of popular movies, often months before they’re officially
released on DVD. Most of these are “cam” films, shot in real
movie theaters using camcorders. Even without access to a
physical copy of the film, pirates are able to capture its analog
expression: the sound and pictures as perceived by a theater-goer.

In music, the analog hole is often used to get around software
preventing digital copying. A user simply plays the the desired song
on their computer using the legal DRM-enabled software, and records
the audio coming out of their computer. Now they have a copy of
the sound recording, which can be re-imported into the computer
and digitally-encoded, with the original DRM stripped out. (A
similar principle is at work when DRM systems go defunct and
users are told to pirate
their own music, although the industry uses the euphemism “making a backup.”)

Film and music companies are painfully aware of the analog hole
and have taken steps to close it, either by
monitoring patron behavior (as in movie theaters) or by
petitioning to legally limit the recording features of consumer electronics.

Because reading is a visual experience,
there is the possibility of an analog hole exploit. Unlike
with camcorder copies or re-burned MP3s, there is a potential for no
loss in quality. And with a little ingenuity, the process can
be completely automatic.

One example: Ebooks and Optical Character Recognition (OCR)

Here’s a sample digital book as displayed in Adobe Digital Editions. (This
book is public domain and isn’t technically covered by DRM, but the principle
is exactly the same.)

I hid as much of the Digital Editions menus as I could and took a screenshot of this first page of Pride and Prejudice.

Next I downloaded some free optical character recognition (OCR) software. OCR
programs can “read” images and output the words in them as plain text. It’s a normal
part of digitization projects, in which archival printed material is first scanned
and its text is automatically extracted. At the consumer level, OCR software is often bundled with commercial scanners and fax machines.

I took my screenshot and fed it to the OCR software. Here’s what I got without any special fine-tuning or spell-checking. Note
that all typos are from the OCR software.

Chapter 1

It is a truth universally acknowledged, that a single man in
possession ofa large fortune must be in want of a wife, However little
known the feelings or views of such a man may be on his first entering
a neighbourhood, this truth is so well fixed in the minds of the
surrounding families, that he is considered the rightful property of
someone or other of their daughters.
“My dear Mr. Bennet,” said his lady to him one day, “have you heard
that Netherfield Park is let at last?”
Mr. Bennet replied that he had not.

…and on through the entire first page. This output was in HTML, ready to be
posted to the Web for anyone to read.

The OCR isn’t 100 percent accurate, of course, but neither are the widely-available
pirated ebooks created by laborious scanning of physical books, page
after page. I was also using free software that requires careful fine-tuning to get working optimally; commercial OCR software is much better, especially when combined with
spell-checking.

It wouldn’t be difficult to automate the process of advancing one page in Digital Editions, taking a screenshot, and passing that on to my OCR software. Once the workflow was in place, I could strip hundreds or thousands of books of their DRM in a matter of minutes.

Another Possibility: Speech Recognition

My local library is kind enough to allow me to check out digital audiobooks. Naturally
they’re also secured with DRM (so much so that I can’t actually play them, as they require Windows Media Player and I have only Mac and Linux computers). But assuming I could play them, I’d have available to me a nice stream of professionally-produced audio.

You’re using speech recognition software every time you call a customer service line and an
automated voice prompts you to speak your credit card number. If that’s happened
to you, you also know that speech recognition isn’t 100 percent accurate yet, but under
certain conditions it can be quite good. Automatic speech-to-text transcription
isn’t nearly as far along as optical character recognition, but it’s another
analog hole exploit that will eventually become trivial to perform.

Does This Mean Publishers Shouldn’t Produce Ebooks or Audiobooks?

No! What I hope to convey is that DRM is not a true safeguard against ebook piracy.
(It is, however, a known deterrent to ebook adoption.) I’ve heard a lot of passing the buck on DRM: publishers claim authors want it, booksellers claim publishers
insist on it. These days it’s hard to find someone to publicly state that they’re
actually for it.

I think of DRM like this: years ago my apartment was broken into and I
called a locksmith to replace the door. My landlord had authorized me to
get “the best lock possible,” and the locksmith obliged with a four-foot steel
bolt. It was almost too heavy to turn but made a very satisfying noise when it snapped shut.

I asked the locksmith, “Is this really unbreakable?”

“The lock is, sure.” He slapped the door frame. “But this is made out of
wood. If I really wanted to get in I’d just kick out the door. That’s why I’m honest
about what I sell.” When I looked
puzzled he handed me his business card.
It contained his name, phone number, and company slogan: “A feeling of security.”

Authors and publishers should be compensated for their talent and their hard work,
and the desire for DRM is understandable. Book lovers, too, want their
favorite authors to succeed. But digital books are a form of technology as
much as they are literature, and technologies that are successful adapt to people’s
needs. Is just a “feeling” of security worth the ire of good customers who
want to read their books wherever and however they like?

Related Stories:

• Tim O’Reilly: “Piracy is Progressive Taxation …”
• “Spore” Backlash: Is DRM Officially Bad for Business?
• Pirates Convince Game Developer to Drop DRM
• Charting the Pitfalls of DRM
• Putting Ebook Piracy into Perspective

Naturally I was interested in applying what I learned to the publishing world. If you couldn’t make it to the event, here were my big take-aways:

Web 2.0 is social software
Consultant Dion Hinchcliffe’s tutorial on the Web 2.0 landscape summed it up best: Web 2.0 means software that gets better the more people use it. This is radically different from traditional software development, which gets better only when programmers add new features. (In the case of Microsoft Word, it generally gets worse.)

The best example in the publishing space is LibraryThing, which has a more accurate book catalog than Amazon.com, but also content found nowhere else. My favorites are the Legacy Libraries, which collect works associated with famous dead people. The Legacy Library project illustrates a related principle of Web 2.0: encourage unintended uses. LibraryThing was designed for individuals to catalog and rate their own books, but this user-driven initiative has added tremendous unexpected value.

Thinking outside the box
That is, outside of a single computer (geeks like to call them “boxes”). More Web applications are either being built on top of other services, or make use of so-called cloud computing. Amazon, Google and other providers now offer a wealth of ready-made software and infinite computing power to allow companies to leapfrog over problems of cost and scaling.

Only a few years ago when I was approached by a publisher to start a project, we would begin at the beginning: purchasing a computer, selecting a service provider, writing some HTML, crunching some data. With services like Amazon’s Elastic Compute Cloud, there’s no longer any need to buy hardware: instantly an application can be deployed on one computer, or a thousand, at very low cost. This makes experimentation much more feasible: if no users come to a new product, no expensive hardware investment has been wasted. If it’s successful, a few keystrokes can add 10X the computing power.

Cloud computing has also created tremendous benefit for offline processing tasks, as shown by The New York Times when converting their digitized archive for use on the Web.

It’s not just about people, it’s about data
Finally, Toby Segaran’s talk on “The Ecosystem of Corporate and Social Data” reminded me how much value publishers have. Toby explored clever ways of finding usually-expensive data for free (for example, rather than paying for Yellow Page listings of restaurants, he scraped the New York City health department Web site, which includes ratings of every food-service facility).

Diving deeper, he emphasized how much value can be added to digital services if they are already full of content. Wikipedia came preloaded with a public domain encyclopedia, as it’s much easier to correct or update old content than to enter it wholesale. The more of your content that users can find and interact with (for example, by providing an extensive full-content backlist), the more engaged they’ll be.

Speaker presentations for the conference are available here: Web 2.0 NYC presentations.

Related Stories:

• Tim O’Reilly: “What Is Web 2.0?”
• Cloud Computing’s Potential Impact on Publishing
• Q&A with Hadrien Gardeur, Co-Founder of Feedbooks
• Q&A with Susan Danziger, CEO of DailyLit
• Q&A with WEbook President Sue Heilbronner

All of the above methods have the same flaw: AZW does not support the kind of advanced layout available in formats like PDF, and non-Latin fonts aren’t easy to convert. What if you need to review a complex legal form, or read a graphic novel, or one in Chinese? A hidden feature can help.

The Kindle has an undocumented picture-viewing mode that was first uncovered by Igor Skochinsky. Although the black and white E Ink screen is not especially good at displaying actual photographs, it is quite good at rendering line art and text.

Here’s how to do it, using PDF as an example. Note that unofficial features may be buggy and could damage your Kindle; proceed at your own risk.

1. Convert the PDF to a series of images. Commercial versions of Acrobat should be able to do this in batch, but users of free readers may have to convert a page at a time. The Kindle can read JPEG, PNG and GIF; the latter two will work best. Because the picture-viewing application doesn’t support a table of contents, you’ll need to name the image files in ascending alphabetical or numeric order (e.g. “0001.jpg,” “0002.jpg,” etc.) For best results, resize the image to 600 x 800, the resolution of the Kindle screen.
2. Connect the Kindle to your computer using the USB cable. Once connected, browse to the Kindle’s drive. If you have an SD card installed that will appear on your computer as well. The following procedure works on either the Kindle or the SD card. I prefer to do everything on the SD card — it feels safer.
3. Create a folder called “pictures,” and a folder inside of that with the name of your “document.” Put the images in the document folder. Disconnect the Kindle from the PC. When you go to the Kindle’s home screen, nothing will have changed. This is where the secret feature comes in:
4. Press Alt-Z from the home screen. Your book title should appear in the list.
5. Click on the book title. It will open the first image. Use the normal Kindle next/previous buttons to page through the “book.” The picture viewer has menu options of its own to control the size of the image and how it’s rendered.

Credit: octopus pie

Of course because the “PDF” is really an image it’s not possible to search the document or rescale the fonts. Text-heavy PDFs should be converted in one of the recommended ways.

This same technique can be used to load image-based documents directly, such as comics. (Peeking inside the “pictures” folder after it’s been read by the Kindle reveals a file with the extension manga, suggesting that the picture viewer was intended to be used for this purpose).

It’s also possible to convert documents in Russian, Chinese or other non-Latin scripts this way. The Kindle does have support for embedded non-Latin fonts as part of its “Topaz” file format, but there are no tools for end-users that output Topaz.

(Screenshots courtesy the undocumented Alt-Shift-G feature, which saves to the root of the SD card.)

Related Stories:

• Optimizing Web Content for the Kindle Browser
• Hacking the Kindle
• Exploring DIY E-Reader Platforms
• What Would Your Ideal E-Reader Look Like?

Amazon’s Kindle store is convenient, easy-to-use and stocked with thousands of titles.
But what about publishers and content distributors who want to reach the
estimated 240,000 Kindle users without going through Amazon’s program? And what about content formats that the Kindle does not directly support?

One selling point of the device is its free, ubiquitous Internet service
and Web browser. Amazon has filed the browser under “Experimental” but it’s
quite usable as-is. With a few simple changes to a Web site’s HTML code, it’s
even possible to specially cater to Kindle users.

The screenshots used in this article are from the mobile version of
Bookworm, my Web application
for reading ebooks in the EPUB format.
Although what’s being displayed is ebook content, it’s being delivered by the Kindle’s browser, not the Kindle ebook technology, which does not yet support EPUB.

Because the mobile Web version is
already heavily optimized for small devices, the layout is simpler than
a traditional Web site. What works for an iPhone or other wireless device
will also be a good starting point for the Kindle, although we’ll see there
are some special considerations that don’t apply to any other device.

Default or Advanced Mode?

When the Kindle ships, its Web browser is in “default mode.” It will not load
images or CSS styles, but it does render basic HTML tags like the italic tag <i>.
Personally, I prefer “advanced mode,” which displays Web pages more like a
traditional browser, but some sites can be unreadable in this mode.

When optimizing for the Kindle it’s best to consider that most users will not
change from “default mode,” or even realize that the option exists.

How different are these modes? Here is a comparison shot of the same screen
from Bookworm in both modes:

My list of books in Advanced mode, showing tabular layout and more advanced font styles	My list of books in Default mode

In Default mode, all the information about the books runs together.
It would be better to present this as a simple
vertical list, the way the Amazon Kindle store does, rather than as a table.

Font Size Considerations

You can choose from six font sizes in the Kindle browser. As a content creator, you can provide a wider range of font sizes in your Kindle-formatted Web page, but take care that they aren’t too small. The device doesn’t clearly display fonts that are smaller than its default six sizes.

In this screenshot, the table of contents for a Bookworm book is not readable,
even though this page has already been tailored for the small display of mobile phones:

This problem is only likely to occur in Advanced mode where stylesheets are
activated.

Usability

The Kindle’s method of selecting and traversing hyperlinks is unique. The user activates links by selecting along the vertical, or Y-axis, using the scroll wheel. When multiple links fall on the same line, the Kindle will open a dialog box so the user can clarify which link is the target.

In Bookworm, users move to the next or previous chapter by selecting navigation
links lined up horizontally (see the top row of the first image). In the Kindle,
this presentation forces the user to click a second time to select the appropriate one:

For commonly-used navigational items like this, line up the links in a vertical row:

1. Next
2. Contents
3. Previous

Now no second click (and accompanying page refresh) is necessary.

It’s also important to remember that the Kindle is a black-and-white
device. If your site uses text color to convey any useful information
(such as what is or is not a hyperlink), re-work the design to
accommodate a grayscale display.

Finally, keep pages short. The Kindle cannot scroll; long Web pages are paginated like books. Pagination with E Ink devices is slow relative to scrolling on a computer screen. If possible, keep all your content on the first Kindle “page” when viewed at the default font size.

Targeting the Kindle

Web browsers are identified using their “user-agent” string. The current
version of the Kindle is broadcasting this user-agent:
Mozilla/4.0 (compatible; Linux 2.6.10) NetFront/3.3 Kindle/1.0
(screen 600x800).
It’s beyond the scope of this article to describe how to set up your
Web site to deliver different kinds of content to different browsers,
a process that varies considerably with your site’s technology.

How do you test your layout if you don’t have a Kindle? There’s no
substitute for having the real device (tell your boss it’s for “research”), and currently Amazon does not offer any kind of browser emulator. Some possibilities:

1. Disable stylesheets on your browser and look at the output. Does
   it still make sense? (Instructions for
   disabling stylesheets; Firefox users should install the Web
   Developer add-on)
2. Use a text-only browser like Lynx

Some Last Advice

Don’t spend too much time on this process. The next version
of the Kindle is expected soon, no doubt with an improved browser. Indeed, Amazon could offer a new version of the existing browser at any time. Most of the changes recommended above should take little time and money to implement, and can make a great difference in user experience.

In addition, optimizing your site for
small-screen browsers can have other benefits: they allow an
increasing number of mobile users to get quick access to your content,
and aid accessibility for screen-readers and other non-standard
browser types.

Related Stories:

• TechCrunch Reports 240K Kindles Sold
• Hacking the Kindle
• Ebook Format Primer
• Exploring DIY E-Reader Platforms
• What Would Your Ideal E-Reader Look Like?

How big is the archive? Eleven million individual articles from 1851 to 1980, or 4 terabytes of data (over 4,000 gigabytes). The Times got it ready for distribution in 24 hours, for a total cost of $240 in computing fees and $650 in storage fees.

As part of their original TimesSelect subscription service, the paper had scanned their entire print archive. Each full-page scan was cut into individual articles. Typical of newspaper format, the articles often spanned column or page boundaries, which meant that many articles were composed of several scans. In the original subscription-based program, whenever a reader requested one of these historical articles, the Times computer would need to stitch together all of the scans for a particular article before presenting it.

This on-demand process used significant computing resources, but because TimesSelect was subscription-based there was never much traffic. Once this archive was open to the public it was expected to generate greater usage, and the safest approach in those cases is to serve pre-generated versions of all 11 million articles. Using traditional software development practices — with a single computer churning through one article at a time — the processing could potentially take weeks and tie up Times servers that were needed for other tasks.

Gottfrid turned to Amazon Web Services (AWS) and its two main products:

Amazon Elastic Compute Cloud (EC2) is a form of “virtualization” where one very large computer is divided up into many virtual computers that can be individually leased out for use. Traditional hosting costs money whether the server is working or idle; in EC2 you pay only as long as the virtual computer is running. When it’s no longer needed, it’s shut down. This makes the service ideal for one-off processing jobs.

In addition, Amazon doesn’t care whether you use one EC2 “instance” 100 times, or 100 instances all at once — the cost is the same. The difference is when you can usefully divide a job into 100 concurrent tasks, because then it takes 1/100th the total time.

Amazon’s other major AWS offering is the Simple Storage Service (S3), for large-scale file hosting. Like EC2, it is a leased model — you pay only for the space that you use in a given time period.

Gottfrid leveraged these technologies in combination with a relatively new software library called Hadoop. Hadoop is written in the Java language and is based on work done at Google. It allows programmers to very easily write programs that can be run simultaneously on multiple computers.

Combining Hadoop concurrency with EC2 and S3, the Times was able to run a job that might have taken weeks of processing time and complete it in 24 hours, using 100 EC2 instances. They were pleased enough with S3 it became their permanent hosting platform for the scans. Hosting with Amazon or other cloud computing services is usually cheaper and has much better bandwidth than the average provider, although downtime can and does occur.

At last year’s OSCON, the Times announced the formation of its developer blog, Open. You can read more about the original AWS project as well as TimesMachine, a project that became economically feasible due to the low cost of AWS.

Related Stories:

• How Hackers Show it’s Not All Bad News at the New York Times
• Cloud Computing’s Potential Impact on Publishing
• The Media Industry’s Perspective Problem

Reference publishing and resource discovery — Reference publishers invest time and money in bespoke search interfaces for advanced users, but are users finding them? In the ALA panel “The Future of Electronic Reference Publishing,” librarians repeatedly commented that multiple reference sources are confusing to users, and that resources must also be discoverable via Google and the library’s own digital catalog.

If users do go directly to an individual resource or platform, the search interface should behave “like Google.” Although the panel of major reference publishers did state that they are converging on Google’s query language, many legacy systems remain that would be economically infeasible to re-tool.

Library catalogs and systems — The need for more transparent, network-based services applies to the library catalog as well. In the marathon session, “The Ultimate Debate on the Future of the Library Catalog,” speakers identified a critical need for geo-based services and APIs for finding what’s in my local library — now. Once a book is located I should be only a few clicks away from reserving it or even ordering it for delivery to my home.

That dream is still far off — even with a service like WorldCat it’s not currently possible for me to find and reserve a book at my local library. The closest offering presented on WorldCat is Harvard University’s library, which is not about to lend to the likes of me. The problem is even worse for rural libraries. As for my local library — I love books and this post is the first time it even occurred to me to visit their site. I’m not alone in that.

Ebooks — This is a transitional time in publishing, and while many patrons still prefer print, an increasing number are asking for electronic books, especially in university libraries. Students and academics emphatically reject DRM and restrictions on usage, but many ebooks sold to libraries have technical barriers to printing, cut-and-paste and downloading.

Licensing and subscription costs are also a concern for libraries. Ebooks may be re-priced or re-bundled, challenging the basic assumption that once a library buys a title, it owns the book indefinitely. Librarians want assurances that the products they purchase are either available perpetually, or at least have clearly-stated licensing terms that do not change without notice.

The ability to safely and permanently archive electronic books has been a long-time concern of some librarians, but the floods in New Orleans and Iowa have changed some minds. Off-site electronic archiving would save at least some resources, especially for very small or rural libraries can’t afford state-of-art preservation facilities.

Related Stories:

• What OpenID Can Do for Academic Publishers
• Aggregated Ebook Service Suits Research Publisher
• Survey Results: Students Rely on Digital Tools for Research
• Google Opens Book Search with API

I’m not a hardware person, but the last few years have seen an emergence of open hardware platforms designed to allow even ordinary programmers like me to modify and customize small devices. As far as software goes, an e-reader is pretty straightforward: it’s just some text on a screen. That shouldn’t be too hard, right?

Surveying the landscape of hardware options, I’ve ranked below a variety of devices from “friendliest” to “most-intensive DIY.” I’m not addressing PDA or phone devices here, largely because I consider their screen size and text rendering insufficient (but plenty of people disagree).

The Chumby — With a 3.5″ touch screen and reasonable $175 price tag, this little wireless computer in a bean bag is an obvious candidate. There’s a full-fledged development environment and large community of users. Most Chumby applications are written in a lightweight version of Flash, which is easy enough to develop in.

It has a few downsides, though. The Chumby doesn’t have much storage space at all, so any ebooks would have to be saved online and streamed to it, a page or a chapter at a time. Since it’s meant to be an always-on wireless device, that seems doable. The screen might be too small to comfortably read lots of text, as it’s meant for short bursts like RSS feeds or Twitter updates.

Unfortunately, it’s powered by a wall outlet, with only a small 9-volt battery for emergency backup. People on the hardware forums have managed to hack in rechargeable batteries, and I wouldn’t be surprised if a totally-wireless Chumby is forthcoming. [Disclosure: O'Reilly AlphaTech Ventures is an investor in Chumby Industries.]

BugLabs — The most open of the commercial hardware platforms, BugLabs sells individual pluggable modules that support various features, from touchscreens to cameras to GPS. It looks like a great platform, but it’s very expensive ($349 for the base module plus $119 for the 2.5″ touch-sensitive screen). The screen is probably too small for comfortable reading, but the company Web site promises a larger display soon.

Both the Chumby and BugLabs have touchscreens, which is key for making small screens more usable.

The Kindle — All the heavy lifting has been done already to get into the Kindle filesystem
and peek inside. It’s probably too difficult to extend the existing Kindle environment without true source code, but it might be possible to do some simple things, like add new fonts. Few people have really explored hacking on e-ink devices, largely due to high cost and low availability. I suspect when the first color e-ink devices come out, used black and white ones will become popular playthings for enthusiasts.

YBox2 — For
the ultimate DIY experience, the YBox2 platform is a pile of electronic parts you solder together and assemble in an Altoids tin. It doesn’t come with a touch-screen, or any screen at all: you connect it to a television or monitor. It uses the tiny Propeller chip, which powers many hobbyist devices and small robots. Like the Chumby, YBox2 comes with networking capability but little storage, and would need to stream book content from the Internet. The networking isn’t
wireless and of course there’s no handy rechargable battery, but if you
are the kind of person who can build a YBox2 you probably know how to
make those too. I am not that kind of person.

While I’d be happy to crawl around a hacked Kindle, I know I’m not ready to program my own microcontroller. BugLabs seems great from a developer standpoint, especially when they release a larger screen, but I’m unwilling to shell out almost $500 just to experiment. The Sony Reader doesn’t have networking, so that’s much less interesting. Perhaps a Chumby is in my future. Any other options?

Related Stories:

• Select O’Reilly Books Soon on Kindle, and as DRM-free Digital Bundles (Including EPUB)
• What Would Your Ideal E-Reader Look Like?
• Ergonomics and Ebook Success
• Hacking the Kindle

This question framed “The Agile IT Organization,” a lively and well-informed discussion at the recent Society for Scholarly Publishing annual conference in Boston. As a software engineer, I’ve used both agile and traditional product development methodologies and I was interested to hear the perspectives of other programmers as well as publishers who’ve gone through the process.

Geoffrey Bilder of CrossRef provided an introduction to agile development practices, which are concisely summarized in plain English by a core set of principles.

Summarizing even further, agile development means:

1. Minimal up-front specification. A project has high-level goals (e.g. “make our back catalog searchable and available for print-on-demand purchase”), but is not fully described before
   development begins.
2. Frequent, short-cycle releases. A project is broken up into mini-projects, each with a small set of features that take only a few weeks to implement. Every release (“iteration”) has a specification,
   development and testing phase. This means that every couple of weeks the software is fully usable, although it may have very few features at the start.
3. Change to the product design is accommodated and even expected. Market conditions, corporate re-organization or user demands may mean that new features are added or old ones are re-worked. Changes are treated as just another iteration.

The panel at SSP focused on two approaches: internal, IT-driven products, and those developed by a third-party vendor. Larry Belmont, manager of online development at the American Institute of Physics, gave an excellent presentation on the in-house approach. His organization ran its first agile project with a timeline measured in days rather than weeks or months.

Leigh Dodds, CTO of Ingenta, provided the vendor perspective, and described the principles of a formal type of agile development known as Scrum.

The panel was, to their credit, enthusiastic about the approach, but agile development requires commitment and is not right for every
organization or project. Some caveats that need to be emphasized:

• Short development cycles come with a price: you will be asked to review and comment on small pieces of the larger project, and be involved on an almost daily basis. Many publishers need vendors they can treat like plumbers: “I want a new sink put here, it should look like this, call me when it’s done.” If someone in your organization isn’t prepared to think very hard every day about copper pipe fittings, agile isn’t right for you.
• Project managers must be empowered to make decisions. Whether the project is in-house or vendor-driven, every day the PM will be asked to make calls without appealing to higher powers. When editorial buy-in is required, or when the product needs a larger review, consider a hybrid approach: appoint a single decision-maker with deep editorial knowledge to work on evaluating, testing and approving each iteration, but use a more traditional alpha/beta/gold release process for the wider group.
• Product features may change, but time and budget should be invariant. Hard deadlines might seem to be antithetical to the free-wheeling, change-friendly agile approach, but in my experience they’re critical. They focus the entire team: key decision-makers cannot spend weeks in committee, IT personnel don’t fear the “death march” project with no end in sight, and it’s more difficult to introduce budget overruns that cause friction with management and vendors. If an agile project
  does run out of time, you will still have a launchable product that’s been thoroughly tested and reviewed all the way down the line, not something just out of beta with weeks of QA ahead. Many agile methodologies use the hard deadline, or timebox, as the primary method of structuring the project.

“Release early, release often” can sound a lot like “throw whatever we’ve got out the door.” This is one reason why the iterative approach has been so embraced by Web startups: each small release has been thoroughly tested and evaluated, and there’s never a moment where the software doesn’t work. It’s possible to to go live with a project that might not be “finished” according to the original master plan, but might otherwise be caught up in insurmountable technical hurdles or tied up in editorial review.

If publishers are going to be ready for an “iPod moment,” this kind of flexibility and responsiveness is critical.

Related Stories:

• Targeting Small Companies with Small Products
• Publishing Industry Not Prepared for “iPod Moment”
• Tutorial: Add AB Meta Tagging to Your Blog
• Ebook Format Primer