# An Open, Webby, Book-Publishing Platform

WordPress as Book Publishing Platform

This short article outlines some ideas about an open source, online platform for making books, based on WordPress. My thoughts here come out of our experience building Book Oven (a thus-far closed, proprietary system envisaged to be just this); and subsequent conversations with John Maxwell, of Simon Fraser University’s Masters of Publishing Program, and Kirk Biglione of Oxford Media Works.

I am happy to report this is more than just thinking: this past term (January to April 2010), John Maxwell and his MPub students built a prototype of this WordPress-based publishing system, and tested the prototype by creating, and publishing the Book of Mpub, available in print-on-demand ready PDF, epub, and html.

Background: Book Oven

At the end of 2008, my co-founder Stephanie Troeth and I started Book Oven, an ambitious venture to work towards transforming the book publishing process into a webby, connected process. The key insights behind Book Oven were the following:

• publishing a book is (almost always) a collaborative enterprise
• online tools (should) make collaboration on making books easy(er)
• if you build a “book” in the cloud, using structured mark-up, then expression of that book in various forms (print, epub, pdf, mobipocket, html, etc), on various devices (including paper & print) becomes arbitrary, and should be nearly trivial
• further, if the “book” exists in the cloud, then the range of things that can be done with this “book” multiplies significantly
• if a system built on these ideals is implemented well, it will be transformative, both for professional publishing workflows, and for the emergence of a new grassroots of indie publishing.

I am still deeply committed to this vision.

But I have shifted towards a belief that the above-described platform should be open source. Or at least, an open source version of such should exist.

The Big Revelation: WordPress

We’ve done some good things with Book Oven, but around November 2009, we shifted focus to the bit of our platform that was (according to our analytics) the most engaging to our users: Bite-Size Edits. That shift occurred in parallel with a revelation:

We were trying to recreate so much in Book Oven that was already handled well by another class of software, namely blogging software, and specifically WordPress.

My thoughts about WordPress were crystalized in October 2009, during a conversation with Shana Kimball of the Scholarly Publishing Office of the University of Michigan Library. I was pitching Book Oven as a good tool (in progress) for academic presses to use in their production workflows. Shana had various hesitations — open source vs proprietary being a big one — but during our conversation, Shana said something like: “It would be great to have a tool that’s as easy to use as WordPress. I love WordPress.”

Indeed.

So, I started having some conversations with some people I knew who were already doing some work in this direction, in particular: John Maxwell, at Simon Fraser University, who was experimenting with prototypes for html-first book publishing systems, and was exploring different candidates, including WordPress; and Kirk Biglione who had independently started poking at WordPress as a book-publishing tool.

I also floated the idea to a few others who are doing some of the most interesting things right now in publishing/tech, especially Liza Daly and James Bridle, and two of the best WordPress hackers I know, Steph Daury (who works for Automattic) and Jeremy Clark.

The idea, everyone agreed, had some legs.

WordPress, it seems, is an ideal candidate as a platform on which to build an open source, online, webby, book-publishing system. There may be other likely candidates, but WordPress has the following characteristic which suggest to me that it is an excellent place to start:

• it is a familiar and comfortable tool to most writers and
publishers who are at all engaged online
• it is a stable platform that can handle just about any scale of
traffic you can throw at it (the New York
Times
, New York Times Blogs, for instance, run on a heavily-hacked version of WordPress)
• it is open source
• through its plugin architecture, it is infinitely extensible
• through
its template architecture, it is infinitely stylable
• through WordPress Mu, it is
infinitely scalable
• it has a huge, world-wide community of committed developers
• existing plugins and plugin suites already achieve much of what would be
wanted in a WordPress-based book publishing system.

[NOTE: I've since discovered Leanpub.com, built by Ruboss, which is already going along this path].

The Outline

I’ve described above some of the reasons why WordPress is, I believe, a good candidate as the basis for an online book-publishing platform. Here is a proposal for some very rough product specs:

• Authors/editors can add text
• Editors can edit text
• The editing/publishing process can be public or private, with easy assignment of various permissions (none, read-only, read/edit, read/edit/admin)
• Formatting creates structured html
• Finished text can be generated in the following formats:
• plain text
• epub
• html
• InDesign-compliant markup – to generate a professional print output from In-Design
• automatic print-ready pdf – using something like a web-based LaTeX system
• etc.

WordPress can do much of this already, but not all of it, and certainly not everything you would want it to do. The finished platform should have (among others) the following plugins/characteristics:

1. robust version control
2. digress.it (based on the old commentpress)- to allow para by para commenting for editors, and later, if desired, for readers
3. wordpress –> epub conversion
4. wordpress –> ~LaTeX –> print-ready pdf conversion (or similar)
5. wordpress –> InDesign-compliant mark-up conversion
6. book-friendly front-end template(s) (including Table of Contents, Title page etc)
7. generation of a download/(sales?) page that lists available formats (epub, html, pdf etc)
9. a book metadata generation/management tool (ONIX, OPDS compliant?)
10. …etc.

This list of plugins can continue, subject to the interest of developers, and the needs of users of such a system.

SFU and the MPub Prototype

All this would be just a lot of writing and good intentions and conversations, except for John Maxwell and his team of talented students at Simon Fraser University, including: Vanessa Chan, Cari Ferguson, Kathleen Fraser, Cynara Geissler, Ann-Marie Metten, and Suzette Smith.

In the span of four months in 2010, the SFU MPub team did two extraordinary things:

1. they built a prototype of this WordPress-based book publishing system (tied in with InDesign for the print book)
2. they published a book using the system – suitably, it was student-essays about the future of publishing: the Book of MPub

I had the pleasure of seeing John and some of his students present the results at BookCamp Toronto this past week, to a crowd of publishers, writers, designers, and technologists.

The Reaction

I was curious to see the reaction to John’s presentation at BookCamp Toronto, with a wide range of people in the room. Particularly encouraging was Ingrid Paulson’s take on it: Ingrid is one of Canada’s best-known book designers, and was excited by the idea of streamlining and formalizing the process of text/mark-up delivery from publishers. She seemed entirely open to a better toolset to make that happen. Others in the room were equally intrigued.

For myself, I was amazed at what the SFU students delivered in such a short time, and was reignited with excitement for this project. I have no doubt that a streamlined online publishing system, using structured mark-up, will transform the publishing industry. And my bet is on WordPress as a great starting platform to do just this.

Whether or not it could be the long-term winner, I know not, but something will be, and WordPress has a whole lot to recommend it.

And how about you? What do you think?

Bio: Hugh McGuire builds webby things, and writes about media, publishing, mass collaboration, and technology. He is the founder of LibriVox.org, the volunteer-run makers of free public domain audiobooks; and Book Oven, which makes Bite-Size Edits, an online editing game/tool. He is a co-founder of BookCampToronto. His personal site is hughmcguire.net and you can find him on twitter at @hughmcguire.

[Pic of Hugh McGuire by Ron Grimes - Flickr / Twitter]

• http://omaniblog.blogs.ie Paul O'Mahony (Cork)

Hi Hugh,
I found you via a RT by @eoinpurcell. Very interested in your article, so full of links to people I don’t know.
My angle is the point of view of someone who’s just finished publishing an epic poem on my WordPress Blog. I’ve put up 72 bits (33 Cantos, in case you’re interested).
I now want to turn this material into a couple of things: a self-published book which I’ll sell at readings (or to those who order it via Twitter or Facebook or email).
Also I’m interested in producing an e-book version, full of hyperlinks for those who want to know about the people & places in the Epic.
I’m un-techie, so I can’t be sure what you’re driving towards but this is the impression I have:
(1) the author can write on WordPress, the bits of the book, piece by piece
(2) the author can edit and do all the usual stuff
(3) an editor could do editorial work
(4) the software can gather it all into one whole and turn it into whatever format the publisher wants (including I hope getting me an ISBN) – for a fee of course

I gave the example of my current challenge, but I think I could as easily be writing a novel and want all the chapters turned into publication.

Have I understood you right?

• http://www.simplissimo.com.br Eduardo Melo

Hi Hugh. I’m a brazilian e-publisher, our e-bookstore will go live on 06/28. I work with e-book since 2008 at an NGO called Editora Plus.

I think you’re on the right path. WordPress is full of possibilities, if people understand how to use and build things upon it’s power, as you seem to understand. I’ve been following Bookoven and bite-size edits since it’s beginning, and had the same feel: wordpress already do a lot.

An open Book-Publishing platform would be revolutionary. You gotta make this happen, even if, as you say, as an open version of a closed system (but I’m not sure if WP licensing would allow you to build a closed system with it). You gotta a lot of experience on the web since LibriVox, you’re the right guy for the right job!

Thanks,
Eduardo

• Mary E. Ulrich EdD

Now this is teaching and learning.

Congrats to Fraser University and these students for not studying “the history of publishing (yawn)” but instead for being a part in actively making history (Rah! Rah!).

• Mary E. Ulrich EdD

Now this is teaching and learning!

Congrats to the professor and students at Fraser University for not just studying the “history of publishing” (yawn) but instead actually creating and leading the rest of us into history.

I tweeted that this seems to me more a book writing platform than a book-publishing platform. I am sure that I dont know the WordPress system at all well enough. But I would expect a ‘digital publishing system’ in the sense I would give that term to be quite neutral about the way in which books may be written/edited/designed, and about the ways in which they could be commented/annotated/searched; but constitutive of the ways in which they can be digitally owned/sold/shared/translated etc. Longer than a tweet but perhaps still too compressed!

• http://mediacommons.futureofthebook.org/mcpress/plannedobsolescence KF

Wow – this is fantastic. I’m co-coordinating editor of MediaCommons, and our book-ish wing, MediaCommons Press, is currently running in WordPress, using the new CommentPress plugin (the version you link to, which has recently been updated). It would be wonderful to have the kind of integrated system for producing multiple formats of our texts.

The one additional element that I’d particularly be interested in would be a plugin to assist with ingest of lengthy texts into WordPress; publishing Planned Obsolescence in this fashion required a lot of cutting, pasting, and reformatting (plus hand coding for our rollover footnotes). A plugin that would easily take a text from a lengthy .doc or .rtf file and transform it into a many-post WordPress site would be an enormous help.

• http://hughmcguire.net Hugh McGuire

@paul: yes, you’ve understood right. You should try leanpress.com & see if it works for you. they’ve build a good amount of this already.

@eduardo: thanks! i’d love to see this happen, would love to push it along.

@mary: amen!

@adam: I suppose this is “publishing” in the sense of desktop publishing. Call it cloud-publishing (or whatever). But the problem of being neutral about how books are written/made etc is that you end up in the situation we are in right now: where the end product is a “print book” and ebooks are just another thing you might do with your print book. That misses the point entirely. A book should be: words+media+structure –> form … where the form (paper, epub, whatever) is arbitrary from a tech point of view, and is simply business decisions. THat’s not the case right now. So I think publishers need to have a fundamental shift in what they consider a “book” to be (that’s another post). And of course all the rest: rights, marketing, distro etc is the business-half of publishing. Still, there is a making part that is not unimportant.

@KF: we’ve done some docx conversion work at Book Oven that could be applicable to this … but yes, good point.

• http://tkbr.ccsp.sfu.ca/ John Maxwell

@adam Let’s not make too much of WordPress’ contribution here. In my mind WordPress is simple a CMS, albeit one used (and even loved) by an awful lot of people. The reason WordPress is central to this discussion, and our projects, has to do with its ubiquity and its flexibity. It’s a tool for writing, editing, and reviewing on the web. Not that it has any special powers.

What we’re up to at SFU, and what I think Hugh’s Bookoven is about, is using WordPress as the core content management piece in a publishing toolkit—because it already exists, and it does a good job. The real work at SFU includes building a XML-based bridge to Adobe InDesign for print production, working out tools for editorial tracking, and so on. E.g., when we (or somebody) get around to it, there’ll be an ePub plugin: turn this set of posts into an ePub file, etc. WordPress lends itself so nicely to plugins and whatnot…

Publishing has to rely on a core set of tools. The opportunity here is to stop using MSWord (and related 20th-century things) and embrace the open web instead.

• http://www.sfu.ca/pubworks Suzanne Norman

Simon Fraser University’s Canadian Centre for Studies in Publishing (CCSP), is the umbrella under which the Master of Publishing and the Summer Publishing Workshops fall. The Centre has been pushing boundaries and test driving new ideas for many years. It works closely with the publishing industry, with emerging technology, and with leading technology strategists. Some of the brightest minds work and study at the CSSP as evidenced with this latest project. Kudos to @jmaxsfu, (who also teaches in the Summer Publishing Workshops), and all the students involved. Hands-on learning at its best!

• http://www.schlagergroup.com Neil Schlager

Great post, Hugh. As it happens, my company has just started down this road. We are working with a tech consultant to migrate pretty much our entire publishing platform to WordPress. So far the testing is going extremely well, and indeed the platform promises all those things you list above. This is a real-world solution that is ready to be adopted now. I’m pretty confident that it will transform not only our internal production processes but also will inform our end products, which include books/ebooks/websites.

• http://www.river-valley.com Kaveh

Great minds think alike. ;-)

We have been working on this too recently. The way to go is to spew out xhtml, convert to your favorite xml, then publish xml-first.

And we would use an automated filter for ePub generation. Very exciting.

• bowerbird

ok, first of all, i could’ve saved you a lot of work
if you woulda talked to me before doing all this…

i’ve got a toolchain that does what you want to do.

(albeit perhaps not the _way_ you’d prefer to do it,
since your working assumptions don’t always work.)

i mean, it’s great you can hack wordpress, but your
maintenance will then require wordpress developers.

might as well build a purpose-focused tool instead…

-bowerbird

• http://oxfordmediaworks.com Kirk Biglione

I wouldn’t stop at just publishing the book. For smaller publishers WordPress can easily be used as an online catalog and even a storefront. Add an OPDS feed and you’ve got a engine for discovery and acquisition.

Of course John is right about WordPress not having any special powers. The system we’re all envisioning could be built on any number of open source content management systems. But I keep seeing the word “love” used in relationship to WordPress. Historically that’s not a word people have used much when talking about a CMS. THAT may be WordPress’ real power. It’s a system people actually like to use.

• http://hughmcguire.net Hugh McGuire

@bowerbird: would love to see it. i’ll email you, maybe we can work something out.

@kirk: LibriVox publishes about 100 books a month, and has a catalog of 3,500 books. All on WordPress:
http://librivox.org …that’s a pretty “big” publisher, even if everything is free.

• Kirk Biglioe

> i mean, it’s great you can hack wordpress, but
> your maintenance will then require wordpress
> developers.

There is a huge and growing community of WordPress developers. If my ZML toolchain needs maintenance there is only one bowerbird.

• bowerbird

> there is only one bowerbird.

you got that right! :+)

but the essence of my tool-chain is pure simplicity.

so anybody with any degree of programming chops
will be able to jump in with great fluency right away.

indeed, i’ll gladly guide and supervise such efforts…

> There is a huge and growing
> community of WordPress developers.

then it should be easy for them to keep up with me.
but if i were you, i’d start recruiting ‘em right away.

-bowerbird

• http://sillybean.net Stephanie Leary

I absolutely love this idea. I’ve just finished writing a book on WordPress, and every day I wished I could just use HTML instead of the publisher’s awkward Word template.

I’ve written an HTML import plugin that, among other things, includes an option to clean up Word’s craptastic HTML. A couple of the plugin’s users have asked about importing other formats, like .txt and .rtf, and I’ll work on those over the summer. I would love to do a Word file importer; right now my process for my author clients is to save as HTML and import those files using my plugin. Getting the stuff into WordPress is the easy part; parsing the original file and handling the character set conversions is tricky.

I’d love to work with you on this. Let me know if there are any other pieces of the puzzle I can help with.

• Laisvunas

Your main idea is correct – ebook publishing system should be CMS based.

But the choice of WordPress I consider unfortunate. Although widely used and popular, WordPress is too much blog-centric and inflexible (despite huge number of add-ons) to be a basis for robust publishing system. Why NY Times online version “runs on a heavily-hacked version of WordPress”? – because WordPress is *inflexible*. For simple projects WordPress based publishing system may be OK, but anything moderately complicated will require massive hacking of WordPress core files.

Much better choice than WordPress would be ExpressionEngine CMS, which despite being commercial product (quite cheap) has all power and flexibility to accomodate even the most fanciful publishing projects. Writing several modules for ExpressionEngine would achieve the purpose of creating CMS based publishing system much better than hacking WordPress. Even most exotic tasks can be achieved with ExpressionEngine without any hacking of its core.

• http://www.river-valley.com Kaveh

Laisvunas:

The argument of open free software is an important one. I have no idea about ExpressionEngine, but what happens if in 5 years’ time they go out of business, or quadruple their prices? The customer will be screwed with nowhere to go.

WordPress is open and free, so rather than “hack” it, we can extend it, or get it to talk to other programs. My advice is to stay away from anything proprietary, for financial, ethical, and practical reasons.

• http://hughmcguire.net Hugh McGuire

@Laisvunas … what exactly does ExpressionsEngine let you do that you could not do with WordPress.

I know lots of reasons WordPress is attractive for this project, but no reasons why ExpressionsEngine might be. You’ve offered some general principles… but:

-what can I do with EE that I can’t do with WP?

• situationmaster

Hugh, have you looked at Rhaptos (what CNX Connexions from Rice Univ built)?

• Kirk Biglione

@laisvunas ExpressionEngine is a very flexible CMS, but it’s not open source. As a result the developer community is likely several orders of magnitude smaller than the WP dev community.

I would encourage you to take a look at WP 3 which will be out any day now. custom post types combined with custom taxonomies bring WP much closer to matching EE’s capabilities without any need to hack the WordPress core.

• bowerbird

sometimes people get so simplistic that it’s scary.

like little kids who have touched a hot stove once,
and now are deathly afraid of it, some people say
“proprietary” like it’s the worst kind of dirty word.

the simplistic part is that they then go on to think
that “open source” is some kind of blessed savior.

it’s not. because something can be “open source”
and _still_ be so darn complicated that you remain
at the mercy of the people with the reference app.

in a phrase, there’s lots of ways you can get burned.

.pdf is now an “open standard”. (one variant is.)
but it’s so complex, even if you’re a programmer,
you have to buy a library in order to make it work.

microsoft open-sourced its x.m.l. format, but it’s
so obtuse and so difficult nobody bothers with it.

look at the code for wordpress. it’s _very_ hairy.
heck, look at the _markup_ on a wordpress page!
and the c.s.s. that goes along with it! you have to
be an expert to be able to know what’s going on…

this gets to be harder and harder and harder when
you try to make a tool designed to do _one_ thing
(i.e., blog) do some other thing (i.e., craft a book)…

maybe you can find some programmers who _want_
to do that kind of code-reworking, who enjoy that…

in my experience, as a programmer, i would rather
build a more purpose-focused entity, from scratch.

-bowerbird

• http://sillybean.net Stephanie Leary

Laisvunas, Kirk is right. WordPress 3 is much more flexible than you’re giving it credit for. The new custom content types and taxonomies are nothing short of amazing.

My biggest problem with Expression Engine’s closed nature is that it takes forever for new releases to appear. EE 2 was previewed at SxSW 2009, and is just now in public beta. WordPress has gone through several release cycles since then. Personally, I also found EE’s template system much harder to work with, although I know it has its fans.

(Hugh, did my earlier comment get eaten by the spam filter?)

• http://hughmcguire.net Hugh McGuire

Stephanie: yes, caught in the spam filter… trying to find the keys…

• http://www.imaginaryplanet.net/weblogs/idiotprogrammer/ Robert Nagle

I’ve long thought about the wordpress-to-ebook transition.

A few sticking points:

a)are you using wp simply for an editing environment or for presentation as well? I’ve been worried about the security issues inherent in php applications. Intranet, sure, but having it for public websites is fraught with risk.

b)what about Live Writer support? As WP becomes more full-featured, I want to offload the editing functions to a desktop app with the ability to save offline. For docbook people like me, I rely on XML Author. I think the key is having some local app that can validate code and do some wysiwig before it saves in wp.

c)Making an epub file is not as easy as it sounds (even with good xml code). Lots of processing and files. the best pdf processors are commercial; i wouldn’t trust wp with producing a nice pdf file.

c)the problem with WP is that even though pages and posts are hierarchical, the infrastructure of wp is certainly focused more towards posts (but with publishing, it would be the reverse).

d)The TOC builder would probably be the first plugin to write.

e)I would trust wp with handling the editing of text, but not so much with the formatting. The rich text editor and the media inserter would have to be totally redesigned and throw out some features.

f)I don’t believe wp has any concept of internal links (i.e., relative links) — in contrast to a cms like Plone. That would be a big problem for producing ebooks.

g)fonts/embeddable fonts/IP issues?

h)This may sound funny, but I’d be happy with a plugin or utility that just lets me dump all posts into one clean xhtml file (removing all the miscellaneous crap, like comments, RSS, theme stuff, widgets, etc) and storing the images in a single folder. Maybe there can be a way to set criteria (Show all posts with the category ARTBOOK and replace dates with chapter numbers, with 1 being the earliest, 2 being the next earliest, etc). I haven’t checked out how wp exports xml/wxr looks. Maybe all we need is a local tool for writing queries to export stuff from the db that looks innocuous.

i)the Rice Connexions project is great (and backed with Plone). They have solved a lot of problems, but they ask people to compose with a builtin xml editor.

Those are my random thoughts.

• http://www.imaginaryplanet.net/weblogs/idiotprogrammer/ Robert Nagle

A change of heart. I thought it was cute to use the word “innocuous” in my previous comment.

Alas, I’ve changed my mind. I prefer to use the word “noncrappy” or “inoffensive.”

• http://tkbr.ccsp.sfu.ca/ John Maxwell

What the CMS evangelists all are missing is that if WordPress were demonstrably the *worst* CMS platform in existence (which it clearly isn’t), it would be even more the right choice here.

The point is that the web and web-native tools can be leveraged, right now, by anyone (including you), as a book-publishing tool.

The point is that if you can do it with WordPress, you can do it with anything. If you can do it with WordPress, then anyone can do it.

It merely helps that WordPress is actually liked by several hundred thousand people.

• bowerbird

john said:
> It merely helps that WordPress is actually
> liked by several hundred thousand people.

we’re already up to 30 comments on this post,
and there still hasn’t been any substantive talk.

so i do not see much productive use referring to
“several hundred thousand people” in this context.

what is it that those people like about wordpress?

i’d say that they like the fact that it’s a simple way
to administer a blog. so how is that relevant here?

as far as i can tell, john, you scraped the entries
and then shoveled them into indesign (which is
not free as in beer and not free as in speech) and
used that to create some of your various formats.

i submit that you could scrape the entries from
_any_ system. nothing special about wordpress,
save for a large number of future author/users.
(except that precious few of them own indesign.)

but by all means, if i’m missing it, please tell me.

-bowerbird

• http://www.river-valley.com kaveh

Bowerbird:

Why don’t you give a link to your name so we can see what systems you have. You came out of nowhere and said we could have saved time by talking to you, but we still don’t know who you are. Please give a link. If you have a system better than WordPress, then we should all see it.

>
> like little kids who have touched a hot stove once,
> and now are deathly afraid of it, some people say
> “proprietary” like it’s the worst kind of dirty word.

I happen to think “priprietary” is a dirty word. I have been burnt twice at least. I was one of the first people in UK to work on HyperCard, one of the most innovative and beautiful languages. I became an expert and put in endless hours. Then Apple killed it.

>
> the simplistic part is that they then go on to think
> that “open source” is some kind of blessed savior.

I have a business built on almost entirely free open software. I know that I can plan as far in advance as I want. No-one can take that software away from me.

>
> it’s not. because something can be “open source”
> and _still_ be so darn complicated that you remain
> at the mercy of the people with the reference app.
>
> in a phrase, there’s lots of ways you can get burned.

Sure, but at least the code is there, not in some fat cat’s vault.

[...]

> microsoft open-sourced its x.m.l. format, but it’s
> so obtuse and so difficult nobody bothers with it.

Well, what do you expect from Microsoft?

>
> look at the code for wordpress. it’s _very_ hairy.
> heck, look at the _markup_ on a wordpress page!
> and the c.s.s. that goes along with it! you have to
> be an expert to be able to know what’s going on…
>
> this gets to be harder and harder and harder when
> you try to make a tool designed to do _one_ thing
> (i.e., blog) do some other thing (i.e., craft a book)…

But you don’t craft a book directly in WordPress. You manage content and even communications between different people. Then you extract pages as xhtml, then other systems take over. We happen to like xslt and Latex downstream.

>
> maybe you can find some programmers who _want_
> to do that kind of code-reworking, who enjoy that…
>
> in my experience, as a programmer, i would rather
> build a more purpose-focused entity, from scratch.

That would be good, but it takes a lot of effort. WordPress already has a lot of capability that can be adapted, and it has a huge user base and is solid as a rock, so we can use it till someone builds a tailor made system.

• Laisvunas

>Kaveh: I have no idea about ExpressionEngine,
>but what happens if in 5 years’ time they go out of
> business, or quadruple their prices? The customer
> will be screwed with nowhere to go.

@Kaveh EE will not go out of business because this CMS have received a cult like following among web designers; the community is huge and very creative – look at big number of high quality add-ons. In case the price will quadruple the product still will be worth the money.

> Hugh McGuire: what exactly does ExpressionsEngine
> let you do that you could not do with WordPress.

I evaluated many CMS and blogging engines for my spare time projects at Classics Unlocked (I’m trying to create there both ebook application supporting many projects and environment enabling cooperation in projects of compiling of ebooks). And I can tell that no CMS/blogging engine which supports one-click theme change fits for the basis of a robust publishing system. One-click theme change functionality brings inflexibility because it forces many backend and frontend constructs upon you.

ExpressionEngine does not have one-click theme change functionality. It has very powerful templating system and very flexible backend. And this is exactly what is needed as the basis of online publishing system. In its flexibility ExpressionEngine is absolutely amazing and unique.

It is not open source but it’s cheap and for your money you get support second to none. From my point of view CMS not being open source is irrelevant. What matters is other components of publishing system being open source, that is add-ons which convert to ePUB or to PDF should be open source. And this is surely possible – you can take commercial CMS as the basis of publishing system and to develop that system as the set of open source add-ons. For my project I’ve developed about 50 open source add-ons for EE and I’m using scores open source add-ons developed by others. BTW, my plans include add-on for EE enabling one-click ePUB creation from online data using predefined project definition.

Until now I had only once a need to hack EE core files – to add additional add-on hook. Nothing similar would be possible using WP. I can predict that with WP as the basis of publishing system you will need hacking its core while doing any moderately complex project. WP is mere blogging engine; it’s wrong idea to repurpose it for doing other thing – publishing books. To do ebook publishing you need a powerful and flexible general-purpose CMS, nothing less than that!

• http://wwwrvt Kaveh

Robert

> c)Making an epub file is not as easy as it sounds (even with good xml

I think it is trivial, with good xml code. Here is our open, free filter to convert any document with Elsvier DTD to ePub. It’s a command line code, but it’s free and open so you can modify it as you like:

http://tinyurl.com/27fmps5

> code). Lots of processing and files. the best pdf processors are
> commercial; i wouldn’t trust wp with producing a nice pdf file. c)the
> problem with WP is that even though pages and posts are hierarchical,
> the infrastructure of wp is certainly focused more towards posts (but
> with publishing, it would be the reverse).

I think for PDF we would use another engine. We would go

xhtml > xml > TeX > PDF

I agree it is post-oriented. What we need is a plug-in to output the whole project as xhtml.

>
> d)The TOC builder would probably be the first plugin to write.

There are already several TOC plug-ins, e.g.

But best is to create the TOC automatically at creation of final output, e.g. PDF.

[...]

• http://www.river-valley.com kaveh

> @Kaveh EE will not go out of business because this CMS have received a
> cult like following among web designers; …

It sounds like a good system, and I very much hope it does not go out of business. But you can’t get much more of a cult following than HyperCard. Who would have thought Steve the dictator would chop that? ;-)

• http://wwwrvt kaveh

Robert

> c)Making an epub file is not as easy as it sounds (even with good xml

I think it can be automated, with good xml code. Here is our filter to convert any document with Elsvier DTD to ePub. It’s a command line code, but it’s free and open so you can modify it as you like:

http://tinyurl.com/27fmps5

> code). Lots of processing and files. the best pdf processors are
> commercial; i wouldn’t trust wp with producing a nice pdf file. c)the
> problem with WP is that even though pages and posts are hierarchical,
> the infrastructure of wp is certainly focused more towards posts (but
> with publishing, it would be the reverse).

I think for PDF we would use another engine. We would go

xhtml > xml > TeX > PDF

I agree it is post-oriented. What we need is a plug-in to output the whole project as xhtml.

>
> d)The TOC builder would probably be the first plugin to write.

There are already several TOC plug-ins, e.g.

But best is to create the TOC automatically at creation of final output, e.g. PDF.

[...]

• http://www.river-valley.com kaveh

Robert

> c)Making an epub file is not as easy as it sounds (even with good xml

I think it is trivial, with good xml code. Here is our open, free filter to convert any document with Elsvier DTD to ePub. It’s a command line code, but it’s free and open so you can modify it as you like:

http://tinyurl.com/27fmps5

> code). Lots of processing and files. the best pdf processors are
> commercial; i wouldn’t trust wp with producing a nice pdf file. c)the
> problem with WP is that even though pages and posts are hierarchical,
> the infrastructure of wp is certainly focused more towards posts (but
> with publishing, it would be the reverse).

I think for PDF we would use another engine. We would go

xhtml > xml > TeX > PDF

I agree it is post-oriented. What we need is a plug-in to output the whole project as xhtml.

>
> d)The TOC builder would probably be the first plugin to write.

There are already several TOC plug-ins, e.g.

But best is to create the TOC automatically at creation of final output, e.g. PDF.

[...]

• http://www.river-valley.com kaveh

Robert

> c)Making an epub file is not as easy as it sounds (even with good xml

I think it is trivial, with good xml code. Here is our open, free filter to convert any document with Elsvier DTD to ePub. It’s a command line code, but it’s free and open so you can modify it as you like:

http://tinyurl.com/27fmps5

> code). Lots of processing and files. the best pdf processors are
> commercial; i wouldn’t trust wp with producing a nice pdf file. c)the
> problem with WP is that even though pages and posts are hierarchical,
> the infrastructure of wp is certainly focused more towards posts (but
> with publishing, it would be the reverse).

I think for PDF we would use another engine. We would go

xhtml > xml > TeX > PDF

I agree it is post-oriented. What we need is a plug-in to output the whole project as xhtml.

>
> d)The TOC builder would probably be the first plugin to write.

There are already several TOC plug-ins, e.g.

But best is to create the TOC automatically at creation of final output, e.g. PDF.

[...]

• http://sillybean.net Stephanie Leary

Laisvunas, I can’t imagine what you’re on about. What difference does it make that WordPress offers one-click theme changes, except that makes it easier to use than EE? You can still write themes that are just as highly customized, as long as you don’t assume that they’ll be useful to someone else or that you’ll get the same results if you do switch themes. Certainly once you start creating custom content types, you must create a very customized theme; otherwise those content types would never be displayed.

EE’s theme system always made me feel like I was being condescended to, as if the developers were sitting on my shoulder saying, “You’re not clever enough to know what you’re doing with themes, so we’re going to make it difficult for you to change them.” WordPress treats me like a grown-up. If I, as the site administrator, am silly enough to give my users a) other themes to choose from and b) permission to switch themes, then I deserve the consequences. But AS THE ADMIN, I should have that ability myself. It makes development infinitely easier. EE doesn’t give it to me.

I didn’t mean to get into a CMS ideological argument here, but you’re simply misinformed. As has been stated several times in this thread, WordPress is no longer just a blogging engine, and has not been for a few years now.

• Laisvunas

> EE’s theme system always made me feel like I was
> being condescended to, as if the developers were
> sitting on my shoulder saying, “You’re not
> clever enough to know what you’re doing with
> themes, so we’re going to make it difficult for
> you to change them.”

ExpressionEngine does not have theme system at all. You cannot install a front-end theme and switch from old theme to a new one by a click of the button. In ExpressionEngine you only can to develop a set of templates. If you need to change a look of the website you need to edit templates.

Theming always means forcing some backend/frontend concepts upon developers. Why? – because developing of a theme is *customizing* of some set of predefined constructs. In expressionEngine you develop your own set of templates instead of customizing anything. This together with very powerful templating engine (i.e abstraction from PHP layer) results into very flexible system. WordPress does not have and cannot have such flexibility because of its architecture. Last time when I looked at WP it did not have even templating system i.e. no abstraction from PHP layer – everything was developed using plain PHP.

• http://tkbr.ccsp.sfu.ca/ John Maxwell

@bowerbird

We created the chapters natively in WordPress, then exported them (via a trivial plugin I wrote which exports the plain post content). The exported XHTML was run through an XSLT conversion to Adobe’s IDML (InDesign’s alternative XML file format), assembled into an InDesign “book” and proofed. Once the content is flowed in this way, the content is updatable without having to re-flow, because it uses the old InCopy linking system. So no, it’s not scraped, nor is it shoveled. That said, no, InDesign isn’t free in any sense of the word (though it does have a truly open file format now).

The point about WordPress is that you could do this from any halfway sensible web CMS. We had done all of this previously with a wiki as well, but it seemed like swimming uphill not to use WordPress, given how much has been put into making it ‘usable’ by writers.

• http://hughmcguire.net Hugh McGuire

@RobertNagle:

“a)are you using wp simply for an editing environment or for presentation as well?”
Could be used for presentation as well. Those worried about security could opt for a different solution.

The principle is: if you have good, structured mark-up from the beginning, you can do whatever you like with the output… which should be good structured mark-up (xml or xhtml or whatever).

“b)what about Live Writer support? As WP becomes more full-featured, I want to offload the editing functions to a desktop app with the ability to save offline.”
I (often) use an offline editor (ecto) for my blogging needs. Again, if you have an open architecture that:
a) can accept good structured markup
b) will output good structured markup …

… then … in theory you can do whatever you like at either end of the process.

The key is having “the book” live on the web (aka in good structured markup) from the start – so that you can do anything you like with it.

“c)Making an epub file is not as easy as it sounds (even with good xml code). Lots of processing and files.”

I have heard differently, but I defer to people who know better than me. I cannot for the life of me imagine why it should be complicated though. epub is XHTM + CSS in a wrapper. It’s a website. So if there are problems with making epub, that’s a problem that will go away soon.

“the best pdf processors are commercial; i wouldn’t trust wp with producing a nice pdf file.”

Again:
Structured markup in –> structured markup out

that means, in theory, you should be able to make a nice PDF file however you like. Indesign. LaTex. Whatever. And wordpress wouldn’t be doing it, your PDF creation engine would.

“c)the problem with WP is that even though pages and posts are hierarchical, the infrastructure of wp is certainly focused more towards posts (but with publishing, it would be the reverse).”
this seems to me to be a solvable porblem – perhaps with a custom display.

again, the key is the structured markup (sorry I sound like a broken record) … which makes displaying however you like theoretically trivial.

“d)The TOC builder would probably be the first plugin to write.”
When can you have it done? ;-)

“e)I would trust wp with handling the editing of text, but not so much with the formatting. The rich text editor and the media inserter would have to be totally redesigned and throw out some features.”
Maybe in an ideal world. Until then, they do a good enough job to get the thing off the ground. Also: when can you have them done by? ;-)

“f)I don’t believe wp has any concept of internal links (i.e., relative links) — in contrast to a cms like Plone. That would be a big problem for producing ebooks.”

structured markup … ;-)

“g)fonts/embeddable fonts/IP issues?”
either:
a) license good fonts
or
b) use free fonts …

a solvable problem.

“h)This may sound funny, but I’d be happy with a plugin or utility that just lets me dump all posts into one clean xhtml file (removing all the miscellaneous crap, like comments, RSS, theme stuff, widgets, etc) and storing the images in a single folder.”
Yep. An excellent start.

“Maybe there can be a way to set criteria (Show all posts with the category ARTBOOK and replace dates with chapter numbers, with 1 being the earliest, 2 being the next earliest, etc). I haven’t checked out how wp exports xml/wxr looks. Maybe all we need is a local tool for writing queries to export stuff from the db that looks innocuous.”
Huzzah! Yes. A good place to start.

Next: what can we do with a nice clean export? Can we do it in WordPress – or do we need something else?

“i)the Rice Connexions project is great (and backed with Plone). They have solved a lot of problems, but they ask people to compose with a builtin xml editor.”

the big advantage of wordpress is that writers are comfortable with it. xml editors are built for programmers…another option would be to have a nice, writer-friendly xml editor … until then….

“Those are my random thoughts.”
thanks!

• Kirk Biglione

@laisvunas I’ve built sites with both EE and WP. A few years ago I might have shared your opinions. Not anymore. You really need to take a fresh look at WP3 when it’s released. Some of your criticism is based on old information.

• bowerbird

kaveh said:
> You came out of nowhere

i’m always amused by the people who don’t know me…

i’ve been making noise about e-books for decades now,
and i do literally mean _decades_, and even if you don’t
think my commentary is intelligent, you’ve gotta admit
that it’s loud, and the thing is, i’ve always _backed_up_
what i said with extensive research and running code…
which is a lot more than i can say for some of you…

if you’re active in e-books, you should know who i am.

if only so you can gain camaraderie with all the people
who hate me.

> Why don’t you give a link to your name so we can
> see what systems you have.

i’m not actively selling anything at the moment, sorry.
you’ll need to ask me if you want to see my systems…
my e-mail address is bowerbird at aol dot com.

you’ll need to carve out a significant chunk of time if
you want to experience the entire tool-chain, however.
if you’d rather have someone else do that work for you,
hugh is now undergoing the process, and i’d expect that
he would be willing to tell people what he thinks of it all.

> I happen to think “priprietary” is a dirty word.
> I have been burnt twice at least.

it seems to have affected you, to the point you can’t even
spell the word correctly. and i sympathize with you, i do.

but it also seems to have affected your ability to reason,
because i’m not defending a “proprietary” approach at all.
as i was clear to point out, my objection is that “open” is
not a magic word that clears the air of all other problems.

> I was one of the first people in UK to work on HyperCard,
> one of the most innovative and beautiful languages.
> I became an expert and put in endless hours.
> Then Apple killed it.

one of the early multimedia authoring-tool program i wrote
– called “breeze” — is, in many ways, a clone of hypercard.
perhaps you should have worked in that particular direction,
namely reproducing software on which you were dependent.

> I have a business built on almost entirely free open software.
> I know that I can plan as far in advance as I want.
> No-one can take that software away from me.

> Sure, but at least the code is there, not in some fat cat’s vault.

you need to keep all the characters here straight…
you’re arguing against proprietary software, but i’m
not the person who suggested a proprietary route…
i agree with you completely that proprietary is bad.
my extension is that “open” is not necessarily good,
not if it’s something that you cannot actually build on.

> But you don’t craft a book directly in WordPress.

that’s kind of my point.

> You manage content and even communications
> between different people. Then you extract pages
> as xhtml, then other systems take over.
> We happen to like xslt and Latex downstream.

and i’ve been saying every part of that process can be
executed better if you build a purpose-focused system,
rather than modify a system built for another purpose…

> That would be good, but it takes a lot of effort.

no more than you are putting in to do it the other way.

and the resultant product will do the job much better…

> WordPress already has a lot of capability that can be
> adapted, and it has a huge user base and is solid as a rock,
> so we can use it till someone builds a tailor made system.

once you make the adaptations, it won’t be “solid as a rock”,
plus it will take you time and energy to make the adaptations,
and the “huge user base” is for another purpose entirely, and
all your work will be thrown away when someone else does
build a tailor-made system. if none of that bothers you, fine.

***

john said:
> So no, it’s not scraped, nor is it shoveled.

actually, i understood the process you went through correctly.
(i’d read it when you brought out the book a few weeks back.)

and i would _most_definitely_ say you scraped and shoveled…
(not that they’re necessarily _bad_ things, it’s one way to do it.
you shouldn’t feel offended by the words i use to describe it.)

a system i see as ideal allows a writer to write in a natural way
– that is, with as few distractions as possible — and then just
click a button to build an .html version (for the web) and a .pdf.
no scraping, no shoveling; no fuss, no muss; just click a button.

***

one problem with modifying an existing tool to a new use is that
you’ll ask the wrong questions and think about the wrong things.
the discussion of c.m.s. systems and “themes” is a good example.

***

another problem is that you end up with some internal discrepancies.
for instance, hugh has a very good consistent focus on _structure_…
as long as the document is structured correctly, you have good power
when it comes to flexing it out into new formats. but then hugh says
“we can use the existing wordpress editor until we build a new one”…

well, yeah, you can, but the problem is that the wordpress editor does
_not_ give you good structured markup. it’s got a presentational bent.
so right off the bat, you’ve lost ground on a very important dimension.

***

but, you know, i don’t really care what you all do, what you work on,
whether you waste your time, or any of that, so do whatever you want,
that’s just fine with me. :+)

-bowerbird

• Keith Fahlgren

WordPress supports editing and authoring content using the Atom Publishing Protocol, and I recall some folks (now dpearted) working on support for Windows Live WriterWP, which I think was released. I never used it, tho.

• http://www.river-valley.com kaveh

I posted a comment earlier about ePub which seems has not been uploaded. Here is our open, free filter to convert any document with Elsvier DTD to ePub. It’s a command line code, but it’s free and open so you can modify it as you like:

http://tinyurl.com/27fmps5

• http://www.river-valley.com Kaveh

Bowerbird said:

> i’m always amused by the people who don’t know me…

I am sorry for not knowing who you are. I am not part of this community really.

Perhaps a web page?

[...]

> you’ll need to carve out a significant chunk of time if
> you want to experience the entire tool-chain, however.

Sorry, too busy with WordPress. ;-)

[...]

> > Sure, but at least the code is there, not in some fat cat’s vault.
>
> you need to keep all the characters here straight…
> you’re arguing against proprietary software, but i’m
> not the person who suggested a proprietary route…
> i agree with you completely that proprietary is bad.
> my extension is that “open” is not necessarily good,

Agree

> not if it’s something that you cannot actually build on.

*What?* Disagree.

[...]

> > That would be good, but it takes a lot of effort.
>
> no more than you are putting in to do it the other way.

Wrong. I had the idea of using WordPress some four weeks ago. It took us perhaps a week to put together a proof of concept. Most of what we needed, e.g. TOC, invoicing, we found within the nearly 10,000 plugins already available. The rest we’ll write ourselves and make those public too. And when we finish the entire system, we’ll also put that up as free and open.

• http://www.river-valley.com Kaveh

Sorry, misread post. Ignore this from above:

——-
> not if it’s something that you cannot actually build on.

*What?* Disagree.
——-

• http://www.imaginaryplanet.net/weblogs/idiotprogrammer/ Robert Nagle

KAVEH: Bowerbird is fairly well known in the ebook community as a techno-gadfly. We have learned to put up with him :).

HUGH: First, I’m not a developer, but just a blogger and technical writer, so I can’t really talk about implementation details.

But it seems that you are outlining a way to take structured HTML content from WordPress and import them into a commercial tool like InDesign. In other words, in your scenario Indesign is still going to be where you do your heavy lifting.

But if you do that, I have a feeling you may still need to do a lot of manually formatting in whatever commercial tool you use. Bibliographies, footnotes, sidebars, indices, etc. Also, if writers are using wordpress to compose things, they may avoid certain structures for which there is no equivalent in the rich text editor.

Also, unless you’re doing all the processing on the server side, management of images and metadata for these images would seem like a challenge.

Perhaps what we need is not some magic tool to create pdfs/ebooks/print books, but a wp theme that approximates the feel of the book as much as possible (while adding some new features to it).
Can anyone recommend some WP themes which are more book-oriented?

I think wordpress is outstanding for composition and collaborative editing. The hard part in writing books is the writing — not the book production; is it really that hard to manually import material created in wordpress into something hardier (like InDesign, MS Word or even docbook)? It’s possible to spend too much time dreaming up the ideal software toolchain and waste too much time in the pursuit of it.

Alternatively, a wordpress-backed site simulates many book features already; perhaps what we need is a way for devices to view blog sites more easily and allows the creator to organize content more easily. The main thing wordpress still links is a method of creating internal links….

One final thought. Given that HTML 5 offers a lot more elements (like footer, header,section, etc), I can’t wait for content management systems that will emerge from it.

• bowerbird

robert doesn’t know me well at all, most certainly not enough
that he should even try to describe me to complete strangers.

even though his description was kinder than what might have
been given by other people, let’s stay away from the personal,
ok?, and stick to the topic, so we don’t drift into ad hominem.

thank you.

-bowerbird

• bowerbird

i googled “blog to book”, and found a half-dozen things…

***

first of all, there’s blurb…

http://www.blurb.com/create/book/blogbook
>
> Your blog. Automatically slurped into a real book.
>
> BookSmart Works with many blogging platforms –
> Blogger, LiveJournal, TypePad, and WordPress.com.
>
> Edit in real time – Allows you to customize and
> edit your book as little or as much as you like.
>
> High quality – Our Hardcover, Dust Jacket, Hardcover,
> ImageWrap and Softcover books all feature professional
> bindings and two choices of paper. Up to 440 pages
> accommodate thousands of blog posts.
>
> Fast turnaround – Orders arrive on your doorstep
> in approximately 7 to 10 business days.
>
> Automatic BookSmart slurp action (watch the video) –
> Imports and maps blog text, images, comments,
> and links into professionally designed page layouts.
>
> Affordable – Prices start at only US $12.95 for a > 40-page softcover, US$22.95 for Hardcover,
> Dust Jacket and US $24.95 for Hardcover, ImageWrap. > The bookmaking software is free or > use PDF to Book worfklow for full ceative control. > > Flexible ordering – Order just one or many copies > of your book. > You have the option to place your book > in Blurb’s bookstore for fans to purchase. *** second of all, there’s blog2print… http://blog2print.sharedbook.com/blogworld/printmyblog/index.html > > YOU PICK > Cover > Images > Number and Order of Posts > Comments > You can edit your book, add/remove comments, > and add or remove pictures. > A 20-page softcover > Blog Book is just$14.95
> Hardcover only $24.95. And extra pages are only 35 cents! > NEW! Get a PDF of your > Blog Book for only$7.95.
> Try It
> I’m on Blogger (e.g. http://blogname.blogspot.com)
> I’m on WordPress (e.g. http://blogname.wordpress.com)

***

third of all, there’s blogbooker…

http://blogbooker.com/
>
> BlogBooker produces a high-quality PDF Blog Book
> from all your blog’s entries and comments.
>
> Archives can be generated from any blog running on
> WordPress, LiveJournal (and derivatives) or Blogger.
>
> The whole process takes about 3-4 minutes,
> depending on the size of your blog.
>
> “BlogBooker: From your Blog to a PDF Book.”
>
> Blogbooker is a FREE service offered thanks to donations.

***

fourth of all, there’s fastpencil…

http://fastpencil.com/
>
> Word processing software is not book writing software.
> It’s designed for creating business documents.
> While you can use Microsoft Word to write a book,
> it is stuffed with functions you don’t need
> (mail merge and drawing tools, anyone?).
>
> Yet all of those toolbars, ribbons and buttons
> still don’t give you the capabilities you really need.
>
> Book writing software should make it easy for you and
> an editor to collaborate on the same document without
> being forced to send email attachments back and forth.
> You should be able to roll back changes
> not just to the last save, but all the way back to
> the moment you started working on your book.
> Because the death of a hard drive shouldn’t mean
> the death of your book,
> you should be able to write your book from any computer.
>
> FastPencil’s Book Writer is
> free online book writing software.
> It makes the writing and collaboration seamless and fun,
> because everything you need is in one place, whether
> you’re writing your book alone or with a dozen colleagues.
> The Book Writer is designed for any text-heavy book,
> be it a novel, a memoir, a business book, a how-to,
> a history book, or a textbook.
>
> Because it was developed explicitly as
> book writing software,
> the Book Writer also encourages good writing habits.
> The table of contents serves as an all-in-one outliner,
> Text formatting tools are uncluttered, and
> they stay right next to the text you’re editing.
> Document formatting is handled
> through sophisticated templates.
> Rather than hassle with style sheets and overrides,
> you can get down to writing, and
> make formatting decisions when you’re ready to publish.
>
> Because the Book Writer is an integral part
> of the FastPencil publishing system, you can
> easily invite colleagues, friends, and family to
> review your work in progress.
> Their notes attach automatically to chapters.
> Co-authors and editors can make changes to
> the same project without fear, because you can
> always compare and roll back changes if necessary.
> It’s the ultimate book collaboration tool.

now, i must say, based on this description alone, that at the
very least, it looks like fastpencil is in touch with the issues.

although i don’t know if their software lives up to this hype,
they seem to have a handle on what can and should be done.

fastpencil.com will also import material from a blog…

http://www.fastpencil.com/projects/blogbook
>
> Select an import option
>
> Enter Blog URL to import directly from your blog
>
> Upload an XML file exported
> from your blog software (How?)
> We currently support XML files
> exported from Blogger and WordPress,
> as well as XML files in the RSS format.

***

fifth of all, there’s pothi.com…

http://blog2book.pothi.com/
> 1. Enter a blog address
> 2. Select Posts
> 3. Customize
> 4. Finish

***

sixth of all, there’s bookemon.com…

http://www.bookemon.com/create-book/Blog+Book
> Transform your blog content into
> a professional, bookstore quality book.
> Preserve and share precious writings
> and thoughts you have with your family and friends.
> Capture and organize your blog pages
> using applications like Word, upload the document
> and your book will be ready in no time.

or maybe not…

looks like they don’t automatically ingest a blog…

***

and here’s your bonus pointer…

matt cutts, from google, recently turned his blog into a book.
he posted about the process. maybe google will take this up…

http://www.mattcutts.com/blog/blog-into-book/

-bowerbird

• http://selfevidenthotdogs.com/ David

Interesting article. But nytimes.com definitely does not run on wordpress.

blogs.nytimes.com does however.

• http://www.river-valley.com Kaveh

Robert said:

> But it seems that you are outlining a way to take structured HTML
> content from WordPress and import them into a commercial tool like
> InDesign. In other words, in your scenario Indesign is still going to be
> where you do your heavy lifting.
>
> But if you do that, I have a feeling you may still need to do a lot of
> manually formatting in whatever commercial tool you use. Bibliographies,
> footnotes, sidebars, indices, etc. Also, if writers are using wordpress
> to compose things, they may avoid certain structures for which there is
> no equivalent in the rich text editor.
>
> Also, unless you’re doing all the processing on the server side,
> management of images and metadata for these images would seem like a
> challenge.

Robert, these are all challenges. Footnotes, citations links to references, etc. Really we need a better authoring engine to deal with all these, and that is what we are going to be thinking about. It is certainly not clear that we can solve all these, but worth trying I feel.

• Steve

The thing I’ve been looking for is a way to put book chapters on a blog in their “natural” order. I.e. chapter one should remain at the top of the page and subsequent chapters should work their way downward, not upward, as they are added. I haven’t found a way to do this. Is there a WordPress plug-in that would work?

-Steve

P.S. I want to be able to do this without having to generate artificial reversed posting dates.

• http://www.river-valley.com Kaveh

Steve, we use “Custom Post Order”:

http://wordpress.org/extend/plugins/custom-post-order/

• Steve

Kaveh,

Thanks much for the pointer. I really thought my question would come up empty, as I tried to google the question some months back and found nothing usable. But this plug-in looks like it does exactly what I need.

Thanks again,
-Steve

• http://hughmcguire.net Hugh McGuire

@robertnagle:

RN: “But it seems that you are outlining a way to take structured HTML content from WordPress and import them into a commercial tool like InDesign. In other words, in your scenario Indesign is still going to be where you do your heavy lifting.”

HM: What you do out the other end – ie in the “book formatting” phase – is up to you, and an open plugin architecture means you can have multiple solutions for different kinds of usecases. John Maxwell and I have hotly debated just this issue.

He chose to plug into InDesign, because the people he wanted to serve are commercial book designers, comfortable working in Indesign.

I would prefer to plug into an automated layout engine, something like a server-side LaTex engine, so that the print book design can come out the other side automatically – with no pros or expensive software involved. Because, I want to serve people who don’t neccesarily have access to commercial book designers, or their expensive software.

However, I’m agnostic about which path to start on – each one has its uses – and again, as long as the output markup is good, and as long as the plugin architecture is flexible, then multiple solutions can and should be available.

RN: “But if you do that, I have a feeling you may still need to do a lot of manually formatting in whatever commercial tool you use.
Bibliographies, footnotes, sidebars, indices, etc. Also, if writers are using wordpress to compose things, they may avoid certain structures for which there is no equivalent in the rich text editor.”

HM: Yes.

RN: Also, unless you’re doing all the processing on the server side, management of images and metadata for these images would seem like a challenge.

HM: challenge? ;-) Yes, this isn’t going to be an ideal solution for all cases, certainly. Let’s get something that is pretty good in certain situations (eg. Book of Mpub) and then build on that.

RN: “Perhaps what we need is not some magic tool to create pdfs/ebooks/print books, but a wp theme that approximates the feel of the book as much as possible (while adding some new features to it).
Can anyone recommend some WP themes which are more book-oriented?”

HM: See, eg:
http://demo.jimbarraud.com/manifest/

But that is not nearly enough in my mind. A fine start… but I want something more.

RN: I think wordpress is outstanding for composition and collaborative editing. The hard part in writing books is the writing — not the book production;

HM: in theory that is true; in practice, few writers/editors can “produce a book” on their own. A tool like the one descirbed above will let them.

RN: is it really that hard to manually import material created in wordpress into something hardier (like InDesign, MS Word or even docbook)?

HM:
step one: make sure the output is optimized to do just that.
step two: automate the export/import

RN: It’s possible to spend too much time dreaming up the ideal software toolchain and waste too much time in the pursuit of it.

HM: Lot’s of things are possible.

RN: Alternatively, a wordpress-backed site simulates many book features already; perhaps what we need is a way for devices to view blog sites more easily and allows the creator to organize content more easily.

HM: the nice thing about doing this as an open source project, on an open plugin architecture platform like wordpress, is that those inspired to solve particular problems can do so. So, chances are that the problems most in need of solving are the problems people will wish to solve. That’s the hope anyway.

RN: The main thing wordpress still links is a method of creating internal links….

HM: perhaps.

RN: One final thought. Given that HTML 5 offers a lot more elements (like footer, header,section, etc), I can’t wait for content management systems that will emerge from it.

HM: yes! and my expectation is that WordPress will be a likely candidate to evolve quickly to incorporate those elements.

• bowerbird

hugh said:
> few writers/editors can “produce a book” on their own

i disagree.

perhaps you could list the reasons why you say this, and
we can “go through them” (i.e., destroy them) one by one.

there are a lot of people who want to stall out the revolution by
telling individuals that it is difficult to produce a book. it’s not.

-bowerbird

• http://hughmcguire.net Hugh McGuire

@robert nagle:

further thought: if wordpress templates are anything to go on, I expect we’ll see a proliferation of book-like themes when we get this project off the ground.

• bowerbird

hugh said:
> http://demo.jimbarraud.com/manifest/

ok, let’s take a look at that…

i scraped the juice out of each of the individual entries,
and combined ‘em into the single file i expect you want.

you can see the results of that merge here:
> http://z-m-l.com/misc/manifest-all.html

does that look the way you want? act the way you want?

you would also need to create a table of contents, and
link it to each of the “chapters” (or “sections” or “posts”
or whatever you want to call the individual entries), but
that’s easily doable too.

doing this was very easy. so what more do you want?

i expect you know that generating a .pdf from (x)html is
quite a slippery beast (which is why john used indesign)…

-bowerbird

• bowerbird

maybe hugh is out at the barbeque.
(do canadians celebrate memorial day?)

anyway, while we’re all waiting for him to
tell us if my “merged manifest” looks ok,
here are some other things you can view:

basically, i took our little sample “book”
through a series of edits to mold it into
something closer to what we might want.

(in the process, i replaced the lorem ipsum
with some text of my own; i prefer real text.)

y’all are welcome to stick that series of files
in your nearest revision-control system, so as
to trace the evolution of the edits if you like.

i don’t think many real-world authors give a crap
about their revision trail enough to even save a
reliable string of backups, let alone deal with the
hassles of managing a revision-control system, but
hey, if any of you is game, go ahead and go for it…

***

i said:
> i scraped the juice out of each of the individual entries,
> and combined ‘em into the single file i expect you want.

you know, i shouldn’t have minimized the work i did,
because that’s exactly what we’re talking about here.

“scraping the juice” from the blog entries wasn’t difficult,
but it did take some time and energy; it wasn’t cost-free.

and it’s work that shouldn’t really need to be done at all.

let me explain.

hugh pointed to that template because it’s clutter-free.
at least it looks clutter free on the _outside_. but if you
look at the markup _inside_ it, you’ll see it’s full of clutter.

the divs run about 5 or 6 layers deep, and the tagnames
are not just plumb crazy, they are also quite numerous…
go ahead, take a look at the .css, and tell me i am wrong.
in various ways, it brings to mind some family reunion in
one of those southern towns with too much inbreeding…

so, in order for me to “scrape the juice” out of the entries,
i had to pull the actual _content_ out of the unnecessary
markup junk, which — since they are often intertwined –
is not all that easy… the complexity of the markup and
the .css also mean that sometimes it’s difficult to know
which parts of the markup are valuable and which are junk.

what you people really need to do — assuming anyone here
is actually intent on hacking wordpress, which i am starting
to doubt, since i don’t detect any motion in that direction –
is to design a template which is free of any markup clutter…
so it _looks_ clean, but is also easy to squeeze juice out of.
of course, then you need to tell authors to use that template.

there are a few other problems with the “merged manifest”.
one big one is the header-structure has a few anomalies in it.
(i think we can agree an h1 header should be the book title,
and that titles on the individual posts should be h2 headers.)

anyway, now i’m gonna go out and join hugh at the barbeque…

-bowerbird

• http://www.imaginaryplanet.net/weblogs/idiotprogrammer Robert Nagle

Following up again.

Thanks for the thoughts. Custom Post Order and Digress.It are great plugins. Can’t wait to play with these things. I actually am a big fan of web-based books. The problem with dynamic content is that they can’t be archived/packaged very easily for reading 20 years from now.

Bowerbird, many projects/companies have tried to automate the blog-to-book transformation (blurb being probably the most prominent). None are particularly successful, if only because they are trying to impose a book structure on content organized by chronology. Drupal has a lot more possibility, but it hasn’t really been viewed as a platform for individual authors — not yet at least.

• bowerbird

robert said:
> Bowerbird, many projects/companies have tried to
> automate the blog-to-book transformation
> (blurb being probably the most prominent).

well, um, yeah, i just pointed to a bunch of them…

> None are particularly successful,

i don’t know that. i’ve never seen their output. have you?

i’d guess they put out a product that is acceptable
to customers, or else they’d stop doing it altogether.

> if only because they are trying to
> impose a book structure on
> content organized by chronology.

seems to me that any attempt to use a blogging platform
to bake books is going to be subject to that factor, not?

> Drupal has a lot more possibility, but
> it hasn’t really been viewed as
> a platform for individual authors
> — not yet at least.

i suppose you guys will look at every tool in the toolbox
to see if you can hack it to be the tool that you want…

in the meantime, i’ll just build the darn thing from scratch.

eventually, you’ll give up in frustration, if you ever even
actually get started on anything, and i’ll have a system built.

perhaps you haven’t noticed that this thread has morphed
already, into one where i’m _doing_ it, and y’all are merely
_talking_ about doing it. or perhaps you don’t understand
that there even _is_ a difference between those two things…

-bowerbird

• bowerbird

ok, the missing post has now show up.
it’s the one dated may 31st at 2:22pm.

it starts with this:
> maybe hugh is out at the barbeque.

it tells of updating the “manifest” merge.

the one thing i forgot to mention there
is that the manifest source-code is in
xhtml 1.0 transitional, whereas .epub
requires xhtml 1.1, if i understand it…
i don’t know if that presents a problem.

anyway, looking forward to any feedback,
from hugh or from anyone else, as i am
pursing this matter, as should be obvious.

-bowerbird

• Laisvunas

Hi,

Just found PHP class which according to the author “provides a pure PHP solution to build ebooks in the EPub format.”

You can find it here.

This class is already used at least in one application.

The solution might be of high quality since it was nominated for PHP innovation award 2010.

• Laisvunas

Hi,

Just found PHP class which according to the author “provides a pure PHP solution to build ebooks in the EPub format.”

You can find it here.

This class is already used at least in one application.

The solution might be of high quality since it was nominated for PHP innovation award 2010.

• bowerbird

no feedback yet, from anyone…

anyway, here’s a new file-comparison product,
which might be the type of thing you’d need to
make revision-control appeal to the masses…

i haven’t reviewed it yet, so i don’t know if it’s
good, but these apps are always fun to look at.

-bowerbird

• bowerbird

never mind fastpencil.com.

i tried their system. it’s ok. not great. but not _too_ bad.

however, their pricing models are simply ridiculous.

$500 is the lowest, and top-tier goes over$1200.

-bowerbird

• Sigi Jottkandt

Excited to hear about your change in thinking about open vs proprietary, Hugh. Like Shana, I think Book Oven could be a great resource for open access scholarly communications. You might also be interested in another open source web-based book publishing platform that had its debut last week – Open Monograph Press, by the Public Knowledge Project. http://pkp.sfu.ca/

• bowerbird

i’m still waiting for someone — anyone! — to start moving on this.

-bowerbird

• bowerbird

john, i’ve taken a good look at your mpub output,
and i would have a few suggestions for you, if you
are interested in hearing them.

can i get permission to repost the content as i’d do it?

-bowerbird

• bowerbird

ok, folks, i’ve programmed a web-based book-creation tool,
a “first-draft” of the kind that was described here, and i would
like to use the content from “the book of mpub” as a demo…

since this is _research_, i believe i could mount the text even
without permission, but i’d prefer to ask for permission first.

so, could i please have permission?

-bowerbird

• bowerbird

ok, i e-mailed john, and he informed me that
there’s a creative commons license on the book,
so i don’t need explicit permission for any use
which is noncommercial. thanks, contributors!

i hope they’ll accept some constructive criticism
as payment in return.

so let’s get the worst of it over first, ok?

they all point to the wrong place, usually a site
that’s mentioned in the chapter on pivot tables.

it’s kinda hard to believe that this huge flaw
was missed in your production quality-control,
and that no users out there have reported it…

but there you go. that’s enough for today.
more to follow tomorrow, and after that…

and yeah, i’ll be mounting my version of this,
so all you people out there who would like to
take _me_ down a notch will get your chance.

but it appears that most of the twitter kids
don’t have the attention span necessary to
follow a long thread like this one. they’re all
off running after some other shiny topic now.

oh, look, annotations! oh, look, indexing!

-bowerbird

• bowerbird

don’t make me come over to twitter and
remind you to approve comments here…

because i _will_, you know. just saying…

-bowerbird

• bowerbird

yet another general constructive criticism of the mpub book

is that there seems to be some major recto/verso confusion.

the .pdf shows the odd-numbered pages on the _left_ side,

which is not how things are done in the western hemisphere.

at first, i thought it was just a blip, caused by what _could_

have been a minor snafu with the verso of the cover-page…

if that verso was just deleted from the .pdf (maybe because

somebody figured that it was an “unnecessary” blank page)

then that would throw the whole book off, recto/verso wise.

but the problem appears to be much deeper than that, since

the odd-numbered pages clearly have their binding margin

on their right, and the even-numbered have it on their left,

which is a solid indication they’d been _designed_ that way.

so, whatever the cause, it’s clearly a big mistake.

***

on the bright side, i notice that “logical page-numbers” were

used in the .pdf. that’s the best solution to a big problem…

(if you don’t know the problem, you should go and look it up,

because i’m not going to explain it here.)

however, it’s not a _good_ solution. first, some users might

not have the preferences set to _see_ “logical page-numbers”,

so they will get the “regular” .pdf page-numbers, regardless.

second, some .pdf viewer-programs, including _preview_ on

the mac platform, do not recognize “logical page-numbers”.

so again, these users get the “regular” .pdf page-numbers…

and, due to the ongoing security problems with adobe apps,

more and more people are turning to these alternate viewers.

so we essentially have a situation where some of the users get

one set of page-numbers, and the rest of the users get another.

the confusion is simply not worth the hassle of dealing with it…

so i recommend that you just start numbering your pages with

the first page in your .pdf, so the .pdf page-numbers _match_

the ones on your printed pages, so everyone can agree on them.

-bowerbird

• Dave Hitchens

john, i’ve taken a good look at your mpub output,
and i would have a few suggestions for you, if you
are interested in self publishing them.

can i get permission to repost the content as i’d do it?

• John Sinclair

I’ve wanted to bend WordPress to the task of publishing short stories as far back as it’s first incarnation — b2/cafelog. But WP never has been, and still isn’t, a really good option. Bless its little pea-picking heart. I, too, am one of those who love it, for it does what it does better than any other program.

WordPress cannot allow users to reliably edit a short story, which, by definition, is three- to ten-thousand words. A chapter or a scholarly paper are probably about the same size — up to a hundred-thousand bytes. That size document, when edited in a textarea field may be arbitrarily truncated. 32k, or 64k, or some such quantity of bytes is the capacity of most browsers. Neither WordPress, PHP, nor MySQL are the problem in entering, storing, or editing large blocks of text. The problem is with the various browsers we use and the number of bytes transported in a POST type transfer.

Personally, I like to write offline on my favorite text editor and FTP the end product to my website as a .txt or .html file. It would be nice to have an interface to get such a text file into the WordPress SQL tables. A desktop application, if you will. So WordPress could then serve only as the database front end and as a display engine.

Another limp biscuit is WP being unalterably date-centric even if you plug-in something to manage presentational re-ordering. You would then have to add a plug-in to generate an index or table of contents, and then add another plug-in for PDF, ePub, or whatever.

No. WordPress as it’s been molded is intended to catalog frequent, short bursts of creativity. Static pages are but secondary asides. And cohesive volumes aren’t yet even tertiary thoughts in the code-base.

We need a new application — BookPress, or BookBaker, or some such. Something built perhaps with components from WP, with the look and feel of WordPress, but we shouldn’t settle for a half-baked kludge of WordPress.