Wed

Aug 2
2006

Tim O'Reilly

Tim O'Reilly

Programming Language Trends

In his presentation at OSCON, Roger Magoulas, the director of O'Reilly Research, provided some interesting graphs based on our trends data mart. Here's the 3-year programming language market share trend based on computer book sales:

Programming Language market share trend in computer books

I wrote yesterday about the rise of Ruby and Javascript, driven by the move towards Web 2.0 applications. Also worthy of note in these graphs is the long, slow decline of Java and C/C++, and the continuing rise in market share of C#. You can see how Ruby's sharp ascent follows the introduction of Rails, and that PHP's fortunes reversed before book sales showed that web developers in search of rapid development languages moved over to RoR (and Microsoft's ASP.Net suite of technologies.)

Also worth noting on this chart is that book sales tend to spike even before the release of commercial languages, as the vendors work with publishers to get books out by the release date, while books on open source projects tend to trail the release date. On the one hand, you could interpret this as meaning that open source is slower to market, but I think it says the opposite: open source projects can move fast, and catch publishers by surprise. Few publishers expected the quick uptake of Java AJAX and Ruby on Rails.


tags:   | comments: 44   | Sphere It
submit:

 
Previous  |  Next

Subscribe to Comments on this Entry:

0 TrackBacks

TrackBack URL for this entry: http://radar.oreilly.com/mt/mt-tb.cgi/8407

Comments: 44

adamsj [08.02.06 06:24 AM]

Few publishers expected the quick uptake of Java and Ruby on Rails.

You mean Javascript, right?


The trendline I find most interesting is the one that isn't like any other one, in at least two senses: SQL. It's not a programming language like any of the others, and it's got a slow, steady trend upward.


What makes this particularly interesting is that various BI vendors will tell you that writing SQL by hand is obsolete, since their products will generate it for you. That's an exaggeration with some truth in it--I never want to write a simple reporting rollup again--and a big lie underneath it, as there is always platform-specific optimization to do even for the SQL that the tools will generate. More importantly, they don't really generate everything you want, just the more routine, repeatable stuff.


Anyway, you'd expect SQL books to trend downward, if that it true, but it isn't, so it's no.


I don't suppose there's any data on SQL dialect, is there?

Tom Welsh [08.02.06 06:32 AM]

Strikes me that the methodology - although it's the only one easily open to you - is suspect. Suppose people are buying lots of books on C#; does that really correlate with high usage of C#? Personally, if I had some people writing an application for me, I would be happier if their first move was NOT to rush out and buy books about it. I would prefer them to know the language pretty well already.

So, while there will be some correlation, I wonder if the high C# book sales might simply reflect the fact that a lot of developers are trying to learn it, in the expectation that it will be useful sometime in the future. And maybe declining Java book sales imply, to some extent, that there are a lot of Java developers out there who already know the language pretty well.

Lastly, on your measure a language that has frequent major changes or extensions will bulk larger than one that stays pretty much the same. I bet Ada book sales have been higher than usual recently (e.g. 20 rather than 12).

Donald Smith [08.02.06 06:33 AM]

Sorry, I know it's staring me in the face somewhere, but where is COBOL and Fortran et al on this list? How was this particular set of languages chosen for comparison?

- Don

Dr Nic [08.02.06 06:40 AM]

I think this graph is a wonderful future indicator of the skill pool that will be available, and an immediate indicator of the desired languages these skill developers want to work with.

One consequence may be that those developers staying in "declining" languages may find themselves in greater short-medium term demand. There are less Java developers but a consistent/growing amount of work for them to do. The amount of money a Ruby developer might make might be less (say per hour) as there are more developers per available project (even though both numbers are probably increasing).

adamsj [08.02.06 06:44 AM]

Speaking of fast writing, when I said:

Anyway, you'd expect SQL books to trend downward, if that it true, but it isn't, so it's no.

I meant to say:
Anyway, you'd expect SQL books to trend downward, if that were true, but it isn't, so it's not.

Anonymous [08.02.06 07:52 AM]

This is an interesting signal of technology trend. How does the data relate to the number and availability of new titles for a given language? For example, I would expect Ruby to be on the rise in book sales as it is a new language. PHP in contrast has been around for a while, so it may be that the content of a newly written book is not sufficiently different from previous books to justify the purchase (for someone who has purchased previously). In addition there are already many books in circulation that may be shared.
In support of your point about OS and commercial releases, I would think much of that is to do with a commercial organisation having large promotional groups and budgets to do this work.
Is there any data for actual usage of the languages. Also, how does this relate to the availability and quality of free "tutorial" resources online. Would a user of an open source language be more likely to look for information online and free? Are commercial languages more likely to be used in large organisations who may fund the purchase of books?

JohnMc [08.02.06 07:52 AM]

The other issue is book sales mapping to language complexity. I would expect a lot more java books to be sold based on the level of difficulty of learning the setup/tear down routines required for it vs say a python or ruby language. I mean like heck, I can read doc strings in any python module and pretty much understand the meaning pretty quickly without a lot of 'skill up' beyond the basics. In something like Java I have to understand pretty deep, about 40% of the classes to get anything done.

The other issue with this methodology is book type. What % are language reference vs application/framework? Language reference sales would tell me number of people attempting to understand a language. Framework numbers would tell me the number of people actually using the language day to day. Heck 'XXXXXX for Dummies' sells a hell of a lot of titles. But how many of those actually then write some app in some paid fashion?

That IS the question is it not?

Perrin Harkins [08.02.06 07:55 AM]

It's interesting to compare this to actual
job listings for some of these languages.

adamsj [08.02.06 08:03 AM]

It's always interesting to add a few terms for perspective.

Tim O'Reilly [08.02.06 12:16 PM]

Perrin -- the indeed job data is very hard to read because of the scale. All of these programming languages are basically rounding errors when all you do is show them as a percentage of all jobs.

What you want to do is to show the rate of change. I did this last week in the posting Ruby Book Sales Pass Perl. I showed some treemaps that show that ruby is the only language with an increase in the number of job postings.

I'll post more detailed information later this week, or early next.

pj [08.02.06 02:39 PM]

The fundamental here is that people are buying less Java books because lots more people know it and don't need to buy books. It doesn't mean Java is in decline.

It'd be interesting to see absolute figures and %s.

Srini [08.02.06 05:33 PM]

Three weaknesses to this measure:

There's also an inherent weakness to this - free online documentation. The best tutorial on languages like python and php are on the web itself; many people skip the books altogether.

Another weakness is that some languages might be easier and/or more intuitive than others, so fewer books are bought for that language.

Yet another weakness is that some languages are often second or third languages learned. Seasoned developers may or may not buy more books.

Donnie Berkholz [08.02.06 05:57 PM]

Are the trends for C and C++ really tied that closely together, or is it deceptive to graph them that way?

Search Engine WEB ♣ [08.02.06 08:13 PM]

Putting AJAX between the first and second quarter of 2005 is open to debate........


Febuary 8 2005 was the debut of the COINING of the term via a publication......


The concept had been was introduced before that time when Microsoft developed XML/http and was growing as Google and Yahoo and others began incorporating that strategy,.......


AJAX as a concept is like Charles Darwin being credited with inventing evolution theory

Zingus J. Rinkle [08.03.06 08:01 AM]

I don't really scope in that diagram any sharp decline in php interest after the Ruby On Rails release; I spot a decreas in java and c++ interest though. That many people switching from c++ to ruby on rails + javascript? The mind boggles. Can hardly wait for the first ATC system based on those two.

Tim O'Reilly [08.03.06 08:19 AM]

Srini -- Obviously, computer book sales are not an absolute indicator. But they are meaningful. For example, I began my open source activism in 1997 after the buyer at Borders told me that Programming Perl was one of their top 100 books in any category for all of 1996 -- and I realized that no one in the computer trade press was talking about it. Then I realized that all my bestsellers were equally "off the radar."

As to your three factors: I agree that online documentation is a huge factor, but it affects almost all languages equally. Do you really think that the online doc available for PHP is that much better than the online doc for say, Java? Sun and Microsoft have huge online developer education resources.

And yes, novelty is certainly a factor in people buying books. But remember, someone buying Learning Perl or Learning Python or Pragmatic Ruby is in exactly the same position: I want to learn a new language. So which are they choosing?

Tim O'Reilly [08.03.06 08:23 AM]

Donnie -- We made a decision long ago to combine C and C++ because there were so many books that covered both. That might have been a mistake. We should probably go back and disambiguate the two.

Search Engine Web: while the technologies behind AJAX did not exist prior to the coining of the term, that coining (and the introduction of the first significant AJAX apps, like GMail and Google Maps) were what defined the category, and made people rush out to start buying books.

Zingus -- I noted that the decline in PHP had started before the RoR release. I actually think it might have been impacted a bit more strongly by the release of ASP.Net 2.0, and the switch of some big hosting providers as a result.

Bob [08.03.06 08:26 AM]

For your C# and VB books above, are you including only those books with those language names in the titles? Every ASP.NET or ADO.NET book uses on or the other language (or both), but if I understand your methodology correctly won't show up above.

This seems to undercount the C# and VB number considerably. No?

Fern quick [08.03.06 09:34 AM]

There's also an inherent weakness to this - free online documentation. The best tutorial on languages like python and php are on the web itself; many people skip the books altogether.

I agree to this.

adamsj [08.03.06 11:09 AM]

But remember, someone buying Learning Perl or Learning Python or Pragmatic Ruby is in exactly the same position: I want to learn a new language. So which are they choosing?
Tim,

Have you broken out the figures on books like that? Learning and beginner books, that is? (I realize there's a certain subjectivity in deciding just what goes into that category.)

Milan Hawkins [08.03.06 01:02 PM]

I find that Google Trends is an interesting way of showing the popularity of current trends in technology.

kevin [08.03.06 06:06 PM]

How much of these sales are imposed as required texts for students?

If you go to the job trends link and add C and C++ (separately), C graphs higher on the scale.

Mike [08.03.06 06:30 PM]

Where does Coldfusion fit into this list? I have someone who wants to use Coldfusion for a huge project and I cannot find statistics on how much it is utilized in the industry.

Anonymous [08.03.06 08:09 PM]

DO you only make the research in the O'Reilly?
If so, I think it only shows that O'Reilly publish no good book in c# and no good book in Ajax

Tim O'Reilly [08.03.06 09:34 PM]

John Adams (adamsj) -- re SQL. I've said about Web 2.0, "data is the intel inside." Hal Varian said, "SQL is the new HTML."

Kevin -- the impact of required texts is relatively small. We do see seasonality from text adoption, but it's less pronounced in programming. To the extent that it happens, it primarily benefits Java and C. A lot of the required texts are sold through channels that don't necessarily report to bookscan.

Mike -- ColdFusion would be showing as a flatline at the bottom if we were to graph it. I showed some visualizations that included ColdFusion in my entry a week or two ago, State of the Computer Book Market, Part 2: Category Winners and Losers. If you look at the web development treemap in that posting, you'll see that ColdFusion is a tiny fraction (about 6%) of the size of PHP or ASP.Net (which are neck and neck in the book marketplace), and shrinking.

Anonymous -- No, as explained in the article linked to at the beginning of the post, this data is reported by Nielsen Bookscan, and represents all publishers. Follow the links in the posting to get more details. And as a matter of fact, O'Reilly is the #1 publisher on Ajax, and the #2 publisher on C# behind Microsoft Press. In the Ajax category, we currently have the top three books on Ajax (Head Rush Ajax, Ajax Design Patterns and Ajax Hacks, and if you count our distribution partners (which include Manning and Pragmatic)

Anonymous [08.04.06 02:59 AM]

I don't mean to disrespect Steve Oullaine or the other authors who have written C++ books for O'Reilly, but with the excellent Addison Wesley C++ In-Depth Series around, I wouldn't expect O'Reilly to sell very many books on the topic.

Daniel James [08.04.06 04:06 AM]

According to the post that you link to from 2005, sales bottomed out in 2005 - which is also the point at which C/C++ books seem to peak in your graph. Similarly Java falls a faster rate after this point.

This seems to be not due to a sudden decline in interest in those two languages, but because of an increase in interest in other languages.

I expect that C/C++ book sales are mostly independent of Ruby book sales. The two languages are designed for very different uses. Lumping together the book sales of programming languages in this way seems very misleading.

Tim O'Reilly [08.04.06 09:04 AM]

Anonymous -- as noted above (and repeatedly in all my blog posts about this data), these trends are not based on O'Reilly sales only. They are based on point of sale data for all computer books, via Nielsen Bookscan. We do the categorization and analysis, but they collect the data directly from bookstores.

Daniel -- just because sales of Ruby and C are independent doesn't mean that it's not interesting to put them on the same graph. People eat bananas at breakfast or for a snack, and apples and oranges just for a snack, or apples sometimes in pies and oranges in salads, but that doesn't mean we shouldn't graph them together if we're trying to figure out the most widely-consumed fruit.

tim finin [08.05.06 08:02 AM]

I wonder if the book sales included text books? CS students probablby learn C, C++ and Java in their undergrad programs and don't have much need for additional books beyond their texts for these languages. At UMBC the texts we use for CS1 and CS2 are not even standard textbooks -- the publisher produces a custom ones for us drawing on material from several other books.

Pete Poorman [08.16.06 01:37 PM]

This is a larger version of my favorite technology popularity metric -- shelf-feet at the Border's in Palo Alto.

Sean Allen [09.08.06 07:13 AM]

Ahem... to the person who said Ruby is a new language. It is far from being new, there is just a lot of interest in it related to Rails.

Dan Weber [09.12.06 07:24 PM]

> Also worthy of note in these graphs is the long, slow decline of Java and C/C++

I think someone is seeing what they want to see, not what is there. C/C++ was at the same place from 2003-Q1 to 2005-Q4. That's 3/4 of the graph.

In theory I would show up in the data as "someone switching from C++ to Ruby-on-Rails + Javascript" because I've bought more books on the latter than the former in the past few years. But that's because I'm way more comfortable with C++ than Ruby and Javascript, so the latter is what I'm practicing. Not to mention that "switching from C++ to Javascript" is insanity. They are different tools for different jobs.

Andrew Norris [09.13.06 12:37 PM]

I agree with the poster(s) above who noted that this is primarily useful as a leading indicator -- this is what people are learning more about, so it's what they're buying books about. A massive number of people know a ton about Java and use it every day, but it's not as high on the list of things people really want to learn more about as it used to be. Obviously, there are still many, many people learning about Java -- through school, through their job, etc. But the trend is clear.

By contrast to book sales, I expect job listings are more of a trailing indicator, because people getting their feet wet with, say, Ruby aren't necessarily advertising for new hires with Ruby experience. After you have a project underway (or maybe even deployed), you're more likely to look for new hires.

P.S. To echo a previous post, I'd love to see Tim elaborate on how something like "ASP.NET 2.0 Cookbook" is accounted for. It clearly is associated with C# and VB, but doesn't have those languages in its title. Is there any way to get a breakdown with C# + VB + .Net grouped together to see the popularity of .Net in general?

Tim O'Reilly [09.13.06 12:49 PM]

I believe ASP.Net Cookbook counts towards both C# and VB measurements. This is a multi-dimensional data mart. We have a category rollup, in which a C# book counts as a C# book, and an ASP.Net book counts as an ASP.Net book, and a language rollup, in which both a C# book and an ASP.Net (or a design patterns book, or ... that uses C#) counts as a C# book.

There is also an overall platform rollup, that is even broader than the language rollup. I'll try to do that post sometime soon.

As to book sales being something of a leading indicator, I'd tend to say it's coincident, as you say, with learning, which tends to lead deployment, where job postings trail deployment decisions.

However, we're getting some huge insight out of the difference in job postings between startups and more established companies. And that does turn job data into more of a leading indicator... More on that soon.

Anonymous [09.14.06 08:18 PM]

I find that Google Trends is an interesting way of showing the popularity of current trends in technology.

How would you determine these languages on Google Trends for sure as Java can also mean coffee, while Ruby can mean the gemstone or a girl's name (sometimes, even a boy's name). If you append language or programming language, you will be cutting down the search trend criteria considerably, but getting a better accuracy:

Java language, Perl language, PHP language, Ruby language

Anyway, I wanted to highlight the foibles of inputting Ruby, Java and such into Google trends and thinking you are getting any accurate picture. There are a lot of us women and men out there who love Ruby gemstones and occasionally search for them. :)

Next, in my opinion, this is simply untrue:

There's also an inherent weakness to this - free online documentation. The best tutorial on languages like python and php are on the web itself; many people skip the books altogether.

I've looked at tutorials on all languages online as I tried to learn each, and the best tutorials I've ever seen are by Why for Ruby, hands down. I haven't seen how the PHP and Python ones are so awesome to entail not buying books for them. In actuality, given PHP has so many tutorials out there, many are equally as crummy as good (and the very depth of the ones available means it is equally as hard to separate the good from the bad in and of itself), not to mention many aren't frequently even updated to account for the (new) PHP versions coming out.

Another point you are missing with the statement of looking at online tutorials is that the people who purchase books usually aren't the same ones who use online tutorials and vice versa. I have a friend who hates to read long texts and only reads via online documentation. He would never pay for any programming book outside what he has to buy for college classes. On the other hand, I love books and often buy them whenever I have an interest in a language even when I have many sites bookmarked for a language. Most people who buy programming books do so despite online documentation because there is an idea by such book buyers that written books have more reference material and depth than online resources do. True or not, that is the sentiment, yet someone who never or rarely buys books is not going to understand that sentiment because that person simply don't feel that way about books, so the individual would not even bother buying books anyway whether the language is new, old or comatose.

Maybe, it would be nice to have a trend comparison for online tutorial site visits for the 100 most popular programming tutorial sites per google searches to the book sales chart so we can see if these awesome PHP and Python tutorials are at the top of the list or not for visits.

Stephen [10.13.06 09:51 AM]

The only thing it accurately measures is what the latest hype is, and what those hype people are buying. You'd almost for sure find the same numbers when dvd players came out and compared to vcr sales and their movies etc.

And that's not to say that one of these hype languages(ruby) couldn't translate into something. But the metrics aren't very useful.

Esteban Pacheco [10.26.06 08:44 AM]

Interesting to see is that this graph doesn't reflect a similar comparison available at http://www.tiobe.com/ that one based on Language popularity based on hits on the 3 major search engines.

Books are no longer a good indicator, On-demand printing, blogging, and forums are nowdays the fast and best way to adquire knowledge on a language.

A clear example is Delphi, it is a very popular language, but few books are publish by year (3,4) based on the TIOBE ranking, the distance in popularity between C# and Delphi is of less than 1%. Being Java and C the kings of the hill.

Tony Dolan [10.27.06 06:40 AM]

Is Coldfusion included with Java in these trends?
Its use seemed to decline sharply with the arrival of PHP but seems to have rebounded as developers have got used to changeover to the java platform and the launch of the MX7 Server a year or so ago.

Tim O'Reilly [10.27.06 09:46 AM]

Tony -- ColdFusion is not included with Java. In the book market, it's too small to notice -- about 1600 units per quarter is now the total CF book market. It's down 49% over the quarter a year ago.

Tim O'Reilly [10.27.06 09:54 AM]

Esteban -- I agree with you that books are not the sole indicator, but they are still a good one. We like to look at multiple indicators -- in the future, when I do one of these writeups, I'll cover data from other sources besides books.

As to Tiobe, saying it is based on "hits on the 3 major search engines" is somewhat inaccurate. What they say is that "The ratings are based on the world-wide availability of skilled engineers, courses and third party vendors." (as found in the search engines.) So it may well depend on what you search for. Because they don't disclose exactly what it is they are searching for, or how recent the web listings are, it's hard to know what Tiobe is measuring. In our job data as well as our book data, C# is well ahead of Delphi.

lLERoLa [01.30.07 11:09 AM]

Ïðèâåò
Çàõîäèòå â àñþ, âñå îáñóäèì : 227-665-282 , ïèøèòå íà ïî÷òó kt222@mail.ru îòâå÷ó íà âñå Âàøè âîïðîñû.
Ñ íàìè ðàáîòàòü âûãîäíî,ó íàñ ÑÀÌÛÅ ÄÅÌÎÊÐÀÒÈ×ÍÛÅ ÖÅÍÛ.
www.doski-inet.com
Áóäåì ðàäû îêàçàòüñÿ Âàì ïîëåçíûìè.

Les [04.13.07 08:28 AM]

Assuming that there are roughly as many pros as cons of using books sales as a trend indicator, this method seems pretty accurate.

Patrick Muir [06.22.07 01:53 AM]

A major exhibition of work by French-American sculptor Louise Bourgeois is to be held at Tate Modern...

Anonymous [09.10.07 04:40 AM]

What do you think about the trend if speed is the major issue?

Post A Comment:

 (please be patient, comments may take awhile to post)






Type the characters you see in the picture above.