Preparing for the realtime web

How the shift to realtime will affect the web (and why info overload is overblown).

The dominance of static web pages — and their accompanying user expectations and analytics — is drawing to a close. Taking over: the links, notes, and updates that make up the realtime web. Ted Roden, author of O’Reilly’s upcoming “Building the Realtime User Experience” and a creative technologist at the New York Times, discusses the realtime web’s impact in the following Q&A.

Web 2.0 Expo San FranciscoMac Slocum: Have we shifted from a website-centric model to a user-centric model?

Ted Roden: It used to be that a user sat down at a computer and checked Yahoo and and whatever else. Now, users get their Yahoo updates via Twitter and pushed into Facebook, wherever they are. So rather than a user going to to a specific website, websites are coming to where the users already are.

MS: Has push technology finally found its footing with realtime applications?

TR: I think so. It’s not that push technology was a solution looking for a problem, it was only a partial solution. But now broadband has a wide penetration, browsers are much more stable and resource friendly, servers are cheap or free, and the development of realtime applications has gotten drastically easier. Using push technology without all of those other bits in place was a lot more painful for everybody involved and as a result, designers and programmers stuck with standard web apps. Now we can take a lot of that for granted and think like desktop application designers.

MS: What skill sets do developers need to take advantage of the realtime web?

TR: Python and Java are certainly strong contenders at this point. Tornado is really exciting to me. But one thing is totally clear: the world of a dominant HTTP server is gone. Developers need to get comfortable with the fact that there’s going to be a huge shakeup in the server world. Instead of big fat Apache servers, we’ll see a lot more apps running with built-in HTTP servers that are specific to the application, or at least the type of application.

MS: How about publishers: what do they need to consider as they’re working with realtime apps?

TR: There is a lot to do on the content side. Creators will have to figure out if pushing each new message is important and which messages are important. They’ll also need to know if content has to change as it’s distributed in different formats. Big publishers like the New York Times have long understood that a headline that works in print doesn’t work on the web. And they’ve also figured out that regular web headlines don’t work on the mobile web. At the Times, we’ve recently started retooling the headlines as they get pushed out to Twitter and Facebook because the content needs to fit those platforms specifically.

MS: How can users organize all this incoming information?

TR: I think a lot of users will start relying on things like the “hide” button and “unfollow.” I think more and more apps are going to ship with a “mute for a bit” button, too. So manual filtering will be a big part of this. Apps and websites will start to get smarter about what they show you as well. Facebook already tries to do this with varying degrees of success.

I tend to think information overload is a lot less of a problem than most people do. Essentially, since not long after the printing press was invented, there has been more information being created than we could consume. But we’ve never really had a problem with it. When people go into a library, they don’t have panic attacks because of all the information. They know they can see it all, but it’s not all for them. I look at my Twitter stream the same way.

MS: Are we at a point yet where realtime analytics are in place?

TR: We’re a long way from having these analytics in place. There are some great services, like Chartbeat, that are getting us there. But we have work to do.

It isn’t completely clear what’s important in realtime analytics. For starters, we need to know a) an article is blowing up and b) why? That is absolutely crucial information to have. But we’re going to have to start mixing realtime analytics with A/B testing and all kinds of other things to really understand.

It isn’t a problem limited to realtime analytics, either. If you want to track when a user comes from a certain URL and ends up checking out with his shopping cart, we can do that easily. But we have a tough time figuring out who that person is. Beyond that, we’re going to have to start tracking the amount of influence the users have. We’ll also need to continue tracking the conversation about a piece of content even if the conversation happens in far off corners of the web, far from our websites and Facebook pages.

Note: This interview was condensed and edited.

tags: , , , ,
  • Alex Tolley

    1. Why has “realtime” become synonymous with “push”, when they are only weakly related at best and realtime already has a meaning in computing?

    2. The vision that Mr. Roden espouses is so narrow, it is boring. All this bandwidth and CPU cycles just to try to probe the user … to sell more stuff? Oh joy.

  • Military Strategist

    @Alex: Realtime probing of the “user” (sensor on the edge) is essential in order to operate a “real-time” feedback-control system. This is Network Centric Systems thinking developed by DoD thinktanks applied to the public sector, where C2 (Command and Control) is executed by system designers and operators, and the public function merely as sensors & task-masters for the system on the edge. This is why Mr. Roden’s “narrative” is so narrow, as he’s holding to military thinktank directives. Lookup the JCOS Joint Vision docs, or Google Art Cebrowski. The strategies are well documented, and I think most here will see the parallels.

  • Alex Tolley

    @Military Strategist – thanks for the heads up. I read up the PDf on JCOS: Joint Vision 2010, although I didn’t see any mention of “realtime” capability in the document, although the concept the military was pushing was clear.

    However what I was objecting to was the use of “realtime” which is used in the interview to mean pushing data to the user as it is created and possibly trying to do some analytics on that. Realtime, as one example, in the financial data feed world means something very different.

    I accept that “realtime” could mean different things to different people, but maybe it shouldn’t.

    Finally, we are possibly emerging into a networked world where all the nodes can rapidly communicate with each other at various levels and integrating that data flow into something that might have really interesting emergent properties and value, and what do we get … a narrow commercial view that seems to be more about exploiting the human sensor than doing anything that useful.

  • rektide

    #junto is a realtime video muc (multi user chat). one thing we discussed last night was capturing and building meta-data from the real time conversation. info overblow wasnt really addressed in the article, i felt, but, for example, Kate or Annodex could be used to annotate the video stream at the time frame when Tim Berners Lee or Oreilly joins.

  • Ted Roden

    @Alex I totally agree with you. The phrase realtime has another meaning. When I started writing Building the Realtime User Experience, I was concerned most people would think it was about programming interrupts on the space shuttle. But I don’t get to pick the names of trends as they emerge, I just get asked about them. It’s the fault of our tiny little English language. I’m certainly not the first to point out how many different meanings of the word “Free” there are.

    But I disagree that this stuff is boring. I certainly don’t advocate using it only for commercial purposes. In my book I spend a lot of time showing all of the uses for this type of tech, but end up focusing mainly on how to use this to push forward the way developers think about the web. When you stop thinking about a rendered web page and start thinking about it as an application, this stuff gets really interesting.

    Thanks for your thoughts!

  • Max Williams

    I couldn’t agree more, this is a really exciting time for the web.

    This is why we recently launched our realtime messaging service ( to aid developers in embracing these developments.

    It’s great to see the traction this space is getting.

  • Rick Bullotta

    I have to echo Alex’s sentiments. Coming from a world where “real-time” is microseconds and milliseconds, and the amount and frequency of data dwarfs the total of all social networks/activity streams by a couple orders of magnitude, I think we need to look at the “real-time web” in the context of not only the human vector but also the “Web of Things” or “Internet of Things”.

    When billions or trillions of devices start generating activity streams, data, and content, the necessary infrastructure, tools, analytics, visualizations, and of course, the potential value, is so completely different than where we’re at today.