The Locker Project: data for the people

The combined efforts of the Locker Project and TeleHash could help people be at the center of their data. Plus: Bloom presents Fizz, a new data app.

Singly, a new company that made an appearance at the Strata Conference Startup showcase, exists to provide oxygen and commercial support to the open source Locker Project, and the new protocol TeleHash.

With some wonderful serendipity I met Singly on my first night at Strata. The next day, I talked in depth to Jeremie Miller and Simon Murtha-Smith, two of the three Singly co-founders (see later in this post). I also had the opportunity to ask Tim O’Reilly and Roger Magoulas for some of their thoughts on the significance of this project (see below for their comments).

It was a real “pinch myself in case I need to wake up from a dream” experience for me, to stumble across Jeremie Miller with Simon Murtha-Smith sitting behind a handwritten sign demoing Singly at Strata (see my pic opening this post). As Marshall Kirkpatrick noted at ReadWriteWeb:

Jeremie Miller is a revered figure among developers, best known for building XMPP, the open source protocol that powers most of the Instant Messaging apps in the world. Now Miller has raised funds and is building a team that will develop software aimed directly at the future of the web.

Singly, by giving people the ability to do things with their own data, has the potential to change our world. And, as Kirkpatrick notes, this won’t be the first time Jeremie has done that.

I was drawn over to the Singly table when an awesome app they were demonstrating caught my eye. Fizz, an application from Bloom, was running on a locker with data aggregated from three different places

Fizz is an intriguing early manifestation of capabilities never seen before on the web. It provides the ability for us to control, aggregate, share and play with our own data streams, and bring together the bits and pieces of our digital selves scattered about the web (for more about Bloom and Singly, see Tim O’Reilly’s comments below). The picture below is my Fizz. The large circles represent people and the small circles represent their status updates. Bloom explained the functionality in a company blog post:

Clicking a circle will reveal its contents. Typing in the search box will highlight matching statuses. This is an early preview of our work and we’ll be adding more features in the next few weeks. We’d love to hear your feedback and suggestions.

If you are not already familiar with the Bloom team — Ben Cerveny, Tom Carden, and Jesper Sparre Andersen — go directly to their about page and you will understand why the match of Bloom and the Locker Project is a cause for great delight.

The Locker Project: A whole new way to connect from the protocol up

Singly, the Locker Project, and TeleHash take on and deliver an elegant and open solution to some of the holy grails of the next generation of networked communications. I have written on, and been nibbling at the edges of some of these grails in various projects myself for quite a while now. A glance at the monster mash of my pre-Strata post will give you an idea of how important I think Singly is.

That previous post raised the question of how to invert the search pyramid and to transform search into a social, democratic act. But if you are really interested in social search, I suggest staying keyed into what Singly is doing with the Locker Project.

One of Singly’s three founders, Simon Murtha-Smith, was building a company called Introspectr, a social aggregator and search product. Singly’s other founder Jason Cavnar was working on another similar project. And they came together as Singly because social aggregation and search is a very hard problem for one company to solve. They realized that the basic infrastructure needs to be open source and built on an open protocol.

To me, what is so important about the Locker Project is that it is built on a new open protocol, TeleHash. And having the Singly team focused on supplying tools and the trust/security layer for the Locker Project will mean that developers will have the whole stack.

I asked Miller to explain the relationship between TeleHash, the Locker Project and Singly.

Tish Shute: What is TeleHash?

Jeremie Miller: It’s a peer-to-peer protocol to move bits of data for applications around. Not file sharing, but it’s for actual applications to find each other and connect. So if you had an app and I had an app, whenever we’re running that app on our devices, we can actually find those other devices from each other and then connect. Our applications can connect and do something. TeleHash is actually what has led to the Locker project itself.

Tish Shute: So TeleHash led to the Locker Project and the Locker Project led to Singly?

Jeremie Miller: Singly is a company that is sponsoring the open source Locker Project.

TeleHash is a protocol that lets the lockers connect with each other and share things. The locker is like all of your data. It’s sort of like a digital person.

Singly interview
Left to right: Jeremie Miller, Jason Cavnar, Simon Murtha-Smith. I took the pic above of all three founders being interviewed by Marshall Kirkpatrick of ReadWriteWeb. I think we will look back on this moment and say it was an inflection point for the web. At least I tweeted that!

Tish Shute: A locker stores bits and pieces of your digital self?

Jeremie Miller: Yes. So TeleHash lets the lockers directly peer-to-peer connect with each other and share things. Singly, as a company, is going to be hosting lockers first and foremost. But the Locker Project is an open source project. You can have a locker in your machine or you can install it wherever you want.

Tish Shute: Will Singly provide the trust layer and hosting?

Jeremie Miller: Yeah. Singly is a company that will host lockers, as well as when people build applications that run inside your lockers or use your data, you need to be able to trust them. Maybe it’s initially social data and you don’t care that much about, but once you add browsing history, your health data, your running logs, or sleeping information, it’s important to be careful about what you’re running inside your locker and sharing.

So Singly will also look at the applications that are available that you can install and actually run them and look at what data they access. It will be able to come back and either certify or vouch for them.

I hope in the long-run, as this grows and builds, that power users may actually be able to buy a small device that they can plug into their home network and that would be their locker. Wouldn’t that be cool? This little hard drive that you plug in.

Tish Shute: Architecturally, are TeleHash and the Locker Project related to your work on XMPP?

Jeremie Miller: XMPP in Jabber was designed for the specific purpose of instant messaging, but it was still a federated model in that you still had to go through a central point. It was designed with that in mind — for the communication path to be routed through somewhere. Where I’ve evolved is that I’m fascinated with truly distributed protocols that are completely decentralized so that things are going peer-to-peer instead of going through any server.

Peer-to-peer has gotten a pretty bad rap over the last 10 years because of file sharing, but the potential for it is awesome. There’s so many really good things that can be done with peer-to-peer, and it hasn’t gotten used much.

But the other side of the peer-to-peer thing that I think is critically important, look at the explosion of the computing devices around an individual — both in the home and on our person. I look at my home network router and I’ve got 30 devices in my house on Wi-Fi. That’s a lot of devices.

But right now, to work with those devices I’m almost always going through a server somewhere, or through a data center somewhere. That’s ridiculous.

Tish Shute: So we need a peer-to-peer network just to manage our own devices?

Jeremie Miller: A peer-to-peer network, yes. My phone should be talking straight to my computer, or to the iPad, or to the washing machine, or to the refrigerator. The applications in my TV should all be talking peer-to-peer. And it should be easy to do that. It shouldn’t be that the only way you can do that is to go through a data center somewhere.

Note: Updates to the Locker Project will be posted through @lockerproject and GitHub

The Locker Project is not just “one more rebel army trying to undo these big data aggregations”

I discussed the impact TeleHash, the Locker Project and Singly might have on social network incumbents with Roger Magoulas and Tim O’Reilly. Both had insightful comments.

Magoulas pointed out:

I think Singly has Facebook-like aspects, but I think a better description is an app platform that integrates your personal and social network data — including data from Facebook.

Singly is likely to have challenges with some of their data sources, particularly if it gains traction with users. I like the app platform business model, although they face risks getting critical mass and app developer attention. I also like how they plan on using open source connectors to keep up with changing social network platforms.

Jeremie [Miller] has credibility with the open source community and is likely to find cooperating developers. The team seems to bring complementary strengths to the project and you can tell they all work well together.

Tim O’Reilly elaborated on the potential of this platform to bring something new to the ecosystem. Our interview follows:

Tish Shute: Will the Locker Project be able to break the lock of big sites, like Facebook, controlling everyone’s data? Sometimes I feel we are stuck in the era of Zyngification, where you have to do what Zynga did and leverage the system in order to gain traction or do anything with social data.

Tim O’Reilly: I don’t think breaking the Facebook lock is the objective of the Locker Project. The value of Facebook is having your data there with other people’s data. What Singly may be able to do is give people better tools for managing their data. If you can take the data from various sites and manage it yourself, then you can potentially make better decisions about what you’re going to allow and not allow. Right now, the interfaces on a lot of these sites make it difficult to understand the implications of making your data available.

If this is done right, it will create a marketplace where people will build interfaces that provide more control over personal data. People will still want to put data on sites for the same reason you put money in the bank: it’s more valuable when it’s combined with other people’s money.

To conceive of this project as one more rebel army trying to undo these big data aggregations is just the wrong way to frame it.

Tish Shute: Framing the question the way you just did — that this is not just one more rebel army — might mean the stage at Strata will be filled with new startups next year. That’s what I thought when I found out what the Locker Project and Singly are about: that we’re about to see an explosion of creativity with personal and social data.

Tim O’Reilly: The tools we have now are pretty primitive. If we get a better set of tools, I think we’ll see a lot of innovation. Some of those startups might be acquired by Facebook or Google, but if those smaller companies give people better visibility and control over their data, that’s a good thing.

Tish Shute: I loved the marriage between Singly and Bloom [mentioned above]. It’s interesting because Ben Cerveny and the Bloom team haven’t really talked a lot about Bloom yet. I gather Bloom is moving toward consumer-facing work with data?

Tim O’Reilly: People think of data visualization as output, and the insight that I think Ben has had with Bloom is that data visualization will become a means of input and control.

I’ve started to feel that visualization as a way of making sense of complex data is kind of a dead-end. What you really want to do is build feedback loops where people can actually figure something out. Being able to manipulate data in real-time is an important shift. Data visualizations would then become interfaces rather than reports.

This post was edited and condensed. A longer version, featuring additional interviews and analysis, is available at UgoTrade.

tags: , , ,

Get the O’Reilly Data Newsletter

Stay informed. Receive weekly insight from industry insiders.

  • http://about.me/steveardire Steve Ardire

    Terrific power packed post on a very exciting project with many ramifications

  • http://about.me/tikva Tikva Morowati

    You can learn more about TeleHash from Jeremie’s Ignite NYC talk given in June 2010: http://www.youtube.com/ignitenyc#p/c/B50F0621101ED1EE/8/FRP86lDCvsw

  • Brett

    In a RL encounter with someone we keep our own personal information and we decide what to share with each individual. Generally, we don’t refer them to a third party for that information.

    It’s peer-to-peer: person-to-person.

    We definitely need to get away from this centralised, federated approach to our data storage, sharing, and retrieval.

    But, why the need for someone to host our lockers? Is it just a technology constraint that will eventually go away? There’s mention of eventually developing a locker box that could plug into your home network. Why wait – it’s called a computer, people already have them plugged into their home networks. One just needs an application to manage it. Anyone who wants to use DropBox downloads the desktop app.

    I think the basic use-case is valid, even long overdue, but I think the implementation needs some work.

  • http://www.tonido.com dyno

    Tonido project addresses this problem as well – Where your personal data is scattered across multiple public online properties.

    One can write Tonido apps in C++ OR php to pull data from different social media resources and finally can be managed by the end user on his/her own device or a hosted VPS.

    http://www.tonido.com/support/Developer:10_Reasons