As part of the Semantic Edge panel tomorrow at the Web 2.0 Summit, Nova Spivack of Radar Networks plans to unveil the first application built on their semantic web platform, twine, a new kind of personal and group information manager. I’ve only seen a demo, and haven’t had a chance to play with it hands-on or load in my own documents, but if it delivers what Nova promises, it could be revolutionary.
Underlying twine is Radar’s semantic engine, trained to do what is called entity extraction from documents. Put in plain language, the semantic engine auto-tags each document, turning each entity into what looks like a web link as well as a tag in the sidebar. Type a note in twine, and it picks out all of the people, places, companies, books, and other types of information contained in the note, separating them out by type. For example, here is an email dropped into twine:
(Note: screen shots shown in this entry were taken during a demo of a twine alpha delivered over the web via glance.net. Ignore the glance.net frame when looking at the images. In addition, some features may have changed since the demo.)
OK. So what, you say? The magic doesn’t happen until you — or a group of people — have collected a large set of documents. Now, you can use the tags associated with any given document to pivot through everything else your collection, or twine, contains about that tag.
So, for example, our team here at the O’Reilly Radar might create a twine on a topic we’re interested in, say sensors, and any time we met with a company doing interesting work with sensors, we’d add our notes into that twine, along with associated email, address book entries, images, web links, and so on. This pile of documents would have relevant tags extracted, which would then allow us to navigate through it. In addition, twine pulls in additional information: give it a bookmark, and it brings in a thumbnail of the web site; give it the name of the book, it brings in a cover and associated metadata from Amazon; and so on.
The key point is that because each entity in any of the documents becomes a meaningful tag, that extracted meaning becomes a semantic layer tying all of the documents together. What’s more, twine has its own built-in semantic taxonomy, based on concepts mined from wikipedia, and Nova claims it can make connections between documents using tags and concepts that are not actually in the documents themselves.
You can then explore the various data that’s been added into the twine by the people sharing it. As Nova said in some notes he sent me by email, ” It’s like a social network for sharing, organizing and finding knowledge….Twine helps people organize, share and find their information more intelligently. Twine learns from users and from folksonomies in order to help them better over time.”
Technorati Tags: freebase, radarnetworks, SemanticWeb, web2.0, web20, web2summit
As Nova mentioned, twine also has elements of social networking: you can share any item, or a whole twine (a group of related items) with other people. And there is the by-now-obligatory news feed:
Knowledge management is certainly a thorny problem. We all have vast collections of data, usually in various silos: our email, our del.icio.us bookmarks, our flickr photos, our address book. Navigating among related items is hard. And when you have a group of people working on a shared project, it becomes even harder. Who knows what? Where is it? This is the knot that Radar Networks hopes to untangle.
Here are some notes on the process of how Twine works from a briefing document Nova sent me:
1. Create Twines
o Create a Twine for yourself or for any group, team or community. You get a private personal one by default. It takes seconds to create one for any group.
o “A Twine” is a place for your knowledge. It’s the next step beyond a file server, wiki, personal home page, or database.
o Twines can be private and personal, private for groups, or for public groups and communities.
o All roles and permissions can be controlled by the creator, and they support moderation.
2. Semantic Social Networking
o Users can form opt-in relationships with individuals and groups to permit communication and sharing in Twine. This is similar to Facebook, except that user-profiles and social relationships are semantically defined and extensible, and accessible via open-standards (RDF and SPARQL). We plan to enable users to extend these as well.
3. Bring in all your stuff
o Pull in all your information from anywhere your Twine(s).
o You can pull information in live off the Web as you surf, or via import, or via API’s
o Emails, bookmarks, RSS/Atom, documents, photos, videos, amazon, contact records, data records, twitter, Digg, other applications, etc.).
o It’s extensible and open — the community can add support for new data sources, or even custom data sources.
o Once your information comes in we turn it into semantic web content (objects of various kinds according to our ontology, and/or other ontologies that can be loaded in, or that we users can actually create themselves).
o Twine will eventually synchronize with all your stuff, across all locations and devices. We’re starting to work on that — there’s plenty to do there.
4. Author and Edit Rich Extensible Open Content
o Twine enables users to author information much like they do in wikis, blogs or databases. There are many types of information that can be added (people, contact records, notes, organizations, etc.)
o Twine supports collaborative editing, like a wiki, according to permissions, down to the object/page level.
o In the near future we will start enabling users to define their own types of content as well (like Freebase, but a different approach).
5. Enrichment and Learning
o Mining. Twine mines content to detect metadata and tags and other descriptive information that may already be present.
o Semantic Tagging. Twine runs NLP [Natural Language Processing] on the content to detect entities like people, places, organizations, products, events, concepts, etc. These become semantic tags. Tags in twine are semantic objects that link to other broader, narrower or related tags, and can also link to ontological definitions.
o User-Tagging. Users also participate by helping to tag the data too — but they don’t have to do as much work. We find 80% of the tags for them automatically and suggest others.
o Auto-Classification. Twine automatically categorizes content, based on machine learning against the wikipeida. There are approx. 300,000 community-generated taxonomic categories in the wikipedia. This is an example of the next generation of collective intelligence — software learning from the collective intelligence and behavior of millions of people.
o Learning. Twine is designed to learn from the community of users in aggregate. As they use Twine, the service begins to learn new concepts and tags, discover new connections and form new relationships automatically.
It’s difficult to see, based on the demo, whether or not they’ve succeeded at this ambitious undertaking. I think a lot of this is still the vision, with many features not yet available in the alpha to be unveiled tomorrow.
But more than that, I’m going to withhold judgment till I can get my hands on the service. Until the system is populated with a lot of data — far more than shows up in the demos — we won’t know whether we’ve spun a smooth twine, or a gnarly knot. But I’ll look forward to trying. I’m seeing a number of startups trying to work this same problem. None has yet gone live. But I’m confident that eventually someone will make some headway, and I’ll be excited if twine gets there first.
Oh, and one more thing. Nova says “Data that put in Twine will be fully exportable. We don’t want to lock anyone in. The freer we can make your data the better for everyone. The fact that Twine turns all your data into RDF means you can reuse it in anything in the future.” Note the tense of that promise, though: “will be,” not “is.” Still, twine.com sounds like it will be well worth a close look.