- Real Time Text Taskforce — standardising live typing ala EtherPad and Google Wave, for accessibility reasons.
- NoSQL Required Reading — papers and presentations to get up to speed in the theory and practice of scalable key-value data stores. (via Hacker News)
- It’s Official, data.gov 2.0 is Coming — pointer to the design and philosophy document for the next iteration of data.gov. Interesting to see so much activity on US open government happening now: open government directive and progress report were released, along with a request for ideas on open access to publicly-funded science research.
- Breakdancing Robot — we live in the future, and it is good. (via @hollowaynz)
"real time" entries
Real Time Text, NoSQL Reading List, New data.gov, and a Breakdancing Robot
After the recent Web 2.0 Expo NY–a sprawling, week-long conference and exhibition–I ducked into the Morgan Library to catch “A Woman’s Wit: Jane Austen’s Life and Legacy.” A one-room show about an 18th century novelist seemed like the perfect antidote to a week of tech talk in the
Death Star Javits Center. As I’d hoped, the Morgan focuses on a handful of objects from Austen’s life, and the commentary is thoughtful. I was surprised, though, to find myself thinking that had Twitter been around in Austen’s time (1775-1817), she would likely have been a fan.
As the web increasingly becomes real-time, marketers and publishers need analytic tools that can produce real-time reports. As an example, the basic task of calculating the number of unique users is typically done in batch mode (e.g. daily) and in many cases using a random sample from relevant log files. If unique user counts can be accurately computed in real-time, publishers and marketers can mount A/B tests or referral analysis to dynamically adjust their campaigns.
This interview covers three “Big Shifts” that have dominated 2009 (1) The move to the real-time web, (2) the move from the information web to the Social Web and (3) the rise of mobile. Since John co-chairs Deloitte’s Center for the Edge I wanted to get his take on each in terms of its impact on larger organizations. This first video covers the Real-Time Web.
Some organizations create their own real-time analysis tools, while others turn to specialized solutions. In a previous post, I highlighted SQL-based real-time analytic tools that can handle large amounts of data. I noted that other big data management systems such as MPP databases and MapReduce/Hadoop were too batch-oriented to deliver analysis in near real-time. At least for MapReduce/Hadoop systems things may have changed slightly. A group of researchers from UC Berkeley and Yahoo recently modified MapReduce to allow for pipelining between operators.
- How Twitter Works in Theory (Kevin Marks) — very nice summary about the conceptual properties of Twitter that let it work. Both Google and Twitter have little boxes for you to type into, but on Google you’re looking for information, and expecting a machine response, whereas on Twitter you’re declaring an emotion and expecting a human response. This is what leads to unintentionally ironic newspaper columns bemoaning public banality, because they miss that while you don’t care what random strangers feel about their lunch, you do if its your friend on holiday in Pompeii.
- Army To Test Wiki-Style Changes to The 7 Manuals — In early July the Army will conduct a 90-day online test using seven existing manuals that every soldier, from private to general officer, will have the opportunity to read and modify in a “wiki”-style environment. (via timoreilly on Twitter)
- Open Data Standards Don’t Apply To The Military — It’s that last particular point that should be the most disturbing to the administration. Apparently all geospatial data being developed and utilized by the USAFA would be unusable without a sole software vendor. This causes concern over broader interoperability with other agencies and organizations, access to important national information, and archivability and retrievability. Expose of the single-source “standard” vendor lockin in US military geosoftware and geodata. (via johnmscott on Twitter)
The emergence of sensors as sources of Big Data highlights the need for real-time analytic tools. Popular web apps like Twitter, Facebook, and blogs are also faced with having to analyze (mostly unstructured) data in near real-time. But as Truviso founder and UC Berkeley CS Professor Michael Franklin recently noted, there are mountains of structured data generated by web apps that lend themselves to real-time analysis.