John Hagel on The Social Web

Subscribe to this video podcast via iTunes. Or, you may download the file.

I am releasing my conversation with John Hagel in three segments. In the first segment we discussed the real-time web. Here we discuss the move from the information web to the Social Web.

John makes the point that the rise of the Social Web feels “a bit like Back to the Future” for people who have a long history with the Internet. In the early days the Internet functioned to link people - scientists, researchers etc. The advent of the World Wide Web saw the Internet functioning more as a publishing platform. Now, with the Social Web, we are back full circle to a network that connects people together. When you connect people to people (as opposed to just brokering information) you are able to surface valuable tacit knowledge that is difficult to express in documents.

Four short links: 2 October 2009

Social Media Parasites, Open Government Data, Prime Numbers, Amazon Image Abuse

1. I'm Tired of Your Analogue Attitude -- hilarious animated clip about social media gurus, made using xtranormal. (via trib on twitter)
2. Three Laws of Open Government Data -- 1. If it can’t be spidered or indexed, it doesn’t exist; 2. If it isn’t available in open and machine readable format, it can’t engage; 3. If a legal framework doesn’t allow it to be repurposed, it doesn’t empower. (also see slide deck)
3. Structure and Randomness in the Prime Numbers -- paper about some of the fun mathematics around prime numbers. (via Hacker News)
4. Abusing Amazon Images -- decoding and doing fun things with the Amazon images API. The cool thing (if you want to generate unlikely Amazon images) is that you're not limited to one use of any of these commands. You can have multiple discounts, multiple shadows, multiple bullets, generating images that Amazon would never have on its site. However, every additional command you add generates another 10% to the image dimensions, adding white space around the image. And that 10% compounds; add a lot of bullets, and you'll find that you have a small image in a large blank space. (You can use the CR command to cut away the excess, however.) Note also that the commands are interpreted in order, which can have an impact on what overlaps what.

Four short links: 1 October 2009

Objectivity Be Gone, Public Screens, Lobbying Patterns, DIY Africa

1. The End of Objectivity, Web2.0 Version -- Our behaviour as journalists is now measurable. And measurability gives the lie to the pretence that journalists behave like scientists, impartially observing the petri dish of society. (via Pia Waugh)
2. Screens in Context -- ideas for the video screens spring up in place of billboards. Whilst the advertising industry has one of the longest histories of trying to understand interaction, it’s a very different set of tools that digitalness brings; ones that designers at the coal face of web and mobile encounter every day. Everything can be considered in context, be timely, reactive, and data-driven. I’m going to try to outline some dimensions to think about, with some incredibly quick, simple, off the cuff dumb ideas [...] The technology to achieve some of these may be over and above what is possible now, but the biggest step - installing powered, networked computers in the real world - is already being taken by advertising media companies.
3. Interactive Network Map of Lobbying Patterns Around Key Senators in Health Care Reform -- fascinating visualization of political activity, via timoreilly on Twitter)
4. The Doers Club -- How DIY design gave a teenager from Malawi electricity, and can help transform Africa.

Four short links: 23 September 2009

Video Art, Synthetic Biology Futures, Crowdsourced Personality, and an 1890s Startup

1. Projections (YouTube) -- the incredible video projection onto an old English manor house by Kiwi Foo Camp alums The Dark Room.
Where Will Synthetic Biology Lead Us? (New Yorker) -- a thoughtful article about the possibilities and cautions of synthetic biology. . “A house pet is a domesticated parasite,” he noted. “ It is evolved to have an interaction with human beings. Same thing with corn”—a crop that didn’t exist until we created it. “Same thing is going to start happening with energy,” he went on. “We are going to start domesticating bacteria to process stuff inside enclosed reactors to produce energy in a far more clean and efficient manner. This is just the beginning stage of being able to program life.”
2. Business Cards and Crowdsourced Personality Assessments -- we scanned images of a person’s business card and asked crowdsourced workers from the Amazon Mechanical Turk channel to write five kind words about the person based on what they saw. I like the idea of being able to crowdsource a quick impartial aesthetic judgement about a design.
3. When Sears Was a Startup (Pete Warden) -- one of the first catalogues from Sears (1897) inspires comparisons to Amazon and other web startups. On a mission with a new business model. They can't stop talking about how they're cutting out the middle men who've been gouging their customers, with pages devoted to messianic rants against the monopolies trying to put them out of business. They contrast their order fulfillment process (dozens of clerks dealing with tens of thousands of orders a day) with the inefficient country stores full of assistants being paid to idly wait for customers, explaining how they can offer such low prices despite the shipping.

Four short links: 1 September 2009

Social Investigative Journalism, Mozilla Service, Gov Data, Video Fun

1. Help Me Investigate -- find other people who want to investigate the same things you do ("on which streets in my town are the most parking tickets issued?", "why is there a giant unused TV screen in the downtown of this city?", "how much does this city council spend on PR?"), work together to resolve it, and leave a record of the answer for others. It's a different angle on MySociety's What Do They Know.
2. Mozilla Service Week -- We believe the Internet should make life better. Join us the week of September 14-21, 2009, as we take action to make a difference in our communities, our world, our Web. (via MySociety)
3. Open Government Data: Starting to Judge Results -- mall, tangible, steps that turn published government data into cost savings, measurable service improvements, or other concrete goods will "punch above their weight" : not only are they valuable in their own right, but they help favorably disposed civic servants make the case internally for more transparency and disclosure. Beyond aiming for perfection and thinking about the long run, the volunteer community would benefit from seeking low hanging fruit that will prove the concept of open government data and justify further investment.
4. Three Frames -- small fun. I love that there are still small fun things to do. (via pleaseenjoy on Twitter)

John Adams on Fixing Twitter: Improving the Performance and Scalability of the World's Most Popular Micro-blogging Site

Twitter is suffering outages today as they fend off a Denial of Service attack, and so I thought it would be helpful to post John Adams’ exceptional Velocity session about Operations at Twitter.

Good luck today John & team… I know it’s going to be a long day!

Update: Apparently Facebook & Livejournal have had similar attacks today. Rich Miller from Data Center Knowledge reminds us that this is just the latest in a series of major attacks.

Four short links: 5 August 2009

Rebooting Britain, Revealing Errors, Reproducing Generators, Netflix Culture

1. Reboot Britain Video Archive -- video from the talks at Reboot Britain are online. The event also produced a essay set (PDF), CC-licensed. (via Paul Reynolds)
2. Revealing Errors -- Benjamin Mako Hill blog using computer errors as starting points for understanding how computers control the world around us. (via Dan Meyer)
3. New Microbe Strain Makes More Electricity, Faster -- University of Amherst researchers made current-generating bacteria work harder to live, and in five months had a strain that made an 8x larger current.
4. Netflix Culture -- readable slide deck which talks about the Netflix company culture. It's hard to read it and not nod in full agreement. (via joshua on Delicious)

Four short links: 19 June 2009

Cute Math, Fast Slo-Mo, Open Source HVAC, xkcd Hack

1. Inside-Out Multiplication Table -- very cool way to view the patterns of factors. Math is beauty with subscripts.
2. High-Speed Camera -- capture 100 frames at up to 1M frames/second. The sample videos, of a bullet liquefying on impact and a shotgun string boiling past, are stunning. The Makezine high-speed photography kit is the cheap amateur version.
3. Open Source Energy Management for Commercial Buildings -- open source project to enable interoperable applications for integrated Building Automation Systems (BAS). From NovusEdge. I wonder how they're planning to spread their open source and use it to disrupt. (via earth2tech and timoreilly on Twitter)
4. xkcd Knapsack Solution -- for those of you who like literal Python geeking with your comics. Have a great weekend!

Four short links: 11 June 2009

Trends, Graffiti, Games, and Streaming Video

1. Trending Topics -- full source code for trendingtopics.org, Wikipedia trend analysis. Rails app running on the Cloudera Hadoop Distribution on EC2. (via mattb on Delicious)
2. Graffiti from Pompeii -- I can't help but read these as Tweets. Herculaneum (on the exterior wall of a house); 10619: Apollinaris, the doctor of the emperor Titus, defecated well here (see also olde style Twitter) (via OvidPerl on Twitter)
3. Online Games Dominate Beijing Startonomics -- presentations from sessions on Chinese game business at Startonomics conference. Though there are many differences between the US and China games market, the one that stands out most is China’s ability to massively monetize games. Tencent, a leading Chinese web portal, social network and game developer, famously announced revenue of over $1 billion earlier this year, much of it coming from their avatar service. (via TinaTranT on Twitter)
4. Ustream's Audience for Apple iPhone Announcement Greater Than Cable News -- Ustream is amazing, you can take a consumer handycam and video broadcast live to a greater audience than many TV shows get.

Four short links: 3 June 2009

Video Chat, NGO Incorp, Smart Grid, and Enterprise Sales Funny

1. Tinychat -- very simple web-based take on videochat. Pro members get higher resolution, more rooms, and privacy. (I like the "free = public, charge for private" business model)
3. One Click Orgs -- One Click Orgs is building a website where groups can quickly create a legal structure and get a simple system for group decisions. We think social enterprises, collectives and activist groups have better things to think about than obscure legal clauses. Still getting built, but a good idea. We're one step closer to Charlie Stross's vision from Accelerando of a twisty maze of cross-shareholding organisations whose bylaws are Python scripts.
4. Trilliant Acquisition Signals Next Phase of Smart Grid -- smart grids rely on networked power meters and consuming devices. Therefore there are possible alliances between powerline broadband and smart meter companies, as this union shows. Finally, a use for broadband power? (via monkchips on Twitter)
5. The Vendor-Client Relationship -- should mandatory watching for everyone in enterprise sales. (via johnclegg on Twitter)

Big Data: SSD's, R, and Linked Data Streams

The Solid State Storage Revolution: If you haven't seen it, I recommend you watch Andy Bechtolsheim's keynote at the recent Mysqlconf. We covered SSD's in our just published report on Big Data management technologies. Since then, we've gotten additional signals from our network of alpha geeks and our interest in them remains high.

R and Linked Data Streams: I had a chance to visit with Dataspora founder and blogger Mike Driscoll, an enthusiastic advocate for the use of the open source statistical computing language, R. After founding and leading online retailer CustomInk.com, Mike went back to grad school and earned a doctorate in Bioinformatics. He has applied data analysis and programming in a variety of domains including retail, biotech, academia, and government projects.

Having been an avid user of S/S-Plus in the 1990's, I seamlessly switched over to R in the early 2000's. To this day, I consider the S/S-Plus user manuals to be the best reference and introductory books on the R programming language. (Mike wholeheartedly agrees.) R has been popular in the statistics community for many years, but I've been noticing that its visualization and analytic capabilities are attracting interest from developers. Moreover, recent efforts by the R community to improve its ability to scale large data sets (see brief update from Jay Emerson), will strengthen R's place in the Big Data stack.

(continue reading)

How Big Data Impacts Analytics

Research for our just published report on Big Data management technologies, included conversations with teams who are at the forefront of analyzing massive data sets. We were particularly impressed with the work being produced by Linkedin's analytics team. [We have more details on Linkedin's analytics team, in an article in the upcoming issue of Release 2.0.]

At the second Social Web Foo camp, I had a chance to visit with Linkedin's Chief Scientist DJ Patil. As a mathematician specializing in dynamical systems and chaos theory, DJ began his career as a weather forecaster working for the Federal government. Years later, he ended up in an analytics role at Ebay where his prior experience with massive data sets came in handy. In the short video below, DJ shares his observations on how analytics has changed in recent years, especially as Big Data increasingly becomes common. Companies are casting a wider net, and are hiring scientists from fields not traditionally known as fertile recruiting grounds for data intelligence teams.

DJ also talks about his personal journey from mathematics to e-commerce and social networks. Among his previous stints, DJ worked with the DOD and used "... social network analysis to identify terrorists."

Other short videos from Social Web Foo camp:

Recent Posts

• Big Data: Technologies and Techniques for Large-Scale Data | by Ben Lorica on March 23, 2009
• Ignite Show: Kati London on Botanicalls: Homegrown Terra-rists | by Brady Forrest on February 24, 2009
• Hope Art | by Brady Forrest on January 28, 2009
• Software for Civic Life: An Interview with Mike Mathieu of Frontseat.org | by Joshua-Michéle Ross on December 29, 2008
• GeoData Explorations: Google's Ever-Expanding Geo Investment | by Brady Forrest on December 16, 2008
• “Technology is the 7th Kingdom of Life” - A conversation with Kevin Kelly | by Joshua-Michéle Ross on November 24, 2008
• Web 2.0 Expo Europe Videos Up | by Brady Forrest on October 29, 2008