Structure and Velocity

Several people have asked me about the differences between Om Malik's Structure conference and our Velocity Web Performance & Operations conference.  Velocity is on June 23 & 24th at the SFO Mariott, and Structure follows on June 25th in San Francisco. 

The conferences are complementary: Structure discusses what is changing in internet infrastructure, and Velocity teaches how to make that change happen.

I've been recommending that anyone considering Structure make sure their engineering teams are going to Velocity.  For many technical leaders I think there is value in attending both.  I plan on doing so, and am looking forward to seeing Danny Kolke (my new boss) and Werner Vogels (my former meta-boss) speak.

The knowledge and skills learned at Velocity can be put to immediate use and will have significant impact on your business.  The reason for this is simple:

Faster, scalable, and highly available websites serve more pages to more customers in the same amount of time.

That's why we've worked hard to make Velocity the best resource for engineers to learn how to build and operate at web scale.  Here are a few examples:

Adam Jacob will give a step-by-step overview of Building an Automated Infrastructure, and then Luke Kanies will follow up with an in-depth session on Puppet.  This is the exact combination I used to explain how effective operations is a huge competitive advantage:

Luiz Barroso will describe Google's approach to energy-efficient datacenter design and management.  Applying these lessons can ultimately save millions of dollars, increase your operational agility, and decrease your environmental footprint.

Mandi Walls will teach how actionable logging can mean the difference between a 20-minute outage and a 2-hour outage while esoteric error codes are deciphered or developers are contacted to investigate.

Eric Lawrence, Program Manager for Internet Explorer, and Mike Connor, lead developer for Mozilla Firefox will explain how to optimize page performance for their respective browsers.  We'll also have demos of leading performance testing tools: HTTPwatch, Fiddler, AOL PageTest, and Firebug.

John Allspaw from Flickr will be be giving a talk about Capacity Management.  John's way of explaining both the problem and the opportunity is wonderfully straightforward:

You can check out the rest of the program and register on the Velocity site.  (Hint: You can use the code "vel08js" for a 20% discount.)  I'll be posting frequently as we add speakers and events.  I hope to see you at Velocity!

Disaster Technology for Myanmar/Burma aid workers

There is an ongoing crisis in Myanmar (Burma) in the aftermath of cyclone Nargis. The ruling military junta is finally allowing humanitarian organizations into the region after denying access for almost a week. The situation is grim, and you can help by donating to organizations like: Doctors without Borders, Direct Relief, and UNICEF.

There has been some incredible discussion on the humanitarian tech and Geo lists in the past 24 hours around adapting/improving existing collaboration services to work with the tools in the field. Mikel Maron and I will be speaking about this at Where2.0 next week, and it looks like some exciting work will be happening there and at WhereCamp.

Eduardo Jezierski from InSTEDD is currently working to localize the Sahana Disaster Management System

Jonathan Thompson's organization, Humanlink, has been working on adapting technology for aid workers for some time. You can follow recent developments on the Aid Worker Daily blog.

Update: Paul Currion posted a big list of other projects now underway to the humanitarian.info blog:

• A Sahana instance is being set up for the use of anybody who needs it, with the support of INSTEDD and possible uptake by NetHope members.
• Direct Relief International have done up a KMZ file of health facilities in-country, based on the WHO 2002 Global Health Atlas.
• OCHA are prepping a HIC to support the existing Myanmar Information Management Unit, who have already put out some W3 maps.
• UNOSAT have also got their sat on with a KMZ file of the cyclone path and the usual satellite mapping.
• Ditto ITHACA, who have released a series of satellite maps showing the impact of Nargis.
• ReliefWeb’s info stream on Cyclone Nargis is of course like drinking water from a hose, with their map filter probably most useful.
• The WorldWideHelp blog roars into action with all the news that’s fit to blog.
• A couple of the mailing list discussions that I’m on are talking about ways in which we might leverage cellphone and/or satellite phone communications if they become available, particularly for tracking relief and relief personnel.
• Digital Globe and Geo-Eye have hopped the NASA satellite for an updating KML layer on the cyclone.
• Microsoft apparently have a team on standby to deploy the refugee tracking software that was developed for Kosovo (no reference yet).
• Telecoms sans Frontieres are also on standby out of Bangkok, waiting for access to free up.
• Also Infoworld points out that - with regards to early warning - IT didn’t fail Myanmar, people did.

(continue reading)

You Become what You Disrupt - (part two)

Google's GrandCentral (Radar coverage) was down over the weekend resulting in missed calls and other phone problems for its users.

This is very similar to the the two day Skype outage last year where I said that "You Become what You Disrupt". I've spoken about this issue several times, most recently at the Princeton CITP "Computing in the Cloud" workshop.

The problem is that it's not particularly clear at what point a disruptive innovation becomes a utility. As innovators it's important that we recognize that this point will arrive and prepare for it. I believe that we have a responsibility to be good stewards of the technologies we create, and to take responsibility for protecting people who come to rely on those technologies to live their daily lives. When we fail to do that, we may find ourselves being cast as either fools or villains who must be regulated and controlled.

Ultimately, I think we will evolve a set of safety standards very similar to building codes. For instance, it appears that a multi-datacenter strategy would have prevented the GrandCentral outage. (As I've said many times before: Datacenters are a Single Point of Failure!)

Cofounder Craig Walker writes: "I wanted to write a quick note to all the GC users and apologize for the service interruption this morning. We had a power issue at our current colo facility and it knocked us off line for a few hours. Unfortunately I’ve been up in the mountains with the family this weekend and had no cell/internet coverage so couldn’t respond earlier. I did want to let you know that we were able to restore the service by noon today and are working extremely diligently to make sure this won’t occur in the future. We’ll do a better job keeping you informed in the future, not only about service related issues but also about upcoming features, soliciting your feedback, and generally making sure that you, the GC user, is well informed as to what’s going on with the service."

Will better industry standards, best-practices, and independent certifying authorities emerge for these new utilities without innovation-stifling regulation? I hope so.

(continue reading)

Velocity preview at Web2.0 Expo

At the Web2.0 Expo this month we have a small preview of some of the topics and speakers at the Velocity Web Performance & Operations conference.  (Radar readers get a 20% discount by using "vel08js" as a discount code... and yes it works with the $300 early registration discount!).

Failure Happens
Friday @ 11:00 am, Room 2009

Artur Bergman and I will kick off the day with an entertaining/informative/eye-opening review of the year’s biggest failures, disasters, and painful lessons learned.

We'll review incidents by underlying root cause with a focus on what could have been done to prevent it. We promise not to be too harsh on anybody, although we will give special attention to particularly ironic failures or those that are "entertainingly coupled" to absurd marketing claims.

(Hint: Send your boss to this talk if they don't understand why you and your whole team need to go to Velocity.)

Even Faster Web Sites
Friday @ 1:30 pm, Room 2012

Steve Souders is the co-chair of Velocity and author of the bestselling book High Performance Web Sites. At the Expo last year Steve gave an incredibly popular talk on the 14 best practices he developed while working as the Chief Performance Yahoo!.

(continue reading)

Data Center heating the Town Pool

According to GreenerComputing.com:

A public swimming pool in Zurich will soon be heated for the comfort of local residents, thanks to an innovative solution: heat generated by a data center that would otherwise be classified as waste. The new data center in Zurich is one of three projects in Europe and the Middle East that IBM has announced in recent days. The Zurich project is a new data center for GIB-Services, a hosting and co-location company. In Austria, IBM has announced a plan to construct a green data center for green furniture company kika/Leiner; IBM has also landed the contract to build what it calls the most energy-efficient data center in Egypt, for Telecom Egypt.

For more see IBM Big Green and The Raised Floor blog.

What is Web Operations?

Theo Schlossnagle wrote a brilliant summary of one of the biggest challenges we discussed at the Velocity Summit in January:

What is this Velocity Summit thing? It was a bunch of web architects from highly trafficked sites sitting around talkin' smack. It was operated in Foo style. However, one thing that made me really appreciate this meet-up was the lack of self-importance displayed by attendees. Everyone was just there to talk -- not to make people understand how much they knew. We were talking about The O'Reilly Velocity Web Performance and Operations Conference: what it should be and why.

Two things that I walked away with were (1) a realization of the lack of a career path for people who do what we do (no standard titles, no standard roles and responsibilities and certainly a lack of sex appeal) and (2) a clear lack of terminology for the technology requirements that are so common in these environments. Terminology is easy, in my opinion -- you just argue until someone wins. Of course, arguing is a hobby of mine, so I have bias. On the other hand, defining a career path that is an industry accepted path is hard.

The term Web Operations was used a lot during this event. While it isn't awful, I really don't like this term. The hard part is that the captains, superstars, or heroes in these roles are multidisciplinary experts. They have a deep understanding of networks, routing, switching, firewalls, load-balancing, high availability, disaster recovery, TCP & UDP services, NOC management, hardware specifications, several different flavors of UNIX, several web server technologies, caching technologies, several databases, storage infrastructure, cryptography, algorithms, trending and capacity planning. The issue: how can we expect to find good candidates that have fluency in all of those technologies? In the traditional enterprise, you have architects which are broad and shallow and their team of experts which are focused and deep. However, in the expectation is that your "web operations" engineer be both broad and deep: fix your gigabit switch, optimize your MySQL database and guide the overall architecture design to meet scalability requirements.

I struggle with this. Not everyone can be a superstar. More importantly, no one can really start as a superstar. If we use an apprentice model (which is common in industries without institutional support) we limit the total number of able workers in this field. So, how do we (re)define the requirements for a junior web operations person?  [read more]

One of the reasons I'm excited about Velocity is that we're increasing the pool of great operations people.  We're getting inquiries from companies interested in sending groups of 30-40 people, and I expect more as we confirm speakers and sessions.  You can secure a spot now and get a $350 early registration discount.

Amazon improves EC2 (by embracing failure)

Amazon just announced two big improvements to EC2:

• Multiple Locations
  Amazon EC2 now provides the ability to place instances in multiple locations. Amazon EC2 locations are composed of regions and Availability Zones. Regions are geographically dispersed and will be in separate geographic areas or countries. Currently, Amazon EC2 exposes only a single region. Availability Zones are distinct locations that are engineered to be insulated from failures in other Availability Zones and provide inexpensive, low latency network connectivity to other Availability Zones in the same region. Regions consist of one or more Availability Zones. By launching instances in separate Availability Zones, you can protect your applications from failure of a single location.
  
  
• Elastic IP Addresses
  Elastic IP addresses are static IP addresses designed for dynamic cloud computing. An Elastic IP address is associated with your account not a particular instance, and you control that address until you choose to explicitly release it. Unlike traditional static IP addresses, however, Elastic IP addresses allow you to mask instance or Availability Zone failures by programmatically remapping your public IP addresses to any instance in your account. Rather than waiting on a data technician to reconfigure or replace your host, or waiting for DNS to propagate to all of your customers, Amazon EC2 enables you to engineer around problems with your instance or software by quickly remapping your Elastic IP address to a replacement instance.

Datacenters and geographic regions are Single Points of Failure (SPOF) too.  Failure Happens, and it's far better (and cheaper) to build services that are resilient to failure than to try to prevent them from happening.  This is a big step in the right direction.

Update: RightScale posted an excellent overview of how this works.

Recent Posts

• Trendalyzer view of the banking crisis on March 19, 2008
• Paging systems and Conference Bridges for startups & small teams on March 10, 2008
• Steve Souders asks: "How green is your web page?" on March 7, 2008
• Today's ETech Hack is Tomorrow's Critical Infrastructure... on March 4, 2008
• DIY Multitouch with the Wiimote on February 25, 2008
• US Judge censors WikiLeaks.org by ordering DNS records removed on February 18, 2008
• Amazon S3 / EC2 / AWS outage this morning... on February 15, 2008