"infrastructure" entries

John Adams on Fixing Twitter: Improving the Performance and Scalability of the World's Most Popular Micro-blogging Site

Twitter is suffering outages today as they fend off a Denial of Service attack, and so I thought it would be helpful to post John Adams’ exceptional Velocity session about Operations at Twitter. Good luck today John & team… I know it’s going to be a long day! Update: Apparently Facebook & Livejournal have had similar attacks today. Rich Miller…

Four short links: 9 July 2009

Four short links: 9 July 2009

  1. Ten Rules That Govern Groups — valuable lessons for all who would create or use social software, each backed up with pointers to the social science study about that lesson. Groups breed competition: While co-operation within group members is generally not so much of a problem, co-operation between groups can be hellish. People may be individually co-operative, but once put in a ‘them-and-us’ situation, rapidly become remarkably adversarial. (via Mind Hacks)
  2. Yahoo! TrafficServer Proposal — Yahoo! want to open source their TrafficServer product, an HTTP/1.1 caching proxy server. Alpha geeks who worked with it are excited at the prospect. It has a plugin architecture that means it can cache NNTP, RTSP, and other non-HTTP protocols.
  3. App Engine ConclusionsI’ve reluctantly concluded that I don’t like it. I want to like it, since it’s a great poster child for Python. And there are some bright spots, like the dirt-simple integration with google accounts. But it’s so very very primitive in so many ways. Not just the missing features, or the “you can use any web framework you like, as long as it’s django” attitude, but primarily a lot of the existing API is just so very primitive.
  4. Microsoft HohmSign up with Hohm and we’ll provide you with a home energy report and energy-saving recommendations tailored to your home. Wesabe for power at the moment, with interesting possibilities ahead should Microsoft partner with smartmetering utility companies the way Google Powermeter does. This is notable because this is a web app launched by Microsoft, with no connection to Windows or other Microsoft properties beyond requiring a “Live ID” to login. For commentary, see Microsoft Hohm Gets Green Light for Launch and PC Mag. (via Freaklabs)

Announcing: Spike Night at Velocity

Guest blogger Scott Ruthfield is a Program Committee member of the O’Reilly Velocity: Web Performance & Operations Conference.  Web Operations is not for the casual observer: it’s for a particular kind of adrenaline junkie that’s motivated by graphs and servers spinning out of control.  Jumping in, on-your-feet analysis, and experience-based-experimentation are all part of solving new problems caused by unexpected user and machine behavior,…

Velocity 2009 – Big Ideas (early registration deadline)

(tag cloud created from Velocity session & speaker information using wordle.net) My favorite interview question to ask candidates is: “What happens when you type www.(amazon|google|yahoo).com in your browser and press return?” While the actual process of serving and rendering a page takes seconds to complete, describing it in real detail can take an hour. A good answer spans every part…

Four short links: 13 Apr 2009

Four short links: 13 Apr 2009

Worms, sorting, languages, and infrastructure:

  1. Twitter XSS Attacks (Lynne Pope) — several incarnations of a worm spread quickly across Twitter this weekend. Twitter profiles are generated by themes, whose parameters users can change. The user-supplied value for the colour was used directly in the CSS color field without filtering, which the original worm strain used to end the CSS and begin Javascript to put the worm into the profile of any Twitter user who viewed the infected profile. Infected users were made to tweet about the worm, with links that would infect anyone who viewed. The worm spread quickly through RTing one of the worm’s messages, which claimed to link to instructions on fighting the worm. Later variants use background-color and background parameters. Initial variations downloaded Javascript from mikeyylolz.uuuq.com, since closed down by its hosting company. Later variants download the code from stalkdaily.com, the site that the initial variation spammed about. I wonder whether the 17-year old author of the variants will be able to pay his inevitable legal bills through Google click dollars? (also interesting: Sophos and bdonews)
  2. Visualising Sorting — some beautiful and informative illustrations of how sorting algorithms work. (via @ajtowns)
  3. Art and Code: Obscure or Beautiful?In the presentation called “50 in 50″ you can see Guy Steele rap about APL and later in the video about spelling keywords backwards. The song about God wrote in Lisp code is also a part of the presentation. Among the languages mentioned are APL, Cobol, AP/I, Scheme, IPL-V, AED, Madcap, Piet, SNOBOL, ADA, Algol60, Intercal, Logo, Perligata, Shakespeare, Lucid, Occam, HQ9+, MUMBLE, Rake, Perl and of course Lisp. It kicks in at about 3m20s and is rather a post-modern presentation. (via
  4. Experiences Deploying Large-Scale Infrastructure in Amazon EC2As an aside, I’ve been very impressed with the reliability of EC2. Like many other people, I didn’t know what to expect, but I’ve been pleasantly surprised. Very rarely does an EC2 instance fail. In fact I haven’t yet seen a total failure, only some instances that were marked as ‘deteriorated’. When this happens, you usually get a heads-up via email, and you have a few days to migrate your instance, or launch a similar one and terminate the defective one. (via Simon Willison)

[Heapsort Illustration]

AT&T Fiber cuts remind us: Location is a Basket too!

The fiber cuts affecting much of the San Francisco Bay Area this week are similar to the outages in the Middle East last year (radar post), although far more limited in scope and impact.   What I said last year still holds true and is repeated below: From an operations perspective these kinds of outages are nothing new, and underscore why…

It's Really Just a Series of Tubes

Molly Wright Steenson hit the Ignite jackpot at Etech this year with her explanation of the steam powered network of pneumatic tubes of the 1800s. If you’re someone that, like me, has a [somewhat obsessive relationship with Internet Infrastructure](http://conferences.oreilly.com/velocity), you must watch this talk.

Four short links: 30 Jan 2009

Four short links: 30 Jan 2009

Two serious links and two fun today, thanks to Waxy and BoingBoing:

  1. EveryBlock Business Model Brainstorming — Adrian Holovaty’s project was funded by a Knight Foundation grant that’s about to run out. The software will be open sourced but he’s inviting suggestions of business models that would enable the project team to continue working on it full-time. Having used and created open source to show newspaper companies how to do journalism online, will he now work on an open source way for them to make money?
  2. Infrastructure for Modern Web Sites — Leonard Lin lays out what’s required in systems and platforms for modern web sites. Perl succeeded in part because its data types were the things you had to deal with (files, text, sockets). Will the next gen of tools (the ‘Rails killer’ if you will) offer users, taggable objects, social objects, etc. as primitives?
  3. Academic Earth — takes open courseware from different universities and integrates them into a coherent UI. Transcripts. Slurp.
  4. Love2D — a Lua-based 2D game engine. I’m looking at it to see whether it works for me as the next step for 9 year-old kids interesting in programming games in my computer club.

Service Monitoring Dashboards are mandatory for production services!

Google App Engine went down earlier today. GAE is still a developer preview release, and currently lacks a public monitoring dashboard. Unfortunately this means that many people either found out from their app and/or admin consoles being unavailable or from Mike Arrington's post on TechCrunch. Google has a strong Web Operations culture, and there are numerous internal monitoring tools in…

Automated Infrastructure Podcast on IT Conversations

Adam Jacob and I did an IT Conversations podcast with Phil Windley last week, which I really enjoyed. We started with a summary of Adam's excellent Web2.0 Expo session, covered the phases of startup growth using virtual infrastructures like EC2 and 3tera, and discussed how Puppet shifts us to "Infrastructure as Code". We even got into the challenges and opportunities…