A Week in the Valley: Hyperic

I spent the week before Where 2.0 driving from San Francisco to San Jose and back again, visiting startups and big companies. I've already written about my trips to Ning, Meebo, and Google. On the Friday of that week I spent a great morning and lunch catching up with Doug MacEachern of Hyperic.

Doug and I go way back to the first Perl Conferences in the late 1990s. Doug's the author of mod_perl, the system that drives Salon, IMDB, LiveJournal, CitySearch, and dozens of other Internet powerhouses. He's been quiet and off the radar for a while, so when Tim sent me mail saying "Doug has resurfaced, you should catch up with him" I leapt at the chance.

Doug's now part of the founding team at Hyperic, running from cheap office space right by the highway in downtown San Francisco. They bought Covalent's enterprise monitoring system and have their own company building, selling, and supporting it. They're growing fast: it was just the five founders when I saw them and by now they're at eight. They're preparing to open source it and wanted to tell me about it.

Sordid History

It grew out of their experience at Covalent, working on an Apache+Tomcat GUI. They started thinking about systems monitoring and management, and the real problems to be solved. Customers wanted more monitored than just Apache and Tomcat, so the CAM team started to build out monitoring on a much stronger and more generalized platform.

Hyperic does five things:

1. Monitoring (keeping two years of data, etc.).
2. Control: start, stop, vacuum tables in PostgreSQL, trigger Java garbage collection, and other app-specific functions.
3. Alerting: "your database server has stopped running, commit hara-kiri now"
4. Log aggregation and monitoring.
5. Auto-inventory: automatic configuration, unlike the ibiquitous open source Nagios.

Their agent does process-table scans for applications, filesystems, interfaces, etc. It's portable across major platforms using the SIGAR library that Doug wrote. Interesting note: he used VMware to make sure SIGAR worked on a bunch of x86 platforms (Linux, FreeBSD, Solaris x86, etc.). For AIX, HP-UX, Solaris on Sparc, Linux on amd64, etc. they had to use real boxes. They also use the VMware for QA and, of course, developing and supporting their VMware management system.

They integrate rather than compete. E.g., at a certain large German bank the hardware people use Tivoli and the software people use Hyperic and it was trivial to get Hyperic to use the established Tivoli alert and escalation system. Ask Jeeves was a big Nagios shop, and you can use Nagios plugins with Hyperic.

Where Are They Now?

They closed a round with Accel and Benchmark, $3.8MM that's mostly Accel. They started by buying the tech from Covalent, rescuing a $12MM investment in development, and assumed the liability for the existing customers (Covalent's customers in the pipeline became Hyperic's customers). Interesting model for a startup: "you've built this product but are focusing elsewhere, let us take it off your hands." It reminds me of ActiveState's recent rebirth. They have a yearly subscription business model. Customers include: Ask Jeeves, eHarmony, a division of AOL, High5.com, National Semiconductor.

They have plugins for business metrics, so you can measure the efficacy of your systems and not just their presence. E.g., their customer eHarmony knows there's a problem if the number of completed registrations drops below a certain level. They've used the money to hire some new people. E.g., John Mark Walker of Linuxworld/IDG is now their community manager.

Open Source?

Part of the strategy for taking investment money is to open source it. They monitor a new product by writing a plugin for it. It's a busy mature space, not a new one. IBM, Tivoli, BMC, HP OpenView are all the big names in the space. Nagios is the most visible open source product in this space, as other companies realize (e.g., GroundWork bundles Nagios with other products to build a package of features). Nagios and Hyperic are cooperative, not competitive, and Hyperic's working with other members of the Open Management Consortium on cooperative existence. Nagios does a few things very well, while Hyperic does the things that Nagios will never do.

They have binaries up now and the source goes up today. They'll be talking about it at OSCON (and also in Tim's O'Reilly Radar Executive Briefing. They're open sourcing the monitoring, auto-inventory code, and all of their plugins with the exception of the VMware plugin. Enterprise (commercial) version will have control, log tracking, high-availability, deployment tools.

They're open sourcing for adoption. They have OEM relationships, e.g., JBoss didn't license all the plugins and added more features specifically for JBoss and TomCat. Doug's baby in the project is this cross-platform library that knows about processes, filesystems, network interfaces, etc. (SIGAR). This will up as a separate open source library from the general open sourcing of Hyperic, and it's something I'm excited to see.

Random Tech

They, like a surprising number of other projects I've heard a lot about, use Atlassian Software's Confluence (Enterprise Wiki) and JIRA for bug tracking.

They're hiring J2EE and Ajax people. The J2EE base of Hyperic means it won't get a lot of look-in in the LAMP world where the Java systems administration skills aren't widespread and there's some developer resistance to Java. In the best of all possible worlds, open source developers would want to write Hyperic plugins to let Hyperic monitor and manage their software because that eases their software's entry into the enterprise.

A bigger problem than the server-side Java is that PHP, Ruby, Python, Perl, etc. don't have built-in management unlike Java's JMX or hardware's SNMP. How do you find out from your Perl program how much memory the interpreter has used? Can you trigger a garbage collection? How do you find out the number of threads in your Ruby program? This is an issue for open source languages rather than Hyperic & anyone hoping to monitor a Ruby/Python/etc. app will have to face these same questions. Hyperic have plans to help fill this need.