Apr 14

Tim O'Reilly

Tim O'Reilly

Is Google App Engine a Lock-in Play?

Venture capitalist Brad Feld just put up an intriguing post comparing Google App Engine to Amazon EC2. The meat of the entry is from an analysis by Brad's friend Scott Moody. Here are the juiciest bits, pro and con:

With EC2, you still have to set-up load balancers, configure multiple replicated database servers, implement scalability hacks if things grow too fast (such as distributed caching of data via memcached), keep distros and apps up-to-date, etc. Bottom Line: EC2-based companies still require sys admins, AppEngine companies don't. That will certainly change as more companies begin offering EC2 server management services.

Google provides a non-relational datastore and that's the only datastore available (no traditional file system, no relational databases). With EC2, people generally use MySQL or Postgresql. Amazon offers a non-relational datastore called SimpleDB, but it's a bit *too* simple. For example, it does not support sorting of results sets. Huh? That makes it non-workable in my opinion. There's also an issue with using EC2 virtual machines for your database servers -- Amazon says that when a virtual machine crashes, all the data managed by it disappears, so virtual machine crash = hard drive crash.

With EC2, programmers can use any (non-Microsoft) language to develop their apps. AppEngine users must code in Python. Also, Google does not support sockets at this time. All cross-app communication must be done via HTTP.

At *this* moment in time, it would be difficult to move apps off of AppEngine. Doing that in EC2 is trivial. This, to me, is the biggest issue, as I believe it could make startups less-interesting from an acquisition perspective by anyone other than Google. This will most likely change as people develop compatibility layers. However, Google has yet to provide any information about how to migrate data from their datastore the best I can tell. If you have a substantial amount of data, you can't just write code to dump it because they will only let any request run for a short period before they terminate it.

This last point is really very serious. I've been warning for some time that the first phase of Web 2.0 is the acquisition of critical mass via network effects, but that once companies achieve that critical mass, they will be tempted to consolidate their position, leading ultimately to a replay of the personal computer industry's sad decline from an open, energetic marketplace to a controlled economy.

Now it may be that this is a temporary oversight, and that Google does intend, long term, to make it easy for developers to export their applications. After all, Eric Schmidt says he reminds his employees all the time, "Don't fight the internet." But it's also possible that this is one more sign that one of the big guys is forgetting the principles -- the internet as a platform (not "my company as a platform"), harnessing the power of user contribution (which as John Musser pointed out means that you always "pay the user first"), small pieces loosely joined-- that brought their success in the first place.

Keeping the internet as an open platform is a choice. We didn't understand what was happening to the PC ecosystem, but we've seen this movie before, so we should recognize and fight this plot line when we see it happen on the internet. We need to keep our cloud services vendors honest, and tell them we want an open, interoperable platform, not one based on lock-in.

Of course, as some wag said, "the only thing we learn from history is that people don't learn from history."

P.S. There's some further good discussion on the lock-in issue in a Q&A about AppEngine put together by Stephen O'Grady.

tags: web 2.0  | comments: 15   | Sphere It

Previous  |  Next

0 TrackBacks

TrackBack URL for this entry:

Comments: 15

  Tony [04.14.08 10:44 AM]

I personally don't feel that google is trying to "lock-in" customers, but rather that the app engine was just release too early with a too constraining feature set. While most of their apps really are awesome (I'm a big fan of their reader, gmail, & igoogle), I get the impression that with the app engine seem to be just wanting to jump on the cloud computing bandwagon.

No argument with the general sentiment of this statement though:
"...the first phase of Web 2.0 is the acquisition of critical mass via network effects, but that once companies achieve that critical mass, they will be tempted to consolidate their position, leading ultimately to a replay of the personal computer industry's sad decline from an open, energetic marketplace to a controlled economy."


  Simon Wardley [04.14.08 11:02 AM]

Hi Tim,

I feel like I'm pointing out the obvious and just repeating what I talked about at OSCON. Then I realise you already know all of this and are just getting the conversation going.

In my view, the real innovation in GoogleAppEngine (GAE) is the open SDK and the possibility that open SDK compliant environments can be re-implemented elsewhere (including on Amazon’s web services). The difference between Google and Amazon is that Google has taken the approach of creating an open sourced standard and in effect encouraging competition. Amazon has always been about their “secret sauce”.

Whilst GAE is at the level of a framework and Amazon is at the level of infrastructure, the approach of an open sourced standard, if adopted, is likely to spread up and down the stack whether this is at SaaS, PaaS, HaaS or whatever other aaS people choose.

  sean [04.14.08 11:39 AM]

There's also an issue with using EC2 virtual machines for your database servers -- Amazon says that when a virtual machine crashes, all the data managed by it disappears, so virtual machine crash = hard drive crash.

According to this blog post (, this will no longer be an issue in the near future.

  Charlie [04.14.08 11:43 AM]

I'd be surprised, and very disappointed, if Google didn't offer a way to get your application's data out of Data Store. After all, they have differentiated themselves by letting you easily get your email out of Gmail. Other webmail providers, notably Microsoft, have not been so accommodating. That's been a point of pride for Google. Not allowing this with App Engine would quite literally go against everything they stand for.

As for the programming interface they provide, it makes perfect sense to me. They've designed a scalable architecture that works a certain way, and if you want to use their cloud, why shouldn't you expect to be writing for that architecture? I expect the short request constraint will be a non issue once they add support for submitting your own MapReduce jobs against your data store objects.

There are open source projects already modeled after MapReduce and BigTable (Hadoop, HBase and Hypertable for instance), and I fully expect that the App Engine API will be duplicated on other cloud computing hosting companies, either by startups acting as intermediaries or by the hosting companies themselves.

  Niraj J [04.14.08 11:55 AM]

comparing AWS and AE at this point of time is really comparing apples and oranges.

Both are targeted at different market segments in the cloud computing market refer here

AWS is coming to the market from bottom up-exposing data center first and then adding layers above it- simple Db , Webstore and I am sure they will add more.

Google is really middle out - Exposing the Hosted Development platform first and then going up via SalesForce relationships and going down(don't know right now if they will).

The higher up you get, more of a lock in situation you will be in but you potentially get more (like better integration , less management overhead etc ).

I think it is key for any vendor to understand that exposing more and more entry points to their services is going to enable them to get the critical mass. Let the customers decide which entry point works for them. AWS seems to be better positioned right now in this approach. Refer here

  Chris Anderson [04.14.08 01:16 PM]


I just released a port of the App Engine environment to EC2. It's open source, and hosted on EC2 at

I hope this puts the lock in concerns to rest. We've needed an application deployment strategy that doesn't involve managing machines for a while. One can hope the App Engine container becomes a standard.

  Stoicho [04.14.08 01:19 PM]

Google simply has to answer these concerns NOW!

They put the thing out there (App Engine) and started listening for feedback. This is sooo the startup way, I can't believe they did not have plans before the release. If they had plan before the release - they are just playing games ...

Both conclusions don't speak well for them.

  Karl Fogel [04.14.08 04:41 PM]

I suspect that pretty soon we'll see Google not only offering a migration plan for big datasets, but also playing nice w.r.t. independent implementations (that are surely on the way -- or already here: see Chris Anderson's comment) of the AppEngine APIs.

Google has every reason to avoid lock-in. They want to attract customers, and in a non-locked-in environment they still start out with the advantages of name recognition, reputation, and well-scaled / well-tested infrastructure. Never mind "Don't Be Evil"; even considered purely as a strategy, lock-in wouldn't make a lot of sense for Google.

[Disclaimer: I used to work there, but no longer do and have no financial ties to the company.]

  ian holsman [04.14.08 05:10 PM]

Initially I thought of it as a lock-in play.

but it is more a way of leading the market. the API which is problematic is the datastore one, but open source will replicate it soon via hbase/hypertable and hadoop. (as Chris has done via EC2)

It will hurt the current hosting providers, but that allways happens when you shift the market like this.

  Jason [04.14.08 06:39 PM]

You are no more locked in to Google than you are to any other SDK. You're making a decision to use the SDK, like making the use of Rails or Drupal or Zope.

The effort required to switch out of the SDK is the same for all of those platforms.

Lock-in? No more than usual.

  friarminor [04.14.08 07:19 PM]

I would like to give Googs the benefit of the doubt regarding the lock-in issue. But as it is, the small players like Morph eXchange can only offer that to their advantage - liek aces up their sleeves.


  Andy Chilton [04.15.08 03:28 AM]

Heh, it seems you're saying the same as I did. I posted this 4 days ago :-)

  Andy Wong [04.15.08 04:20 PM]

I think portability is out of question here. Google App Engine is to not about getting cheap and efficient storage, bandwidth and CPU. Google wants you to build applications with Google web servicessss. Cheap storage and CPU are just for your to build app around Google services easier. So, whether Google is a "Lock-in" play is out of question too.

If your app is really good and popular, your app will become good object of being acquired by Google.

Because your app is hosted in Google, Google has first hand info about the quality and the popularity of your app, and Google then easily give you a fair (from Google's point of view) price to acquire.

  Massimo [04.24.08 02:51 PM]

I agree with your concerns. Here is a solution. Use web2py. You can develop on your system, deploy and run everywhere including EC2 and the appengine. If available you can run with postgresql, mysql or Oracle, as well as the GQL. You will not be locked in any vendor's solution. You can even bring your apps with yu and run them off a USB drive. It is free as in beer and as in speech.

click for a video example

  Aral Balkan [12.28.08 03:36 PM]

Hi Tim,

Just a quick note to anyone reading this that as of December 24th, there is an open source data export solution that I released for Google App Engine for backing up and restoring your datastore called Gaebar (Google App Engine Backup and Restore). See for a screencast and links to the source.

There is also AppRocket, another open source project, that allows you to sync your datastore with a MySQL database. You can find that project at

Also, as I understand it, Google is working on its own solution.

Post A Comment:

 (please be patient, comments may take awhile to post)

Type the characters you see in the picture above.