App Engine: Host Your Apps with Google

Tonight at their second CampFireOne Google Code is announcing App Engine, a hosted platform for web developers. The actual service will launch later tonight in a closed Beta. 10,000 developers will be granted access on a first come, first serve basis.

It’s about time that developers get access to Google’s platform! We’ve been hearing about Google’s server farms and development tools for years. After Amazon Web Services started doing so well we all knew it was just a matter of time (next will be Microsoft we can can safely assume). Though the obvious comparison is to AWS, they aren’t really the same beast. Amazon has released a set a disparate services that can be used to created a general computing platform. The services, though they work together, do not come bundled.

App Engine on the other hand is almost literally an engine for powering web applications. It bundles together many of the features that AWS offers into a singular package: storage like S3, auto-scaling and processing power like EC2, and a datastore like SimpleDB. App Engine also offers things that are not available on AWS like a Python runtime, Google-specific APIs and perhaps most notably a free portion of the service. App Engine is more like other web application platform services like Bungee Labs (similar but with its own language), Heroku (which supports Rails), or the now-defunct Zimki (which supported server-side JavaScript; Radar post).

App Engine includes a scalable datastore (based on BigTable and GFS as TechCrunch wrote on Friday; this will allow apps to scale to millions of users with millions of rows of data), a Python runtime that was heavily influenced by Django (the templating system particularly) and an open-source Python logging system. App Engine will auto-scale for applications and provide fail-over. Global configuration settings provide authentication, analytics and access to Google service APIs. There is also a downloadable SDK for local development and debugging. Moving locally-developed apps to the web is just a single command.

The apps all appear on the appspot.com domain. Each developer currently gets three application ids. When apps are uploaded they will appear at http://application-id.appspot.com. Developers can, of course, bring their own domains. You can see the current set of apps in the application gallery. I love the Appspot domain name; it’s an homage of sorts to Blogspot and fits in nicely with Jotspot.

App Engine will initially include several APIs. The Users API allows apps to authenticate against Google accounts and to make those accounts Admins. The Mail API allows apps to send mail — from the developer’s registered email address. Other APIs permit interactions with the Datastore and fetching content from other URLs.

Initially the service will be free and will provide 500 MBs of storage and enough bandwidth and processing power to serve 5 Million pages per month. Some portion of the service will always be free (they’ll decide on the exact levels during this Beta period). Popular apps will end up paying for their success. I imagine that in the future an Adsense account can be tied directly to an app so that any App Engine fees can just be paid from that pool.

One other important thing to note is that App Engine is not a general computing platform. It is not meant to be used for batch processing. It is a request driven platform. I can picture some App Engine’s using EC2 for batch processing behind an AppSpot front end..

It is also important to note the use of Python. Each of the other hosted platforms I mentioned had picked a specific language to make available at their release. Google’s is Python and all of the Python-built tools seem to borrow heavily from Django, a Python web framework. App Engine will work with other Python frameworks (these ones were all specifically mentioned in the documentation: EZT, Cheetah, ClearSilver, Quixote, Django and CherryPy).

I like the approach that Google is taking. They have always been clear that they want more people on the web (more pages means more ads served). They are providing the platform for people who want to put up a quick app; the real question is will they stick with it when the app hits the bigtime?

As this post goes live I am at the CampFireOne where they will be announcing App Engine. If you have any questions post them in the comments of this post or direct message me on Twitter. I will try to ask them.

Google will be speaking on App Engine at the Web 2.0 Expo in SF on Thursday, May 24th. The session will be open to anyone with an Expo Badge. Google will also cover it at their conference, Google I/O, on May 28th and 29th.

tags:
  • http://mndoci.com Deepak

    I think the problem is that some of us were expecting a general platform. Nothing wrong with the approach they’ve taken. It looks like a well designed system and for the Python community it might be perfect. Just wish they had gone for broke. For example, to run scientific computing apps, I am not sure this works (apps that are not necessarily data driven)

    The scalable datastore though is very cool

    Also, while this does not directly impact AWS, what is the impact on Bungee etc, esp if you are a Python developer?

  • http://marcusestes.com Marcus

    As soon as they implement a few more web app frameworks then it will certainly impact the AWS products. I love the idea of push-button rails deployment and I’m certain that’s coming right down the pike.

  • http://smoothspan.wordpress.com/ Bob Warfield

    Great post Brady. Deepak, don’t fret the universal platform. They will get there. Google has actually raised the bar by not forcing you to start with a raw virtual linux environment.

    There are a lot of interesting ramifications.

    More on my blog:

    http://smoothspan.wordpress.com/2008/04/08/well-done-google-app-engine-congratulations-python/

    Cheers,

    BW

  • http://mndoci.com Deepak

    Bob

    Oh not fretting. I was just expecting more. On the flipside, I kinda like the raw Linux part. I don’t see how I could implement a MPI version of BLAST on the Google cloud quite yet :)

  • http://dasht-exp-1a.com Thomas Lord

    Once again the social irresponsibility (or is it maliciousness) of O’Reilly comes to the fore.

    DO NOT HOST YOUR APPS ON GOOGLE.

    Google is trying to implement the cyc project on the sly and in more effective form and in order to gain economic advantage over all contributors (and everyone else).

    dummies.

    -t

  • http://www.2maomao.com/blog/ realfun

    Sounds really cool~!

    I am caring about if they provide fast computing environment and fast web access.

  • http://dasht-exp-1a.com Thomas Lord

    (aside to George Dyson: it’s not just “would look like” — it’s “is”)

  • http://webshopinabox.peter-tashjian.com/WebShopInABox.htm Peter T – Webshop

    Good post, does anyone know where I could get an invite for the bets? Thanks.

  • http://mike.teczno.com Michal Migurski

    Agreeing with Deepak here… the raw linux environment of EC2 is very much a feature. I think the push-button Rails deployment mentioned above requires only that some enterprising AWS user gin up a properly-configured machine image.

    The thing I’m very interested in is the environmental / ethical dimension of these services. A few more words on that on my own site: http://mike.teczno.com/notes/google-app-engine.html

  • http://www.saibposervices.com Sai BPO Services(UK) Ltd

    Nice one!!!
    Is App Engine really based on FCFS?

  • http://ramin.firoozye.com Ramin

    As a long-time Django developer, I’m pretty stoked about this.

    There appear to be a few missing pieces, like support for one’s own SSL certificates and cron. But being able to run web-apps without having to mess with OS and web-server configuration, load-balancing, and all the niggly hosting issues is going to be very handy.

  • http://dasht-exp-1a.com Thomas Lord

    Oh, and Tim…. about what you think of “Alpha Hackers” … funny story there…. i’ll have to tell ya some time…

  • andE

    so does this mean (to a not 100% techie) that basically google is trying to move the internet into their data centre?
    I guess this would make it easier for their trawling SW to work…

  • http://www.leagueofbrilliance.com Manfred Z

    The date for the Google presentation at the Web 2.0 Expo is not May 24th but April 24th.

  • Brennan Hildebrand

    The thing to consider here is that what this allows, as with Amazon’s set of tools, is merely a easier way for more people to put together and serve up web apps. If you want heavier computation, put your own server out there and reference it through which ever web service platform you like. In truth, nothing here is really groundbreaking. Get a fat pipe, throw down some linux servers running apache and build your service using whatever language/platform/database you like. You could have done it without google or amazon years ago. In fact, you probably did. These are simply more legos. The big thing that I see missing, someone else brought it up, is encryption. My bet is that is that it will be there soon.
    Don’t get me wrong, though, I’m in the beta.

  • http://www.adelph.us William

    This seems so very much like dark days of the Microsoft OS monopoly. Who can forget the memories of; stifled competition, and price fixing. In this case Google does the arm twisting AND knuckle cracking by dropping the pricing of a service to free. Goolge is using the advantage of massive scale built on the backs of consumers and media to choke the life out of them and any competitor.

    Do we now face a world where all apps are run on the google platform and coded to their specifications. Have we forgotten what happened to Borland, Netscape, and Novel.

    Make no bones about it; Google is trying to be the next OS at any cost. If they go on this path unchallenged we will see the snuffing out of a vibrant industry that was once the home of rebels that believed in the possibilities and the freedom of choice.

    It is also clear that Google is the biggest danger to Open Source software that we have ever seen.
    Where is the cloud/OS for those of us that want a choice and don’t think its a good idea for one company to own the very life of the internet.

    We at adelph.us will be launching an alternative

    “You take the blue pill and the story ends. You wake in your bed and believe whatever you want to believe.”

    “You take the red pill and you stay in Wonderland and I show you how deep the rabbit-hole goes. Remember — all I am offering is the truth, nothing more.”

    from the Matrix written by Andy Wachowski & Larry Wachowski

  • Fumanchu

    First Facebook devalues developers by giving you access to millions of users and making development very easy…

    Then OpenSocial is launched to allow you to build once and launch everywhere (supposedly)…

    And now Google (and previously Amazon) provide access to their server farms to make hosting/scaling a non-issue.

    At this rate, it won’t be long until we’re replaced by “robots” (aka one-click installs for any of your Web needs).

  • http://www.atilus.com/blog Zach Katkin

    Excellent post and rundown on the differences between this new app engine and AWS. I look forward to working with it (hopefully soon ;) ).

  • http://www.greports.com Mitch Stephens

    I am going to the Google IO conference already, so a free pass is nice, but not why I am writing.

    OK, so on face value, the Google Apps platform is very appealing.. free processing power & storage.. who could fault that. But wait… lets look at the restrictions, at least for now:
    1. Python??? Come on, that means learning a whole new language which by the way, looks nothing like any of the c-derivative languages (Java, C#, etc)

    2. How do you use pre-built libraries? Almost any REAL application will be pieced together with building blocks from other vendors. For example: An imaging library, or PDF library, or crypto, or zip handling.. get the point?? You need to find python-based tools to link to, or you will need to make external calls to handle this stuff.. that’s not going to work.

    3. Pre-built tools like DNN, which can save you years of work, will not run here.

    3. From what I can tell, this is not a normal ISP type of offering, so managing your site will not be as easy as the tools from ISPs.

    And finally, what is a developer actually saving? You can get a pretty nice ISP package for about $10 a month.. at least good enough to handle the LIKELY traffic you will have in your new killer application. Some companies even give unlimited bandwidth. If you really have an application that is used by millions of people, then trust me, you can afford to host it anywhere. Pop a few ads on the site, and you will easily cover the cost of the ISP

    Add up the cost of a whole new learning curve, and I think you will see that you can get to end-of-job a lot sooner with the tools and technology you already know how to use.. your time is WAY more valuable than the few bucks you will save on paying for the platform.

    Having said all that, I do think that Google is the platform of the future, and I love their bold moves and aggressive embracing of ‘freeconomics’. However, this specific offering is actually less important to most developers that some of the other things they are working on, such as Google Docs, Google Apps etc. You can still create very cool mash-ups using those other services, from your existing tools.. with Google sanctioned API wrappers.

  • http://blog.museway.com marco

    Hi,
    I can’t avoid to be a little bit worried, as many other I see, about this Google App Engine thing. As I couldn’t avoid about the Amazon one.
    You know, it seems to me that while offering a services as the one we are talking about it will be quite simple to use it in the future as a way to control the net.
    I’m not saying this is the goal they want to achieve, it is only a thought.

    Anyway I’m also happy to see it’s happening.

    I’m interested also on considering the effects such kind of services/tools will have on the creativity behind the proposals internet will bring to us.

    Cheers

  • http://puremvc.org/ Cliff Hall

    As with any platform where you can write your own apps, you can still make a mess without applying some patterns to your development.

    The PureMVC Framework for Python has a nice demo that is a traditional server side web app; a Blog where you can view, create, edit and delete timestamped posts.

    This demo is well documented and easy to get running quickly, even if you haven’t been granted a Google Apps account yet.

    Check it out:
    http://trac.puremvc.org/Demo_Python_GoogleAppEngine_Blog

    -=Cliff>

  • http://www.web2py.com Massimo

    you can also run code developed with a high level framework like web2py on the appengine.
    Here is an example

  • Fred

    All your base are belong to Google.

  • http://www.modpoll.com Tom

    Love it! Me and buddy put together http://www.modpoll.com on Appengine in like 3 weeks. Outside of the recent outage we’ve had no problems.

  • http://www.joyent.com Bryan

    Brady,

    Thank you for summing up the recent GAE announcement. However, your post did keep me up last night (for a few minutes at least) because, like so many articles before, this perpetuates the myth that AWS delivers what GAE delivers and vice versa.

    EC2 is hosting – that’s it, that’s all. Yes, it’s by the hour and there is the concept of infinite, on-demand supply of infrastructure but, it still hosting. Even if you consider AWS storage products, AWS is hosting with additional products. It is nowhere near to a Platform as a Service (PaaS). It is so-much-so raw iron that until recently, third parties like Rightscale were the only ones supplying the management software to actually make operating it straightforward.

    GAE, on the other hand is a software development platform – a true PaaS. With GAE you get an application run-time environment on which you write code and once you deploy the code, it runs. GAE just happens to provide “hosting” as part of its service because that is what is needed to run an application. It auto-scales without any manual or third party intervention. And, GAE certainly does not provide general hosting like AWS.

    So, making a direct price comparison between the two is like comparing apples to oranges and perpetuates the myth that the two offering are comparable.

    With a PaaS you do not need third party software, network administrators or large consulting firms to manage deployment or ongoing operations. With AWS you do. Therefore, AWS is just more expensive to operate than software written on GAE’s PaaS (because you actually have to manage it).

    Hopefully, once GAE, and more importantly to me, the soon to be launched Joyent Smart Platform (F.K.A. http://reasonablysmart.com) evolve into open, language agnostic, fully distributed PaaS solutions, the value proposition becomes even more clear and this somewhat illogical comparison stops happening.

    I understand that in the software business, as in many others, some people make money by building something from the ground up and others just want to focus on making money by running a business.

    AWS is an offering suited for running legacy apps, replacing existing on-site infrastructure and for those who, for whatever reason, think they can build the entire stack better themselves (shameless plug: in all three cases, Joyent supports these endeavors better than AWS but, that is for a different discussion).

    A PaaS is suited to those who want to focus on differentiation, business building and strategy without having to worry about power generation.

    Once each of the value propositions is clearly understood the costs certainly will not be the same…

  • http://ww.joyent.com Bryan

    Clearly commenting on a different post – sorry Brady.