"server" entries

Zero Downtime Application Updates with Ansible

OSCON 2013 Speaker Series

Automating the configuration management of your operating systems and the rollout of your applications is one of the most important things an administrator or developer can do to avoid surprises when updating services, scaling up, or recovering from failures. However, it’s often not enough. Some of the most common operations that happen in your datacenter (or cloud environment) involve large numbers of machines working together and humans to mediate those processes. While we have been able to remove a lot of human effort from configuration, there has been a lack of software able to handle these higher-level operations.

I used to work for a hosted web application company where the IT process for executing an application update involved locking six people in a room for sometimes 3-4 hours, each person pressing the right buttons at the right time. This process almost always had a glitch somewhere where someone forgot to run the right command or something wasn’t well tested beforehand. While some technical solutions were applied to handle configuration automation, nothing that could perform configuration could really accomplish that high level choreography on top as well. This is why I wrote Ansible.

Ansible is a configuration management, application deployment, and IT orchestration system. One of Ansible’s strong points is having a very simple, human readable language – it allows users very fine, precise control over what happens on what machines at what times.

Getting started

To get started, create an inventory file, for instance, ~/ansible_hosts that defines what machines you are managing, and which machines are frequently organized into groups. Ansible can also pull inventory from multiple cloud sources, but an inventory file is a quick way to get started:

[webservers]
www01.example.com
www02.example.com
# add more webservers here

[monitoring]
nagios1.example.com

[lbservers]
haproxy1.example.com
haproxy2.example.com

Now that you have defined what machines you are managing, you have to define what you are going to do on the remote machines.

Ansible calls this description of processes a “playbook,” and you don’t have to have just one, you could have different playbooks for different kinds of tasks.

Let’s look at an example for describing a rolling update process. This example is somewhat involved because it’s using haproxy, but haproxy is freely available. Ansible also includes modules for dealing with Netscalers and F5 load balancers, so this is just an example — ordinarily you would start more simply and work up to an example like this:
Read more…

Easily invoke common protocols with Twisted

Spin up Python-friendly services with 0 lines of code

Twisted is a framework for writing, testing, and deploying event-driven clients and servers in Python. In my previous Twisted blog post, we explored an architectural overview of Twisted and examples of simple TCP, UDP, SSL, and HTTP echo servers.

While Twisted makes it easy to build servers in just a few lines of Python, you can actually use Twisted to spin up servers with 0 lines of code!

We can accomplish this with twistd (pronounced twist-dee), a command line utility that ships with Twisted for deploying your Twisted applications. In addition to providing a standardized deployment interface for common production features like daemonization, logging, and authentication, twistd can use Twisted’s plugin architecture to run flexible servers for a variety of protocols. Here are some examples:

twistd web --port 8000 --path .

Run an HTTP server on port 8000, serving both static and dynamic content out of
the current working directory. Visit http://localhost:8000 to see the directory listing:

Read more…

What Kind of JavaScript Developer Are You?

Fault lines make conversation difficult

“JavaScript developer” is a description that hides tremendous diversity. While every language has a range of user skill levels, JavaScript has a remarkably fragmented community. People come to JavaScript for different reasons from different places, and this can make communication difficult. Sometimes it’s worse than that—not everyone likes everyone else.

“JavaScript developer” used to mean “web developer,” specifically a developer who spends a lot of time working in the browser. Even as JavaScript became a specialty of its own, most JavaScript developers came through a broader web practice first, learning HTML and CSS before tackling the DOM. This was my path, and is still a common one. It was reasonably easy to absorb JavaScript by example, using it as an object manipulation language before pushing into the harder corners.

Many programmers, however, started on the server-side, building code that filled templates. Server-side JavaScript existed, in the Netscape Enterprise Server, for example, but was a tiny fraction in a world dominated by Perl, Java, Python, Ruby, and ASP’s languages. Developers who spent most of their time writing code that ran on the server worked with a different set of tools and expectations, and had to shift gears as JavaScript became a more critical part of web applications. (Some of these developers generate a lot of JavaScript—JSON is, after all, JavaScript—but would prefer to think of JavaScript’s role in that as an accident.)

Another group of developers came from desktop applications, and expected more direct control over the interface. Many of these developers now understand JavaScript because the browser forces them, not because they want to.

Involuntary JavaScript creates a tremendous amount of tension around the language. Normally, the people who dislike a programming language can just work with something else. JavaScript’s browser dominance makes that hard.

Other developers, though, are much much happier and much deeper into JavaScript. The JavaScript revival that became visible with the rise of Ajax in 2005 gave the language greater credibility. Douglas Crockford’s work “Unearthing the Excellence in JavaScript,” JavaScript: The Good Parts demonstrated a powerful language lurking in a toolset many had considered trivial.

Over the past decade, JavaScript’s power and reach have grown thanks largely to a core group of developers whose focus on the language has created best practices and frameworks, while driving browser vendors to improve their implementations. Node.js emerged from this work, giving JavaScript a unique server-side framework that deeply reflects the language.

Read more…

Twisted Python: The engine of your Internet

Learn to build event-driven client and server applications

I want to build a web server, a mail server, a BitTorrent client, a DNS server, or an IRC bot—clients and servers for a custom protocol in Python. And I want them to be cross-platform, RFC-compliant, testable, and deployable in a standardized fashion. What library should I use?

Use Twisted

Twisted is a “batteries included” networking engine for writing, testing, and deploying event-driven clients and servers in Python. It comes with off-the-shelf support for popular networking protocols like HTTP, IMAP, IRC, SMTP, POP3, IMAP, DNS, FTP, and more.

To see just how easy it is to write networking services using Twisted, let’s run and discuss a simple Twisted TCP echo server:

from twisted.internet import protocol, reactor

class Echo(protocol.Protocol):
    def dataReceived(self, data):
        self.transport.write(data)

class EchoFactory(protocol.Factory):
    def buildProtocol(self, addr):
        return Echo()

reactor.listenTCP(8000, EchoFactory())
reactor.run()

With Twisted installed, if we save this code to echoserver.py and run it with python echoserver.py, clients can now connect to the service on port 8000, send it data, and get back their echoed results. Read more…

Understanding Mojito

Yahoo's Mojito lets you run code where it's easiest.

O'Reilly editor Simon St. Laurent talked with Yahoo's Bruno Fernandez-Ruiz about the possibilities Node opened and Mojito exploits. Yahoo's Mojito is a different kind of framework: all JavaScript, but running on both the client and the server.

The secrets of Node’s success

Why Node.js has caught on while other server-side JavaScript implementations faltered.

What is it about Node.js that makes it interesting to developers? The key factors are performance, timing, and focusing on a real problem that wasn't easily solved with other server-side dynamic languages.