How to create a Swarm cluster with Docker

Using Docker Machine to create a Swarm cluster across cloud providers.

Editor’s note: this is an Early Release excerpt from Chapter 7 of Docker Cookbook by Sébastien Goasguen. The recipes in this book will help developers go from zero knowledge to distributed applications packaged and deployed within a couple of chapters. One of the key value propositions of Docker is app portability. The following will show you how to use Docker Machine to create a Swarm cluster across cloud providers.

Problem

You understand how to create a Swarm cluster manually (see Recipe 7.3), but you would like to create one with nodes in multiple public Cloud Providers and keep the UX experience of the local Docker CLI.

Solution

Use Docker Machine to start Docker hosts in several Cloud providers and bootstrap them automatically to create a swarm cluster.

Warning

This is an experimental feature in Docker Machine and is subject to change.

The first thing to do is to obtain a swarm discovery token. This will be used during the bootstrapping process when starting the nodes of the cluster. As explained in Recipe 7.3, swarm features multiple discovery process. In this recipe, we used the service hosted by Docker, Inc. A discovery token is obtained by running a container based on the swarm image and running the create command. Assuming we do not have access to a Docker host already, we use docker-machine to create one solely for this purpose.

With the token in hand, we can use docker-machine and multiple public Cloud drivers to start worker nodes. We can start a swarm head node on VirtualBox, a worker on DigitalOcean and another one on Azure.

Tip

Do not start a swarm head in a public cloud and a worker on your localhost with VirtualBox. Chances are the head will not be able to route network traffic to your local worker node. It is possible to do, but you would have to open ports on your local router.

Your swarm cluster is now ready. Your swarm head node is running locally in a Virtualbox VM, one worker node is running in DigitalOcean and another one in Azure. You can set the local docker-machine binary to use the head node running in VirtualBox and start using the swarm subcommands:

Discussion

If you start a container, swarm will schedule it in round-robin fashion on the cluster. For example, starting three nginx container in a for loop with:

Will lead to three nginx container on the three nodes in your cluster. Remember that you will need to open port 80 on the instances running in the Cloud to access the container.

Tip

Do not forget to remove the machine you started in the Cloud.

See Also

  • Using Docker machine with Docker swarm.


Editor’s note: if you’re interested in learning more about networking at scale, you’ll want to check out Jay Edwards’ Distributed Systems training session at Velocity in Santa Clara May 27-29, 2015.

tags: , , , , , , , ,