CFEngine is both the oldest and the newest of the popular tools for automating site administration. Mark Burgess invented it as a free software project in 1993, and years later, as deployments in the field outgrew its original design he gave it a complete rethink and developed the powerful concept of promise theory to make it modular and maintainable. In this guise as version 3, CFEngine stands along with two other pieces of free software, Puppet and Chef, as key parts of enterprise computing. Along the way, Burgess also started a commercial venture, CFEngine AS, that maintains both the open source and proprietary versions of CFEngine.
Diego Zamboni has recently taken the position of Senior Security Advisor at CFEngine AS and is writing a book for O’Reilly on CFEngine 3. I talked to him this week about the recent new release of the open source version (3.2.4) in tandem with a new commercial release of CFEngine 3 Nova (version 2.1.3). Here’s are excerpts of what he has written to introduce CFEngine 3.
CFEngine 3 is fine-tuned to the features and design that make it possible to automate very large numbers of systems in a scalable and manageable way. CFEngine 3 is also very lightweight–its binaries normally use less than 30MB of disk space, it requires a single TCP port to communicate among servers and clients, and it has been designed to be very resource-efficient. CFEngine 3 can run on everything from smartphones to supercomputers.
CFEngine 3 is different from many other automation mechanisms in that you do not need to tell it what to do. Instead, you specify the state in which you wish the system to be, and CFEngine 3 will automatically and iteratively decide the actions to take to reach the desired state, or as close to it as possible. Underlying this ability is a powerful theoretical model known as Promise Theory, which was initially developed for CFEngine 3, but which has also found other applications in Computer Science and in other fields such as Economics and Organization.
This allows you to develop building blocks for complex promises that remain readable and manageable because the lower-level components are encapsulated. Each promise represents the desired state of certain parts of the system. At the lowest level, these are some of the things that you can express to CFEngine 3 as desired states:
“Make sure file /foo/bar contains line xyz”
“Make sure user foobar exists/does not exist”
“Make sure process foo is/is not running”
At a higher level of abstraction, you can encapsulate CFEngine 3 operations and express high-level desired states:
“Make sure all web servers have Apache installed”
“Make sure all root accounts have the same, centrally-designated password”
“Make sure parameters EnableDNS and AllowRoot are disabled on all sshd configurations”
And at an even higher level, you can express top-level desired states like these:
“Configure host xyz as a database server”
“Create a new cluster of VMs to use as web servers”
So what’s in the new versions? CFEngine 3 Nova includes:
-
System monitoring extensions, which extend the monitoring capabilities of CFEngine 3 Community (to monitor system state such as CPU load, number of processes and network connections, disk utilization, etc.) to allow for defining custom monitors for any type of information.
-
Support for manipulating virtual machines on Xen, VMware ESX, and KVM.
-
Native Windows support.
-
Flexible searching of reports in a brand new scalable interface that supports thousands of hosts on a single hub.
-
Improved machine learning and anomaly monitoring for diagnostics and capacity planning. Additional sensors have been added to detect operating system performance and behavioral trends, especially on Linux kernels.
-
The NoSQL document-oriented database MongoDB, used instead of MySQL for all storage on Nova’s Mission Portal.
-
Generic JSON return values so that users can customize the interface and JQuery framework of the Mission Portal. This allows direct access to data in a way that makes higher levels of scripting more effective.
CFEngine 3 Community also includes a large number of improvements, all of which are in Nova too:
-
A vastly improved bootstrapping process, which makes it easy to get new CFEngine 3 servers and clients up and running with very little manual configuration.
-
Support for environments, which are a way of grouping hosts according to arbitrary definitions. This makes it very easy to define, for example, “development,” “testing,” and “production” environments for CFEngine 3 policies.
-
The new cf-report command, available in both Community and Nova, which allows extraction of data and generation of reports from the command line. It can produce reports both about the behavior of the current CFEngine 3 environment (policies, hosts, etc.) and about internal information, such as a CFEngine 3 syntax summary.
-
Many performance and concurrency improvements and bug fixes.
-
Several new functions and parsing improvements, including and(), not(), and or() functions, to ease writing of complex class expressions.
-
A new and improved Emacs mode for editing CFEngine 3 policy files.
Velocity Europe, being held Nov. 8-9 in Berlin, will bring together the web operations and performance communities for two days of critical training, best practices, and case studies.