In the open source and free software movement, we always exalt community, and say the people coding and supporting the software are more valuable than the software itself. Few communities have planned and philosophized as much about community-building as ZeroMQ. In the following posting, Pieter Hintjens quotes from his book ZeroMQ, talking about how he designed the community that works on this messaging library.
How to Make Really Large Architectures (excerpted from ZeroMQ by Pieter Hintjens)
There are, it has been said (at least by people reading this sentence out loud), two ways to make really large-scale software. Option One is to throw massive amounts of money and problems at empires of smart people, and hope that what emerges is not yet another career killer. If you’re very lucky and are building on lots of experience, have kept your teams solid, and are not aiming for technical brilliance, and are furthermore incredibly lucky, it works.
But gambling with hundreds of millions of others’ money isn’t for everyone. For the rest of us who want to build large-scale software, there’s Option Two, which is open source, and more specifically, free software. If you’re asking how the choice of software license is relevant to the scale of the software you build, that’s the right question.
The brilliant and visionary Eben Moglen once said, roughly, that a free software license is the contract on which a community builds. When I heard this, about ten years ago, the idea came to me—Can we deliberately grow free software communities?
Ten years later, the answer is “yes,” and there is almost a science to it. I say “almost” because we don’t yet have enough evidence of people doing this deliberately with a documented, reproducible process. It is what I’m trying to do with Social Architecture. ØMQ came after Wikidot, after the Digital Standards Organization (Digistan) and after the Foundation for a Free Information Infrastructure (aka the FFII, an NGO that fights against software patents). This all came after a lot of less successful community projects like Xitami and Libero. My main takeaway from a long career of projects of every conceivable format is: if you want to build truly large-scale and long-lasting software, aim to build a free software community.
Psychology of Software Architecture
Dirkjan Ochtman pointed me to Wikipedia’s definition of Software Architecture as “the set of structures needed to reason about the system, which comprise software elements, relations among them, and properties of both”. For me this vapid and circular jargon is a good example of how miserably little we understand what actually makes a successful large scale software architecture.
Architecture is the art and science of making large artificial structures for human use. If there is one thing I’ve learned and applied successfully in 30 years of making larger and larger software systems, it is this: software is about people. Large structures in themselves are meaningless. It’s how they function for human use that matters. And in software, human use starts with the programmers who make the software itself.
The core problems in software architecture are driven by human psychology, not technology. There are many ways our psychology affects our work. I could point to the way teams seem to get stupider as they get larger or when they have to work across larger distances. Does that mean the smaller the team, the more effective? How then does a large global community like ØMQ manage to work successfully?
The ØMQ community wasn’t accidental. It was a deliberate design, my contribution to the early days when the code came out of a cellar in Bratislava. The design was based on my pet science of “Social Architecture”, which defines as “the conscious design of an environment that encourages a desired range of social behaviors leading towards some goal or set of goals.” I define this as more specifically as “the process, and the product, of planning, designing, and growing an online community.”
One of the tenets of Social Architecture is that how we organize is more significant than who we are. The same group, organized differently, can produce wholly different results. We are like peers in a ØMQ network, and our communication patterns have a dramatic impact on our performance. Ordinary people, well connected, can far outperform a team of experts using poor patterns. If you’re the architect of a larger ØMQ application, you’re going to have to help others find the right patterns for working together. Do this right, and your project can succeed. Do it wrong, and your project will fail.
The two most important psychological elements are that we’re really bad at understanding complexity and that we are so good at working together to divide and conquer large problems. We’re highly social apes, and kind of smart, but only in the right kind of crowd.