"CD" entries
Continuous Delivery versus Continuous Deployment
Download DevOps for Finance.
The DevOps Audit Defense Toolkit tries to make a case to an auditor for Continuous Deployment in a regulated environment: that developers, following a consistent, disciplined process, can safely push changes out automatically to production once the changes pass all of the reviews and automated tests and checks in the CD pipeline.
Continuous Deployment has been made famous at places like Flickr, IMVU (where Eric Ries developed the ideas for the Lean Startup method), and Facebook:
Facebook developers are encouraged to push code often and quickly. Pushes are never delayed and [are] applied directly to parts of the infrastructure. The idea is to quickly find issues and their impacts on the rest of the system and surely [fix] any bugs that would result from these frequent small changes.1
While organizations like Etsy and Wealthfront work hard to make Continuous Deployment safe, it is scary to auditors, to operations managers, and to CTOs like me who have been working in financial technology and understand the risks involved in making changes to a live, business-critical system.
Continuous Deployment requires you to shut down a running application on a server or a virtual machine, load new code, and restart. This isn’t that much of a concern for stateless web applications with pooled connections, where browser users aren’t likely to notice that they’ve been switched to a new environment in Blue-Green deployment.2 There are well-known, proven techniques and patterns for doing this that you can follow with confidence for this kind of situation.
DevOps for Finance
How DevOps will help you surpass the common challenges of financial services software development.
Download DevOps for Finance.
Download a free copy of DevOps for Finance, an O’Reilly report by Jim Bird for the financial services software insider who’s heard about DevOps, but is unsure whether it represents solution or suicide.
DevOps, until recently, has been a story about unicorns. Innovative, engineering-driven online tech companies like Flickr, Etsy, Twitter, Facebook, and Google. Netflix and its Chaos Monkey. Amazon deploying thousands of changes per day.
DevOps was originally about WebOps at Internet companies working in the Cloud, and a handful of Lean Startups in Silicon Valley. It started at these companies because they had to move quickly, so they found new, simple, and collaborative ways of working that allowed them to innovate much faster and to scale much more effectively than organizations had done before.
But as the velocity of change in business continues to increase, enterprises — sometimes referred to as “horses,” in contrast to the unicorns referenced above — must also move to deliver content and features to customers just as quickly. These large organizations have started to adopt (and, along the way, adapt) DevOps ideas, practices, and tools.
This short book assumes that you have heard about DevOps and want to understand how DevOps practices like Continuous Delivery and Infrastructure as Code can be used to solve problems in financial systems at a trading firm, or a big bank or stock exchange. We’ll look at the following key ideas in DevOps, and how they fit into the world of financial systems:
- Breaking down the “wall of confusion” between development and operations, and extending Agile practices and values from development to operations
- Using automated configuration management tools like Chef, Puppet, CFEngine, or Ansible to programmatically provision and configure systems (Infrastructure as Code)
- Building Continuous Integration and Continuous Delivery (CI/CD) pipelines to automatically test and push out changes, and wiring security and compliance into these pipelines
- Using containerization and virtualization technologies like Docker and Vagrant, together with Infrastructure as Code, to create IaaS, PaaS, and SaaS clouds
- Running experiments, creating fast feedback loops, and learning from failure
To follow this book you need to understand a little about these ideas and practices. There is a lot of good stuff about DevOps out there, amid the hype. Read more…
The cloud-native future
Moving beyond ad-hoc automation to take advantage of patterns that deliver predictable capabilities.
Can you release new features to your customers every week? Every day? Every hour? Do new developers deploy code on their first day, or even during job interviews? Can you sleep soundly after a new hire’s deployment knowing your applications are all running perfectly fine? A rapid release cadence with the processes, tools, and culture that support the safe and reliable operation of cloud-native applications has become the key strategic factor for software-driven organizations who are shipping software faster with reduced risk. When you are able to release software more rapidly, you get a tighter feedback loop that allows you to respond more effectively to the needs of customers.
Continuous delivery is why software is becoming cloud-native: shipping software faster to reduce the time of your feedback loop. DevOps is how we approach the cultural and technical changes required to fully implement a cloud-native strategy. Microservices is the software architecture pattern used most successfully to expand your development and delivery operations and avoid slow, risky, monolithic deployment strategies. It’s difficult to succeed, for example, with a microservices strategy when you haven’t established a “fail fast” and “automate first” DevOps culture.
Continuous delivery, DevOps, and microservices describe the why, how, and what of being cloud-native. These competitive advantages are quickly becoming the ante to play the software game. In the most advanced expression of these concepts they are intertwined to the point of being inseparable. This is what it means to be cloud-native.
A software engineer’s role traversal
Software engineer and author Jason Myers on changing roles in a changing market.
We often hear about how the tech job market is booming and has space for newcomers, but what does that mean for the developers already in the market? In December of 2014, Fortune.com predicted that 2015 would be an excellent year for developers to change jobs. Citing Dice.com, they note that jobs are popping up all over the country. In fact, Dice’s survey also reports 40% of hiring managers seeing voluntary departures, a higher number than was seen just six months earlier.
These are all large, general numbers. What does a job change, and the changing market, look like for individual developers? To get a better sense, I spoke with Jason Myers, who is working on our upcoming Essential SQLAlchemy, 2e title. Jason recently went from working for the email marketing service Emma, Inc., to working for networking giant Cisco. Here, he talks about how a change like that feels, and how the market looks to him.
What is DevOps (yet again)?
Empathy, communication, and collaboration across organizational boundaries.
I might try to define DevOps as the movement that doesn’t want to be defined. Or as the movement that wants to evade the inevitable cargo-culting that goes with most technical movements. Or the non-movement that’s resisting becoming a movement. I’ve written enough about “what is DevOps” that I should probably be given an honorary doctorate in DevOps Studies.
Baron Schwartz (among others) thinks it’s high time to have a definition, and that only a definition will save DevOps from an identity crisis. Without a definition, it’s subject to the whims of individual interest groups, and ultimately might become a movement that’s defined by nothing more than the desire to “not be like them.” Dave Zwieback (among others) says that the lack of a definition is more of a blessing than a curse, because it “continues to be an open conversation about making our organizations better.” Both have good points. Is it possible to frame DevOps in a way that preserves the openness of the conversation, while giving it some definition? I think so.
DevOps started as an attempt to think long and hard about the realities of running a modern web site, a problem that has only gotten more difficult over the years. How do we build and maintain critical sites that are increasingly complex, have stringent requirements for performance and uptime, and support thousands or millions of users? How do we avoid the “throw it over the wall” mentality, in which an operations team gets the fallout of the development teams’ bugs? How do we involve developers in maintenance without compromising their ability to release new software?
The DevOps identity crisis
Why DevOps needs a manifesto after all, but may never get one.
DevOps is everywhere! The growth and mindshare of the movement is remarkable. But if you care deeply about DevOps, you might agree with me when I say that although its moment has “arrived,” DevOps is in serious trouble. The movement is fragmented and weakly defined, and is being usurped by those who care more about short-term opportunities than the long-term viability of DevOps.
They are the ninety-nine percent, and nobody cares
How bad could it be? Travel back in time. It is mid-November 2011, and Occupy Wall Street is occupying the headlines. One of the major reasons is that the protestors are targeting shipping ports on the West Coast, causing shutdowns and even violence. As things are getting out of hand, parts of the movement start condemning these actions as counter-productive, hurting the 99% instead of the intended 1%. Spokespeople for the movement are quoted in the media as saying the instigators “don’t represent the movement.”
Why did the Occupy movement become a footnote in history so fast? There were several reasons: there was no cohesive agreement on its identity, values, goals, and mission; in an effort to be unlike “them,” the OWS proponents avoided anything that looked like centralized leadership; and it seemed to be entirely negative, advocating nothing to replace what it wanted to remove.
I believe a similar thing is happening to DevOps right now, for many of the same reasons. Let’s talk about some of these problems.
DevOps in the enterprise
Follow Nordstrom's journey to continuous delivery and a DevOps culture.
Would you open an email with a subject line of DevOps and pants? I’m not sure I would.
Six months ago I sent Rob Cummings an email with exactly that subject and he did. And we can be thankful he opened it, because by doing so, he invited us to look back at the fascinating history of Nordstrom’s implementation of continuous delivery and a “DevOps culture.”
The story begins in 2004, in a different era of web operations and performance. Back then, Rob and his team drove out to the colocation facility to deploy the e-commerce site. It was an era in which everything was a bit more heavyweight and things moved a bit slower. But that was OK, because most companies were still figuring the web out, almost as much as users were trying to figure it out.
Then the world started changing. Customer expectations changed. The business’ expectations changed. Heck, even developer expectations changed. As a leader in Nordstrom’s operations department, Rob had to adapt. And all of this was complicated by the fact that the increased pace was starting to strain his team and the systems he was responsible for maintaining. Read more…
Working like a startup at IBM
How a small and passionate team used modern techniques to shift a business on a short timeline.
Over the past year, I assisted in creating an application that helped shift a major part of IBM to a software-as-a-service (SaaS) model. I did this with the help of a small but excellent development team that was inspired by the culture and practices of web startups. To be clear, it wasn’t easy – changing how we worked led to frequent friction and conflict – but in the end it worked, and we made a difference.
In mid-2013, the IBM Service Management business and engineering leaders decided to make a big bet on moving our software to the cloud. Traditionally we have sold “on premises” software products. These are software products that a customer buys, downloads, and installs on their own equipment, in their own data centers and facilities. Although we love the on-premises business, we realized that cloud delivery of software is also a great option, and as our customers evolved to a hybrid on-premises / cloud future, we needed to be there to help them.