"continuous delivery" entries

Continuous Delivery versus Continuous Deployment

Download DevOps for Finance.

Download DevOps for Finance.

Download a free copy of DevOps for Finance, an O’Reilly report by Jim Bird for the financial services software insider who’s heard about DevOps, but is unsure whether it represents solution or suicide.

The DevOps Audit Defense Toolkit tries to make a case to an auditor for Continuous Deployment in a regulated environment: that developers, following a consistent, disciplined process, can safely push changes out automatically to production once the changes pass all of the reviews and automated tests and checks in the CD pipeline.

Continuous Deployment has been made famous at places like Flickr, IMVU (where Eric Ries developed the ideas for the Lean Startup method), and Facebook:

Facebook developers are encouraged to push code often and quickly. Pushes are never delayed and [are] applied directly to parts of the infrastructure. The idea is to quickly find issues and their impacts on the rest of the system and surely [fix] any bugs that would result from these frequent small changes.1

While organizations like Etsy and Wealthfront work hard to make Continuous Deployment safe, it is scary to auditors, to operations managers, and to CTOs like me who have been working in financial technology and understand the risks involved in making changes to a live, business-critical system.

Continuous Deployment requires you to shut down a running application on a server or a virtual machine, load new code, and restart. This isn’t that much of a concern for stateless web applications with pooled connections, where browser users aren’t likely to notice that they’ve been switched to a new environment in Blue-Green deployment.2 There are well-known, proven techniques and patterns for doing this that you can follow with confidence for this kind of situation.

Read more…

DevOps for Finance

How DevOps will help you surpass the common challenges of financial services software development.

Download DevOps for Finance.

Download DevOps for Finance.

Download a free copy of DevOps for Finance, an O’Reilly report by Jim Bird for the financial services software insider who’s heard about DevOps, but is unsure whether it represents solution or suicide.

DevOps, until recently, has been a story about unicorns. Innovative, engineering-driven online tech companies like Flickr, Etsy, Twitter, Facebook, and Google. Netflix and its Chaos Monkey. Amazon deploying thousands of changes per day.

DevOps was originally about WebOps at Internet companies working in the Cloud, and a handful of Lean Startups in Silicon Valley. It started at these companies because they had to move quickly, so they found new, simple, and collaborative ways of working that allowed them to innovate much faster and to scale much more effectively than organizations had done before.

But as the velocity of change in business continues to increase, enterprises — sometimes referred to as “horses,” in contrast to the unicorns referenced above — must also move to deliver content and features to customers just as quickly. These large organizations have started to adopt (and, along the way, adapt) DevOps ideas, practices, and tools.

This short book assumes that you have heard about DevOps and want to understand how DevOps practices like Continuous Delivery and Infrastructure as Code can be used to solve problems in financial systems at a trading firm, or a big bank or stock exchange. We’ll look at the following key ideas in DevOps, and how they fit into the world of financial systems:

  • Breaking down the “wall of confusion” between development and operations, and extending Agile practices and values from development to operations
  • Using automated configuration management tools like Chef, Puppet, CFEngine, or Ansible to programmatically provision and configure systems (Infrastructure as Code)
  • Building Continuous Integration and Continuous Delivery (CI/CD) pipelines to automatically test and push out changes, and wiring security and compliance into these pipelines
  • Using containerization and virtualization technologies like Docker and Vagrant, together with Infrastructure as Code, to create IaaS, PaaS, and SaaS clouds
  • Running experiments, creating fast feedback loops, and learning from failure

To follow this book you need to understand a little about these ideas and practices. There is a lot of good stuff about DevOps out there, amid the hype. Read more…

Towards continuous design

A deep integration across design, development, and operations is critical to digital business success.

cont_cent_620

I just finished reading Thomas Wendt’s wonderful book, Design for Dasein. I recommend it to anyone who practices, or just is interested in, experience design. Wendt’s ideas have profound implications for rethinking and improving our approach to designing experiences. They also have profound implications for how we think about DevOps, and its relationship to design, and how that relationship impacts the nature and purpose of digital business.

Design for Dasein introduces what Wendt calls “phenomenological design thinking.” This is a new approach to design that expands the designer’s attention beyond creating things that people use, to encompass thinking about the ways in which things influence, interact with, and are influenced by how people experience the world. Phenomenological design thinking reflects two key insights about the role of designed objects in peoples’ lives. First, designers create possibilities for use rather than rigid solutions. Wendt cites the example of using an empty coke bottle to hold open a door in an old, crooked apartment. By itself, the bottle wasn’t heavy enough to keep the door from swinging shut, so he filled it with pennies. At that point, the bottle suddenly had three overlapping uses: containing and drinking soda, holding opening one’s bedroom door, and storing spare change. Wendt’s point is that the designer does not entirely control the object’s destiny. That destiny is co-created by the designer and the user.

Read more…

The shape of software architecture

Five things we learned from the O’Reilly Software Architecture Conference 2015.

Last week, I had the opportunity to see the first Software Architecture Conference spring to life after a winter of preparation. Software architects, with or without the official title, swarmed the halls learning from speakers and attendees alike. I count myself among the people who were learning. Many notions about this profession and skill set have become clearer to me and I’m already planning to keep the content coming. I’m also in the early stages of developing out the next Software Architecture Conference (spring 2016).

Within this piece you’ll find my takeaways and lessons learned from the event. I expect these initial impressions to both shape our upcoming exploration of software architecture and be shaped by continued shifts within software architecture.
Read more…

Elvis has left the ivory tower

Pragmatism now rules in team structure, technology, engineering practices, and operational innovation.

Karnakpanorama_b

Ancient history in computer science (2004) provides a gem about the personas that Microsoft envisioned as users of the development environment Visual Studio. They developed three:

  • Mort, the opportunistic developer, likes to create quick-working solutions for immediate problems. He focuses on productivity and learns as needed.
  • Elvis, the pragmatic programmer, likes to create long-lasting solutions addressing the problem domain, and learning while working on the solution.
  • Einstein, the paranoid programmer, likes to create the most efficient solution to a given problem and typically learns in advance before working on the solution.

These designations received a lot of negative press, particularly around the Mort persona, but I want to focus on Einstein and Elvis.

Formerly, software architects exemplified the Einstein persona: isolated from day-to-day development details, focused on building abstractions and frameworks. The isolation is so common that it spawned its own “Ivory Tower Architect” derogatory phrase. But the realities of building systems that scale as fast as the business does invalidates that approach. Now, Elvis, the pragmatic developer, has ascended to architect while simultaneously descending from the Ivory Tower. Modern architects don’t have the luxury of isolation from the gritty realities of software development today. Pragmatism now rules in team structure, technology, engineering practices, and operational innovation because:

Read more…

Four short links: 28 October 2014

Four short links: 28 October 2014

Continuous Delivery, UX Resources, Large-Screen Cellphone Design, and Scalable Sockets

  1. Build Quality Inan e-book collection of Continuous Delivery and DevOps experience reports from the wild. Work in progress, and a collection of accumulated experience in the new software engineering practices can’t be a bad thing.
  2. UX Directory — collection of awesome UX resources.
  3. Designing for Large-Screen Cellphones (Luke Wroblewski) — 
In his analysis of 1,333 observations of smartphones in use, Steven Hoober found about 75% of people rely on their thumb and 49% rely on a one-handed grip to get things done on their phones. On large screens (over four inches) those kinds of behaviors can stretch people’s thumbs well past their comfort zone as they try to reach controls positioned at the top of their device. Design advice to create interactions that don’t strain tendons or gray matter.
  4. fastsocket (Github) — a highly scalable socket and its underlying networking implementation of Linux kernel. With the straight linear scalability, Fastsocket can provide extremely good performance in multicore machines.

DevOps in the enterprise

Follow Nordstrom's journey to continuous delivery and a DevOps culture.

DevOps in Practice CoverWould you open an email with a subject line of DevOps and pants? I’m not sure I would.

Six months ago I sent Rob Cummings an email with exactly that subject and he did. And we can be thankful he opened it, because by doing so, he invited us to look back at the fascinating history of Nordstrom’s implementation of continuous delivery and a “DevOps culture.”

The story begins in 2004, in a different era of web operations and performance. Back then, Rob and his team drove out to the colocation facility to deploy the e-commerce site. It was an era in which everything was a bit more heavyweight and things moved a bit slower. But that was OK, because most companies were still figuring the web out, almost as much as users were trying to figure it out.

Then the world started changing. Customer expectations changed. The business’ expectations changed. Heck, even developer expectations changed. As a leader in Nordstrom’s operations department, Rob had to adapt. And all of this was complicated by the fact that the increased pace was starting to strain his team and the systems he was responsible for maintaining. Read more…

Working like a startup at IBM

How a small and passionate team used modern techniques to shift a business on a short timeline.

http://commons.wikimedia.org/wiki/File:PSM_V04_D595_Blue_yellow_color_mix_experiment.jpg

Over the past year, I assisted in creating an application that helped shift a major part of IBM to a software-as-a-service (SaaS) model. I did this with the help of a small but excellent development team that was inspired by the culture and practices of web startups. To be clear, it wasn’t easy – changing how we worked led to frequent friction and conflict – but in the end it worked, and we made a difference.

In mid-2013, the IBM Service Management business and engineering leaders decided to make a big bet on moving our software to the cloud. Traditionally we have sold “on premises” software products. These are software products that a customer buys, downloads, and installs on their own equipment, in their own data centers and facilities. Although we love the on-premises business, we realized that cloud delivery of software is also a great option, and as our customers evolved to a hybrid on-premises / cloud future, we needed to be there to help them.

Read more…

Exploring lightweight monitoring systems

Toward unifying customer behavior and operations metrics.

lightweight_systemsFor the last ten years I’ve had a foot in both the development and operations worlds. I stumbled into the world of IT operations as a result of having the most UNIX skills in the team shortly after starting at ThoughtWorks. I was fortunate enough to do so at a time when many of my ThoughtWorks colleagues and I where working on the ideas which were captured so well in Jez Humble and Dave Farley’s Continuous Delivery (Addison-Wesley).

During this time, our focus was on getting our application into production as quickly as possible. We were butting up against the limits of infrastructure automation and IaaS providers like Amazon were only in their earliest form.

Recently, I have spent time with operations teams who are most concerned with the longer-term challenges of looking after increasingly complex ecosystems of systems. Here the focus is on immediate feedback and knowing if they need to take action. At a certain scale, complex IT ecosystems can seem to exhibit emergent behavior, like an organism. The operations world has evolved a series of tools which allow these teams to see what’s happening *right now* so we can react, keep things running, and keep people happy.

At the same time, those of us who spend time thinking about how to quickly and effectively release our applications have become preoccupied with wanting to know if that software does what our customers want once it gets released. The Lean Startup movement has shown us the importance of putting our software in front of our customers, then working out how they actually use it so we can determine what to do next. In this world, I was struck by the shortcomings of the tools in this space. Commonly used web analytics tools, for example, might only help me understand tomorrow how my customers used my site today.

Read more…