"continuous integration" entries

DevOps for Finance

How DevOps will help you surpass the common challenges of financial services software development.

Download DevOps for Finance.

Download DevOps for Finance.

Download a free copy of DevOps for Finance, an O’Reilly report by Jim Bird for the financial services software insider who’s heard about DevOps, but is unsure whether it represents solution or suicide.

DevOps, until recently, has been a story about unicorns. Innovative, engineering-driven online tech companies like Flickr, Etsy, Twitter, Facebook, and Google. Netflix and its Chaos Monkey. Amazon deploying thousands of changes per day.

DevOps was originally about WebOps at Internet companies working in the Cloud, and a handful of Lean Startups in Silicon Valley. It started at these companies because they had to move quickly, so they found new, simple, and collaborative ways of working that allowed them to innovate much faster and to scale much more effectively than organizations had done before.

But as the velocity of change in business continues to increase, enterprises — sometimes referred to as “horses,” in contrast to the unicorns referenced above — must also move to deliver content and features to customers just as quickly. These large organizations have started to adopt (and, along the way, adapt) DevOps ideas, practices, and tools.

This short book assumes that you have heard about DevOps and want to understand how DevOps practices like Continuous Delivery and Infrastructure as Code can be used to solve problems in financial systems at a trading firm, or a big bank or stock exchange. We’ll look at the following key ideas in DevOps, and how they fit into the world of financial systems:

  • Breaking down the “wall of confusion” between development and operations, and extending Agile practices and values from development to operations
  • Using automated configuration management tools like Chef, Puppet, CFEngine, or Ansible to programmatically provision and configure systems (Infrastructure as Code)
  • Building Continuous Integration and Continuous Delivery (CI/CD) pipelines to automatically test and push out changes, and wiring security and compliance into these pipelines
  • Using containerization and virtualization technologies like Docker and Vagrant, together with Infrastructure as Code, to create IaaS, PaaS, and SaaS clouds
  • Running experiments, creating fast feedback loops, and learning from failure

To follow this book you need to understand a little about these ideas and practices. There is a lot of good stuff about DevOps out there, amid the hype. Read more…

How CI removes the pain

The tradeoffs of (accidentally) discarding continuous integration.

Engineering Practices for Continuous Delivery

I’ve given a continuous delivery workshop a few times with ThoughtWorks Chief Scientist Martin Fowler, who tells an interesting story about continuous integration, from the first software project he ever saw. When Martin was a teenager, his father had a friend who was running a software project, and he gave Martin the nickel (or five pence) tour — a bunch of men, predominately on mainframe terminals, working in an old warehouse. Martin remarked that the thing that struck him the most was when the guide told him that all the developers were “currently integrating all their code.” They had finished coding six months prior, yet despite that they weren’t sure when they were going to be done “integrating.”

That revelation surprised Martin: in his mind, software development was a discreet, scientific, deterministic process, not at all represented by the vague comments of these developers. But the practice of as-late-as-possible integration was common back in an earlier era of software development. If you look at software engineering texts of the ’60s and ’70s, every project included an integration phase. This isn’t how we think of integration today, which happens at the granularity of services and applications. Rather, a common practice was to have developers code in isolation for weeks and months at a time, then integrate all their code together into a cohesive whole. And that phase was, not surprisingly, a painful part of most projects. Yet, that type of 60s and 70s workflow is still codified in some version control tools today, even though we have now determined that late integration is the opposite of how we should approach this problem.

Read more…

Four short links: 14 April 2014

Four short links: 14 April 2014

dategrep, Agile Signoff, Feedback Speed, and Modern Dev

  1. dategrepprint lines matching ranges of dates. Genius!
  2. Business Case Guidance in Agile Projects (gov.uk) — how the UK govt signs off on Agile projects, which normally governments have no clue over how to handle properly.
  3. Hyper Growth Done Right“While I was at Oracle, it took a month before a new engineer would get any code in,” Agarwal says. “It sent this implicit message that it’s okay to take a month to write some code.” First time I’d heard this wise point articulated: slow feedback loops send the message that progress can be slow.
  4. Docker + Github + Jenkins — clever integration of the three tools to get repeatable continuous integration. The modern dev environment has workflow built on git, VMs, and glue.
Four short links: 11 September 2013

Four short links: 11 September 2013

NSA Crypto, Web Traps, Learn by Doing, and Distributed Testing

  1. On the NSA — intelligent unpacking of what the NSA crypto-weakening allegations mean.
  2. Overview of the 2013 OWASP Top 10 — rundown of web evil to avoid. (via Ecryption)
  3. Easy 6502 — teaches 6502 assembler, with an emulator built into the book. This is what programming non-fiction books will look like in the future.
  4. Kochiku — distributing automated test suites for faster validation in continuous integration.
Four short links: 14 February 2012

Four short links: 14 February 2012

Corruption Remains, Automated Instruction, Eolas Victory Incomplete, and CI Arduino Hack

  1. Why I Hate The STOCK Act (Clay Johnson) — an attempt to reform insider trading within government, but because Congress exempts itself from substantial penalties then it has little effect where it’s needed most. We won’t see change on the issues that matter to us (copyright, due process for Internet takedowns, privacy, etc.) while the lawmakers are distracted by money.
  2. Instruction Medium is the Message (Dan Meyer) — Print is a medium. Same as digital photos. Same as a teacher’s voice. Same as a YouTube video. Same as a podcast. These are all different media. And as we know, the medium is the message. The medium defines and constrains and sometimes distorts the message. The math that can be conveyed in a YouTube video is not the same math that can be conveyed in a digital photo or a podcast or a print textbook. Anything that can be replaced by a computer should be; it’s doubtful that successful widespread education consists only of things a computer can replace.
  3. Eolas Patent a Hollow Victory (Simon Phipps) — those who were extorted by the patent troll will go uncompensated, and the loss of one patent leaves their business model still intact. The patent system is extremely broken in the US, it’s a giant cost of doing business, a regulation-created tax that is paid to trolls instead of to the US Government. What idiot supports a tax that doesn’t go to the government? An ethically-corrupted one (see point 1 above).
  4. Monitor your Continuous Integration Server with Traffic Lights and an Arduino — nifty little hardware hack. It’s an example of making physical objects which control or portray virtual systems, and it’s tied into this Continuous Integration trend whereby software changes go live as soon as possible rather than being held off until 2am on the first Thursday of the month, when the IT team come in to manage the rollout of the new code. CI, in turn, is an example of failing early on something small rather than failing later and larger. (via Sandy Mamoli)
Four short links: 23 August 2011

Four short links: 23 August 2011

Piracy of Convenience, Machina Ex Artist, Disaster Art, and CI Conference

  1. Late to Hulu Means More Piracy — more evidence that price isn’t the main reason people pirate. If they can get it legally online in a convenient fashion, they will. If you delay online release, or make it inconvenient, your erstwhile customers will turn to piracy because “it’s illegal” is less important than “it’s convenient”. Welcome to the modern world, Fox, please use the designated bin to dispose of your buggy whips.
  2. Samuel Morse’s Reversal of Fortune (Smithsonian Magazine) — Morse gave up painting entirely, relinquishing the whole career he had set his heart on since college days. No one could dissuade him.“Painting has been a smiling mistress to many, but she has been a cruel jilt to me,” he would write bitterly to Cooper. “I did not abandon her, she abandoned me.” He must attend to one thing at a time, as his father had long ago advised him. The “one thing” henceforth would be his telegraph, the crude apparatus housed in his New York University studio apartment. Later it would be surmised that, had Morse not stopped painting when he did, no successful electromagnetic telegraph would have happened when it did, or at least not a Morse electromagnetic telegraph. (via Courtney Johnston)
  3. Mudbird — beautiful ceramics made from silt and clay revealed in the Christchurch earthquake. When life hands you liquefaction, make art. (via Bridget McKendry)
  4. Jenkins User Conference — first conference on the continuous integration tool that I’m seeing in a lot of places. (via Kohsuke Kawaguchi)
Four short links: 7 July 2011

Four short links: 7 July 2011

C64 Presales, Coding Lessons Learned, Feedback Loops, and Continuous Integration

  1. Commodore 64 PC — gorgeous retro look with fairly zippy modern internals. (via Rob Passarella)
  2. Designing Github for Mac — a retrospective from the author of the excellent Mac client for github. He talks about what he learned and its origins, design, and development. Remember web development in 2004? When you had to create pixel-perfect comps because every element on screen was an image? That’s what developing for Cocoa is. Drawing in code is slow and painful. Images are easier to work with and result in more performant code. Remember these days? This meant my Photoshop files had to be a lot more fleshed out than I’ve been accustomed to in recent years. I usually get about 80% complete in Photoshop (using tons of screenshotting & layer flattening), then jump into code and tweak to completion. But with Cocoa, I ended up fleshing out that last 20% in Photoshop.
  3. Feedback Loops (Wired) — covers startups and products that use feedback loops to help us change our behaviour. The best sort of delivery device “isn’t cognitively loading at all,” he says. “It uses colors, patterns, angles, speed—visual cues that don’t distract us but remind us.” This creates what Rose calls “enchantment.” Enchanted objects, he says, don’t register as gadgets or even as technology at all, but rather as friendly tools that beguile us into action. In short, they’re magical. (via Joshua Porter)
  4. continuous.io — hosted continuous integration. (via Jacob Kaplan-Moss)