"devops" entries

Four short links: 7 October 2015

Four short links: 7 October 2015

Time for Change, Face Recognition, Correct Monitoring, and Surveillance Infrastructure

  1. The Uncertain Future of Emotion AnalyticsA year before the launch of the first mass-produced personal computer, British academic David Collingridge wrote in his book “The Social Control of Technology” that “when change is easy, the need for it cannot be foreseen; when the need for change is apparent, change has become expensive, difficult, and time consuming.”
  2. Automatic Face Recognition (Bruce Schneier) — Without meaningful regulation, we’re moving into a world where governments and corporations will be able to identify people both in real time and backwards in time, remotely and in secret, without consent or recourse.
  3. Really Monitoring Your SystemsIf you are not measuring and showing the maximum value, then you are hiding something. The number one indicator you should never get rid of is the maximum value. That’s not noise — it’s the signal; the rest is noise.
  4. Haunted by Data (Maciej Ceglowski) — You can’t just set up an elaborate surveillance infrastructure and then decide to ignore it. These data pipelines take on an institutional life of their own, and it doesn’t help that people speak of the “data-driven organization” with the same religious fervor as a “Christ-centered life.”
Comment
Four short links: 5 October 2015

Four short links: 5 October 2015

Semantic Sensors, Broadening "Sensor," Moving Fast, and Presidential Campaigns

  1. Semantic Sensors (Pete Warden) — tiny, cheap, all-in-one modules that capture raw noisy data from the real world, have built-in AI for analysis, and only output a few high-level signals.
  2. What if People Were Sensors, Not Things To Be Sensed? (Cory Doctorow) — Even in the Internet of Allegedly Free Things, humans and comput­ers are adversaries. Medical telemetry and implant companies envision selling shockingly intimate facts about your body’s internal workings to data-mining services and insurers. Car companies see their vehicles as platforms for gathering data on your driving, on traffic patterns, and on the sense-able facts of the streets you pass by, to sell it to, you guessed it, data-mining companies and insurers. John Deere has argued that its tractors are copyrighted works, and that it, not the farmers, own the soil-density data collected by the torque sensors on the wheels (it sells this data to Monsanto, which charges farmers for the right to know about it).
  3. Move Fast and Break Nothing (Zach Holman) — the first step is identifying what you cannot break.
  4. I’m Trying to Run for President But the Democrats Won’t Let Me (Larry Lessig) — A “democracy” in which 400 families give 50% of the money in campaigns is not American democracy. It is a banana republic democracy.
Comment
Four short links: 2 October 2015

Four short links: 2 October 2015

Automatic Environments, Majority Illusion, Bogus Licensing, and Orchestrating People and Machines

  1. Announcing Otto — new Hashicorp tool that automatically builds development environments without any configuration; it can detect your project type and has built-in knowledge of industry-standard tools to setup a development environment that is ready to go. When you’re ready to deploy, Otto builds and manages an infrastructure, sets up servers, builds, and deploys the application.
  2. The Majority Illusion in Social Networks (arxiv) — if connectors do something, it’s perceived as more popular than if the same number of “unpopular” people in the social graph do it. (via MIT TR)
  3. Scientist Says Researcher in Immigrant-Friendly Countries Can’t Use His Software — software to build phylogenetic trees, but the author’s a loon. It’s another sign that it’s unwise to do science with non-free software.
  4. Orchestraan open source system to orchestrate teams of experts and machines on complex projects.
Comment

Continuous Delivery versus Continuous Deployment

Download a free copy of DevOps for Finance, an O’Reilly report by Jim Bird for the financial services software insider who’s heard about DevOps, but is unsure whether it represents solution or suicide.

The DevOps Audit Defense Toolkit tries to make a case to an auditor for Continuous Deployment in a regulated environment: that developers, following a consistent, disciplined process, can safely push changes out automatically to production once the changes pass all of the reviews and automated tests and checks in the CD pipeline.

Continuous Deployment has been made famous at places like Flickr, IMVU (where Eric Ries developed the ideas for the Lean Startup method), and Facebook:

Facebook developers are encouraged to push code often and quickly. Pushes are never delayed and [are] applied directly to parts of the infrastructure. The idea is to quickly find issues and their impacts on the rest of the system and surely [fix] any bugs that would result from these frequent small changes.1

While organizations like Etsy and Wealthfront work hard to make Continuous Deployment safe, it is scary to auditors, to operations managers, and to CTOs like me who have been working in financial technology and understand the risks involved in making changes to a live, business-critical system.

Continuous Deployment requires you to shut down a running application on a server or a virtual machine, load new code, and restart. This isn’t that much of a concern for stateless web applications with pooled connections, where browser users aren’t likely to notice that they’ve been switched to a new environment in Blue-Green deployment.2 There are well-known, proven techniques and patterns for doing this that you can follow with confidence for this kind of situation.

Read more…

Comment

DevOps for Finance

How DevOps will help you surpass the common challenges of financial services software development.

Download a free copy of DevOps for Finance, an O’Reilly report by Jim Bird for the financial services software insider who’s heard about DevOps, but is unsure whether it represents solution or suicide.

DevOps, until recently, has been a story about unicorns. Innovative, engineering-driven online tech companies like Flickr, Etsy, Twitter, Facebook, and Google. Netflix and its Chaos Monkey. Amazon deploying thousands of changes per day.

DevOps was originally about WebOps at Internet companies working in the Cloud, and a handful of Lean Startups in Silicon Valley. It started at these companies because they had to move quickly, so they found new, simple, and collaborative ways of working that allowed them to innovate much faster and to scale much more effectively than organizations had done before.

But as the velocity of change in business continues to increase, enterprises — sometimes referred to as “horses,” in contrast to the unicorns referenced above — must also move to deliver content and features to customers just as quickly. These large organizations have started to adopt (and, along the way, adapt) DevOps ideas, practices, and tools.

This short book assumes that you have heard about DevOps and want to understand how DevOps practices like Continuous Delivery and Infrastructure as Code can be used to solve problems in financial systems at a trading firm, or a big bank or stock exchange. We’ll look at the following key ideas in DevOps, and how they fit into the world of financial systems:

  • Breaking down the “wall of confusion” between development and operations, and extending Agile practices and values from development to operations
  • Using automated configuration management tools like Chef, Puppet, CFEngine, or Ansible to programmatically provision and configure systems (Infrastructure as Code)
  • Building Continuous Integration and Continuous Delivery (CI/CD) pipelines to automatically test and push out changes, and wiring security and compliance into these pipelines
  • Using containerization and virtualization technologies like Docker and Vagrant, together with Infrastructure as Code, to create IaaS, PaaS, and SaaS clouds
  • Running experiments, creating fast feedback loops, and learning from failure

To follow this book you need to understand a little about these ideas and practices. There is a lot of good stuff about DevOps out there, amid the hype. Read more…

Comment
Four short links: 8 September 2015

Four short links: 8 September 2015

Serverless Microservers, Data Privacy, NAS Security, and Mobile Advertising

  1. Microservices Without the Servers (Amazon) — By “serverless,” we mean no explicit infrastructure required, as in: no servers, no deployments onto servers, no installed software of any kind. We’ll use only managed cloud services and a laptop. The diagram below illustrates the high-level components and their connections: a Lambda function as the compute (“backend”) and a mobile app that connects directly to it, plus Amazon API Gateway to provide an HTTP endpoint for a static Amazon S3-hosted website.
  2. Privacy vs Data Science — claims Apple is having trouble recruiting top-class machine learning talent because of the strict privacy-driven limits on data retention (Siri data: 6 months, Maps: 15 minutes). As a consequence, Apple’s smartphones attempt to crunch a great deal of user data locally rather than in the cloud.
  3. NAS Backdoors — firmware in some Seagate NAS drives is very vulnerable. It’s unclear whether these are Seagate-added, or came with third-party bundled software. Coming soon to lightbulbs, doors, thermostats, and all your favorite inanimate objects. (via BetaNews)
  4. Most Consumers Wouldn’t Pay Publishers What It Would Take to Replace Mobile Ad Income — they didn’t talk to this consumer.
Comment
Four short links: 31 August 2015

Four short links: 31 August 2015

Linux Security Checklist, Devops for Water Bags, Summarising Reviews, and Exoskeleton with BMI

  1. Linux Workstation Security ChecklistThis is a set of recommendations used by the Linux Foundation for their systems administrators.
  2. Giant Bags of Mostly Water (PDF) — on securing systems that are used by humans. This is what DevOps is about: running Ops like you’re Developing an app, not letting your devs run your ops.
  3. Mining and Summarising Customer Reviews (Paper a Day) — redux of a 2004 paper on sentiment extraction from reviews.
  4. Brain-Machine-Interface for Exoskeleton — no need to worry about the “think of sex every seven seconds” trope, the new system allows users to move forwards, turn left and right, sit and stand simply by staring at one of five flickering LEDs.
Comment

Coding in a cloud-based enterprise

Mapping the future of development by designing for distributed architectures.

departure-platform_620

With the advent of DevOps and various Platform-as-a-Service (PaaS) environments, many complex business requirements need to be met within a much shorter timeframe. The Internet of Things (IoT) is also changing how established applications and infrastructures are constructed. As a result of these converging trends, the enterprise IT landscape is becoming increasingly distributed, and the industry is starting to map how all the various components — from networking and middleware platforms, to ERP systems and microservices — will come together to create a new development paradigm that exists solely in the cloud.

Read more…

Comment

Towards continuous design

A deep integration across design, development, and operations is critical to digital business success.

cont_cent_620

I just finished reading Thomas Wendt’s wonderful book, Design for Dasein. I recommend it to anyone who practices, or just is interested in, experience design. Wendt’s ideas have profound implications for rethinking and improving our approach to designing experiences. They also have profound implications for how we think about DevOps, and its relationship to design, and how that relationship impacts the nature and purpose of digital business.

Design for Dasein introduces what Wendt calls “phenomenological design thinking.” This is a new approach to design that expands the designer’s attention beyond creating things that people use, to encompass thinking about the ways in which things influence, interact with, and are influenced by how people experience the world. Phenomenological design thinking reflects two key insights about the role of designed objects in peoples’ lives. First, designers create possibilities for use rather than rigid solutions. Wendt cites the example of using an empty coke bottle to hold open a door in an old, crooked apartment. By itself, the bottle wasn’t heavy enough to keep the door from swinging shut, so he filled it with pennies. At that point, the bottle suddenly had three overlapping uses: containing and drinking soda, holding opening one’s bedroom door, and storing spare change. Wendt’s point is that the designer does not entirely control the object’s destiny. That destiny is co-created by the designer and the user.

Read more…

Comment
Four short links: 25 August 2015

Four short links: 25 August 2015

Microservices Anti-Patterns, Reverse Engineering Course, Graph Language, and Automation Research

  1. Seven Microservices Anti-PatternsOne common mistake people made with SOA was misunderstanding how to achieve the reusability of services. Teams mostly focused on technical cohesion rather than functional regarding reusability. For example, several services functioned as a data access layer (ORM) to expose tables as services; they thought it would be highly reusable. This created an artificial physical layer managed by a horizontal team, which caused delivery dependency. Any service created should be highly autonomous – meaning independent of each other.
  2. CSCI 4974 / 6974 Hardware Reverse Engineering — RPI CS course in reverse engineering.
  3. The Gremlin Graph Traversal Language (Slideshare) — preso on a language for navigating graph data structures, which is part of the Apache TinkerPop (“Open Source Graph Computing”) suite.
  4. Why Are There Still So Many Jobs? The History and Future of Workplace Automation (PDF) — paper about the history of technology and labour. The issue is not that middle-class workers are doomed by automation and technology, but instead that human capital investment must be at the heart of any long-term strategy for producing skills that are complemented by rather than substituted for by technological change. Found via Scott Santens’s comprehensive rebuttal.
Comment