"testing" entries

Avoid disruption through exploration

Support experimentation and continuously evaluate to stay ahead.


Businesses have always come and gone, but these days it seems that companies can fall from market dominance to bankruptcy in the blink of an eye. Kodak, Blockbuster and HMV are just a few recent victims of the rapid market disruption that defines the current era.

Where did these once iconic companies go wrong? To my mind, they forgot to keep challenging their assumptions about what business they were actually in.

Businesses have two options when they plan for the road ahead: they can put all their eggs into one basket, and risk losing everything if that basket has a hole in the bottom, or they can make a number of small bets, accepting that some will fail while others succeed.

Taking the latter approach, and making many small bets on innovation, transforms the boardroom into a roulette table. Unlike a punter in a casino, however, businesses cannot afford to stop making bets.

Business models are transient and prone to disruption by changes in markets and the external competitive environment, advances in design and technology, and wider social and economic change. Organizations that misjudge their purpose, or cannot sense and then adapt to these changes, will perish.

The sad truth is that too many established organizations focus most of their time and resources on executing and optimizing their existing business models in order to maximize profits. They forget to experiment and explore new ideas for customer needs of tomorrow.

Read more…

Four short links: 22 January 2015

Four short links: 22 January 2015

MSVR, The Facebook, Social Robots, and Testing Microservices

  1. Microsoft HoloLens Goggles (Wired) — a media release about the next thing from the person behind Kinect. I’m still trying to figure out (as are investors, I’m sure) where in the hype curve this Googles/AR/etc. amalgam lives. Is it only a tech proof-of-concept? Is it a games device like Kinect? Is it good and cheap enough for industrial apps? Or is this the long-awaited climb out of irrelevance for Virtual Reality?
  2. The Facebook (YouTube) — brilliant fake 1995 ad for The Facebook. Excuse me, I’m off to cleanse.
  3. Natural Language in Social Robotics (Robohub) — Natural language interfaces are turning into a de-facto interface convention. Just like the GUI overlapped and largely replaced the command line, NLP is now being used by robots, the Internet of things, wearables, and especially conversational systems like Apple’s Siri, Google’s Now, Microsoft’s Cortana, Nuance’s Nina, Amazon’s Echo and others. These interfaces are designed to simplify, speed up, and improve task completion. Natural language interaction with robots, if anything, is an interface. It’s a form of UX that requires design.
  4. Microservices and Testing (Martin Fowler) — testing across component boundaries, in the face of failing data stores and HTTP timeouts. The first discussion of testing in a web-scale world that I’ve seen from The Mainstream.
Four short links: 20 October 2014

Four short links: 20 October 2014

Leaky Search, Conditional Javascript, Software Proofs, and Fake Identity

  1. Fix Mac OS Xeach time you start typing in Spotlight (to open an application or search for a file on your computer), your local search terms and location are sent to Apple and third parties (including Microsoft) under default settings on Yosemite (10.10). See also Net Monitor, an open source toolkit for finding phone-home behaviour.
  2. A/B Testing at Netflix (ACM) — Using a combination of static analysis to build a dependency tree, which is then consumed at request time to resolve conditional dependencies, we’re able to build customized payloads for the millions of unique experiences across Netflix.com.
  3. Leslie Lamport Interview SummaryOne idea about formal specifications that Lamport tries to dispel is that they require mathematical capabilities that are not available to programmers: “The mathematics that you need in order to write specifications is a lot simpler than any programming language […] Anyone who can write C code, should have no trouble understanding simple math, because C code is a hell of a lot more complicated than” first-order logic, sets, and functions. When I was at uni, profs worked on distributed data, distributed computation, and formal correctness. We have the first two, but so much flawed software that I can only dream of the third arriving.
  4. Fake Identity — generate fake identity data when testing systems.
Four short links: 18 September 2014

Four short links: 18 September 2014

Writing Testable Code, Magical UIs, High-Performance ssh, and BASIC Lessons

  1. Guide to Writing Testable Code (PDF) — Google’s testable code suggestions, though C++-centric.
  2. Enchanted Objects (YouTube) — David Rose at Google talking about the UX of magical UIs. (via Mary Treseler)
  3. hpn-sshHigh Performance SSH/SCP.
  4. Lost Lessons from an 8-bit BASICThe little language that fueled the home computer revolution has been long buried beneath an avalanche of derision, or at least disregarded as a relic from primitive times. That’s too bad, because while the language itself has serious shortcomings, the overall 8-bit BASIC experience has high points that are worth remembering.
Comments: 2
Four short links: 22 August 2014

Four short links: 22 August 2014

Crowd Problems, Robot Butler, Opportunistic Encryption, and A/B Framework

  1. Blame the Crowd, Not the Camera (Nina Simon) — Cameras weaponize an already unwieldy mob of people.
  2. The Botlr — the Cupertino Starwood hotel has a robot butler (botlr) doing room service.
  3. tcpcrypt — opportunistic encryption of all network traffic.
  4. Sixpack — language-agnostic A/B testing framework.


Variations in Test-Driven Development

london_traffic_lights“Red-Green-Refactor” is a familiar slogan from test-driven development (TDD), describing a popular approach to writing software. It’s been both popular and controversial since the 2000’s (see the recent heated discussions between David Hansson, Bob Martin, and others). I find that it’s useful but limiting. Here I’ll describe some interesting exceptions to the rule, which have expanded the way I think about tests.

The standard three-step cycle goes like this. After choosing a small improvement, which can be either a feature or a bug fix, you add a failing test which shows that the improvement is missing (“Red”); add production code to make the test pass (“Green”); and clean up the production code while making sure the tests still pass (“Refactor”). It’s a tight loop with minimal changes at each step, so you’re never far from code that runs and has good test coverage.

By the way, to simplify things, I’ll just say “tests” and be vague about whether they’re technically “unit tests”, “specs,” “integration tests,” or “functional tests”; the main thing is that they’re written in code and they run automatically.

Red-Green-Refactor is a very satisfying rhythm when it works. Starting from the test keeps the focus on adding value, and writing a test forces you to clarify where you want to go. Many people say it promotes clean design: it’s just easier to write tests when you have well-separated modules with reasonable interfaces between them. My personal favorite part, though, is not the Red but the Refactor: the support from tests allows you to clean things up with confidence, and worry less about regressions.

Now for the exceptions. Read more…

Four short links: 21 May 2014

Four short links: 21 May 2014

Funnel Tool, Security Tools, Inside Mac Malware, and Everything is Broken

  1. EventHub — open source funnel/cohort/a-b analysis tool.
  2. Mantra — a collection of free/open source security tools, integrated into a browser (Firefox or Chromium).
  3. Reverse Engineering Mac Malware (PDF) — fascinating to see how it’s shipped, bundled, packaged, and distributed.
  4. Everything is Broken (Quinn Norton) — Computers have gotten incredibly complex, while people have remained the same gray mud with pretensions of Godhood. Today’s required read, because everything is broken and it’s the defining characteristic of this age of software. We have built computers in our image: our cancerous STD-addled diabetic alcoholic lead-sniffing telomere-decaying bacteria- and virus-addled image.
Four short links: 30 April 2014

Four short links: 30 April 2014

Critical Making, Torrent Filesystem, Testing Infrastructure, and Reproducible Research

  1. Critical Making — essays from 70 contributors looking at the politics, choices, and ethics of a lot of the makery going on.
  2. torrent-mount — mount a torrent as a filesystem in real time using Javascript. (via Joe McCann)
  3. Continuous Integration for Infrastructure — slides on the emerging tools for large-scale automated testing integrated into development and deployment workflow.
  4. Implementing Reproducible Research — book by Victoria Stodden and Johanna Cohoon on tools, practices, and platforms for making science that others can verify (another step in improving velocity and quality of scientific research).
Four short links: 28 April 2014

Four short links: 28 April 2014

Retail Student Data, Hacking Hospitals, Testing APIs, and Becoming Superhuman

  1. UK Government to Sell Its Students’ Data (Wired UK) — The National Pupil Database (NPD) contains detailed information about pupils in schools and colleges in England, including test and exam results, progression at each key stage, gender, ethnicity, pupil absence and exclusions, special educational needs, first language. The UK is becoming patient zero for national data self-harm.
  2. It’s Insanely Easy to Hack Hospital Equipment (Wired) — Erven won’t identify specific product brands that are vulnerable because he’s still trying to get some of the problems fixed. But he said a wide cross-section of devices shared a handful of common security holes, including lack of authentication to access or manipulate the equipment; weak passwords or default and hardcoded vendor passwords like “admin” or “1234″; and embedded web servers and administrative interfaces that make it easy to identify and manipulate devices once an attacker finds them on a network.
  3. Postman — API testing tool.
  4. App Controlled Hearing Aid Improves Even Normal Hearing (NYTimes) — It’s only a slight exaggeration to say that the latest crop of advanced hearing aids are better than the ears most of us were born with. Human augmentation with software and hardware.
Four short links: 24 February 2014

Four short links: 24 February 2014

Your Brain on Code, Internet of Compromised Things, Waiting for Wearables, and A/B Illusions

  1. Understanding Understanding Source Code with Functional Magnetic Resonance Imaging (PDF) — we observed 17 participants inside an fMRI scanner while they were comprehending short source-code snippets, which we contrasted with locating syntax error. We found a clear, distinct activation pattern of five brain regions, which are related to working memory, attention, and language processing. I’m wary of fMRI studies but welcome more studies that try to identify what we do when we code. (Or, in this case, identify syntax errors—if they wanted to observe real programming, they’d watch subjects creating syntax errors) (via Slashdot)
  2. Oobleck Security (O’Reilly Radar) — if you missed or skimmed this, go back and reread it. The future will be defined by the objects that turn on us. 50s scifi was so close but instead of human-shaped positronic robots, it’ll be our cars, HVAC systems, light bulbs, and TVs. Reminds me of the excellent Old Paint by Megan Lindholm.
  3. Google Readying Android Watch — just as Samsung moves away from Android for smart watches and I buy me and my wife a Pebble watch each for our anniversary. Watches are in the same space as Goggles and other wearables: solutions hunting for a problem, a use case, a killer tap. “OK Google, show me offers from brands I love near me” isn’t it (and is a low-lying operating system function anyway, not a userland command).
  4. Most Winning A/B Test Results are Illusory (PDF) — Statisticians have known for almost a hundred years how to ensure that experimenters don’t get misled by their experiments […] I’ll show how these methods ensure equally robust results when applied to A/B testing.
Comment: 1