"programming" entries

Four short links: 1 July 2015

Four short links: 1 July 2015

Recovering from Debacle, Open IRS Data, Time Series Requirements, and Error Messages

  1. Google Dev Apologies After Photos App Tags Black People as Gorillas (Ars Technica) — this is how you recover from a unequivocally horrendous mistake.
  2. IRS Finally Agrees to Release Non-Profit Records (BoingBoing) — Today, the IRS released a statement saying they’re going to do what we’ve been hoping for, saying they are going to release e-file data and this is a “priority for the IRS.” Only took $217,000 in billable lawyer hours (pro bono, thank goodness) to get there.
  3. Time Series Database Requirements — classic paper, laying out why time-series databases are so damn weird. Their access patterns are so unique because of the way data is over-gathered and pushed ASAP to the store. It’s mostly recent, mostly never useful, and mostly needed in order. (via Thoughts on Time-Series Databases)
  4. Compiler Errors for Humans — it’s so important, and generally underbaked in languages. A decade or more ago, I was appalled by Python’s errors after Perl’s very useful messages. Today, appreciating Go’s generally handy errors. How a system handles the operational failures that will inevitably occur is part and parcel of its UX.
Comment
Four short links: 30 June 2015

Four short links: 30 June 2015

Ductile Systems, Accessibility Testing, Load Testing, and CRAP Data

  1. Brittle SystemsMore than two decades ago at Sun, I was convinced that making systems ductile (the opposite of brittle) was the hardest and most important problem in system engineering.
  2. tota11y — accessibility testing toolkit from Khan.
  3. Locustan open source load testing tool.
  4. Impala: a Modern, Open-source SQL Engine for Hadoop (PDF) — CRAP, aka Create, Read, and APpend, as coined by an ex-colleague at VMware, Charles Fan (note the absence of update and delete capabilities). (via A Paper a Day)
Comment
Four short links: 23 June 2015

Four short links: 23 June 2015

Irregular Periodicity, Facebook Beacons, Industry 4.0, and Universal Container

  1. Fast Lomb-Scargle Periodograms in Pythona classic method for finding periodicity in irregularly-sampled data.
  2. Facebook Bluetooth Beacons — free for you to use and help people see more information about your business whenever they use Facebook during their visit.
  3. Industry 4.0 — stop gagging at the term. Interesting examples of connectivity and data improving manufacturing. Human-machine interfaces: Logistics company Knapp AG developed a picking technology using augmented reality. Pickers wear a headset that presents vital information on a see-through display, helping them locate items more quickly and precisely. And with both hands free, they can build stronger and more efficient pallets, with fragile items safeguarded. An integrated camera captures serial and lot ID numbers for real-time stock tracking. Error rates are down by 40%, among many other benefits. Digital-to-physical transfer: Local Motors builds cars almost entirely through 3-D printing, with a design crowdsourced from an online community. It can build a new model from scratch in a year, far less than the industry average of six. Vauxhall and GM, among others, still bend a lot of metal, but also use 3-D printing and rapid prototyping to minimize their time to market. (via Quartz)
  4. runCa lightweight universal runtime container, by the Open Container Project. (OCP = multi-vendor initiative in hands of Linux Foundation)
Comment
Four short links: 19 June 2015

Four short links: 19 June 2015

Computational Journalism, Bio Startups & Patents, Ad Blocker Wars, and The Night Watch

  1. Computational Journalism — Google awards to projects around computational journalism. Sample: The goal of the project is to automatically build topic “event threads” that will help journalists and citizens decode claims made by public figures, in order to distinguish between personal opinion, communication tools, and voluntary distortions of the reality.
  2. Editing the Software of Life — research yielding the ability to edit DNA has spawned a new set of biotech startups, and a patent morass. Zhang already holds the first of several vital and broad patents covering cas9 genome editing. Yet, Doudna and Charpentier had filed patent applications covering similar ground earlier than Zhang.
  3. Ad Blocker-Stopping Software — the evolutionary battle between ads and blockers is about to get gunpowder. As Ethan Zuckerman said, advertising is the original sin of the Internet.
  4. The Night Watch (PDF) — There is nothing funny to print when you have a misaligned memory access, because your machine is dead and there are no printers in the spirit world.
Comment

The art of design patterns

Communicate more efficiently, concisely, and accurately.

Download a free copy of An Engineering Manager’s Guide to Design Patterns, a brain-friendly report that shows you how object-oriented design patterns are ideal for solving specific problems in application design.

If you haven’t had the pleasure of viewing Hal Abelson & Gerald Sussman’s 1986 MIT introductory computer science course, you owe it to yourself to set aside a few hours to view it. “1986?”, you say — “Could that really be relevant to my work today?” Unless you came through MIT or a similar program that teaches from their seminal book The Structure and Interpretation of Computer Programs, I’d bet you are most likely going to learn a few new things (even if you consider yourself a seasoned software developer).

Play the video, and right away you might be surprised, as Abelson, in the first five minutes of the class, states that not only is computer science not a science, it doesn’t have all that much to do with computers. Rather, Abelson suggests, computer science is more of an engineering discipline, or perhaps even an art; and, rather than being concerned with computers, computer science is more an exercise in creating imperative knowledge and managing complexity.

Anyone who has ever been late on a software development project (who hasn’t?) can relate to this. Software development starts to feel more like an art or craft when the best you can do is roughly estimate the size and scope of a job and then cross your fingers and hope for the best — certainly, it is at times like these when our field doesn’t feel like much of a science. And, for anyone who has worked on a project of moderate size, at some point you find complexity staring you in the face. All too often our first designs, and our code, turn into the dreaded big ball of mud (yes, that is a technical term).
Read more…

Comment: 1
Four short links: 17 June 2015

Four short links: 17 June 2015

Academic Publishing Concentration, Hardware Independence, Exception Monitoring, and Negotiating Tactics

  1. The Oligopoly of Academic Publishers in the Digital Era (PLoSone) — Combined, the top five most prolific publishers account for more than 50% of all papers published in 2013. (via CBC)
  2. LLVM Bitcode Gives Apple Hardware Independence (Medium) — Bob [Mansfield] has been quietly building a silicon team with the skills to rival all other players in the industry. Bob works for one of 15 companies with an ARM architecture license, giving his team carte blanche to modify and extend ARM in any way they see fit. And Bob’s CPUs only have to satisfy a single customer.
  3. Github Exception Monitoring and Response — I need another word than “porn” to describe something that makes me sigh fervently with desire to achieve at that level.
  4. 31 Negotiation Tactics (Nick Kolenda) — he mysteriously omitted my power tactics of (a) crying, (b) greeting my opposite number with the wrong name, and (c) passing a napkin covered with random scrawls as I say, “what do you make of this?”
Comment
Four short links: 12 June 2015

Four short links: 12 June 2015

OLAP Datastores, Timely Dataflow, Paul Ford is God, and Static Analysis

  1. pinota realtime distributed OLAP datastore, which is used at LinkedIn to deliver scalable real time analytics with low latency. It can ingest data from offline data sources (such as Hadoop and flat files) as well as online sources (such as Kafka). Pinot is designed to scale horizontally.
  2. Naiad: A Timely Dataflow System — in Timely Dataflow, the first two features are needed to execute iterative and incremental computations with low latency. The third feature makes it possible to produce consistent results, at both outputs and intermediate stages of computations, in the presence of streaming or iteration.
  3. What is Code (Paul Ford) — What the coders aren’t seeing, you have come to believe, is that the staid enterprise world that they fear isn’t the consequence of dead-eyed apathy but rather détente. Words and feels.
  4. Facebook Infer Opensourced — the static analyzer I linked to yesterday, released as open source today.
Comment
Four short links: 3 June 2015

Four short links: 3 June 2015

Filter Design, Real-Time Analytics, Neural Turing Machines, and Evaluating Subjective Opinions

  1. How to Design Applied FiltersThe most frequently observed issue during usability testing were filtering values changing placement when the user applied them – either to another position in the list of filtering values (typically the top) or to an “Applied filters” summary overview. During testing, the subjects were often confounded as they noticed that the filtering value they just clicked was suddenly “no longer there.”
  2. Twitter Herona real-time analytics platform that is fully API-compatible with Storm […] At Twitter, Heron is used as our primary streaming system, running hundreds of development and production topologies. Since Heron is efficient in terms of resource usage, after migrating all Twitter’s topologies to it we’ve seen an overall 3x reduction in hardware, causing a significant improvement in our infrastructure efficiency.
  3. ntman implementation of neural Turing machines. (via @fastml_extra)
  4. Bayesian Truth Seruma scoring system for eliciting and evaluating subjective opinions from a group of respondents, in situations where the user of the method has no independent means of evaluating respondents’ honesty or their ability. It leverages respondents’ predictions about how other respondents will answer the same questions. Through these predictions, respondents reveal their meta-knowledge, which is knowledge of what other people know.
Comment
Four short links: 2 June 2015

Four short links: 2 June 2015

Toyota Code, Sapir-Wharf-Emoji, Crowdsourcing Formal Proof, and Safety-Critical Code

  1. Toyota’s Spaghetti CodeToyota had more than 10,000 global variables. And he was critical of Toyota watchdog supervisor — software to detect the death of a task — design. He testified that Toyota’s watchdog supervisor ‘is incapable of ever detecting the death of a major task. That’s its whole job. It doesn’t do it. It’s not designed to do it.’ (via @qrush)
  2. Google’s Design Icons (Kevin Marks) — Google’s design icons distinguish eight kinds of airline seats but has none for trains or buses.
  3. Verigames — DARPA-funded game to crowdsource elements of formal proofs. (via Network World)
  4. 10 Rules for Writing Safety-Critical Code — which I can loosely summarize as “simple = safer, use the built-in checks, don’t play with fire.”
Comment: 1
Four short links: 1 June 2015

Four short links: 1 June 2015

AI Drives, Decent Screencaps, HTTP/2 Antipatterns, Time Series

  1. The Basic AI Drives (PDF) — Surely, no harm could come from building a chess-playing robot, could it? In this paper, we argue that such a robot will indeed be dangerous unless it is designed very carefully. Without special precautions, it will resist being turned off, will try to break into other machines and make copies of itself, and will try to acquire resources without regard for anyone else’s safety. These potentially harmful behaviors will occur not because they were programmed in at the start, but because of the intrinsic nature of goal-driven systems.
  2. PreTTY — how to take a good-looking screencap of your terminal app in action.
  3. Why Some of Yesterday’s HTTP Best Practices are HTTP/2 Antipatterns — also functions as an overview of HTTP/2 for those of us who didn’t keep up with the standardization efforts.
  4. Tiseana software project for the analysis of time series with methods based on the theory of nonlinear deterministic dynamical systems. (via @aphyr)
Comment