Nat has chaired the O'Reilly Open Source Convention and other O'Reilly conferences for over a decade. He ran the first web server in New Zealand, co-wrote the best-selling Perl Cookbook, and was one of the founding Radar bloggers. He lives in New Zealand and consults in the Asia-Pacific region.
Decoding Jeff Jonas (National Geographic) — “He thinks in three—no, four dimensions,” Nathan says. “He has a data warehouse in his head.” And that’s where the work takes place—in his head. Not on paper. Not on a computer. He resorts to paper only to work the details out. When asked about his thought process, Jonas reaches for words, then says: “It’s like a Rubik’s Cube. It all clicks into place. “The solution,” he says, is “simply there to find.” Jeff’s a genius and has his own language for explaining what he does. This quote goes a long way to explaining it.
How Apple Uses Mesos for Siri — great to see not only some details of the tooling that Apple built, but also their acknowledgement of the open source foundations and ongoing engagement with those open source communities. There have been times in the past when Apple felt like a parasite on the commons rather than a participant.
Cheaper Bandwidth or Bust: How Google Saved YouTube (ArsTechnica) — Remember YouTube’s $2 million-a-month bandwidth bill before the Google acquisition? While it wasn’t an overnight transition, apply Google’s data center expertise, and this cost drops to about $666,000 a month.
AWS Business Numbers — Amazon Web Services generated $5.2 billion over the past four quarters, and almost $700 million in operating income. During the first quarter of 2015, AWS sales reached $1.6 billion, up 49% year-over-year, and roughly 7% of Amazon’s overall sales.
On Code Review (Glen D Sanford) — Pending code reviews represent blocked threads of execution.
Four Days of Go (Evan Miller) — Reading Go’s mailing list and documentation, I get a similar sense of refusal-to-engage — the authors are communicative, to be sure, but in a didactic way. They seem tired of hearing people’s ideas, as if they’ve already thought of everything, and the relative success of Go at Google and elsewhere has only led them to turn the volume knob down. Which is a shame, because they’ll probably miss out on some good ideas (including my highly compelling, backwards-incompatible, double-triple-colon-assignment proposal mentioned above). Under this theory, more of the language choices start to make sense. There is no ternary operator because the language designers were tired of dealing with other people’s use of ternary operators. There is One True Way To Format Code — embodied in gofmt — because the designers were tired of how other people formatted their code. Rather than debate or engage, it was easier to make a new language and shove the new rules onto everyone by coupling it with Very Fast Build Times, a kind of veto-proof Defense Spending Bill in the Congress of computer programming. In this telling, the story of Go is really a tale of revenge, not just against slow builds, but against all kinds of sloppy programming.
Tempescope — Ambient weather display for your home. In my home, that’s a window. (via Matt Webb)
Perfect Security (99% Invisible) — Since we lost perfect security in the 1850s, it has has remained elusive. Despite tremendous leaps forward in security technology, we have never been able to get perfect security back. History of physical security, relevant to digital security today.
keywhiz — a system for managing and distributing secrets. It can fit well with a service oriented architecture (SOA).
Call Me Maybe: MongoDB Stale Reads — a master class in understanding modern distributed systems. Kyle’s blog is consistently some of the best technical writing around today.
3rd Person Driving (IEEE) — A Taiwan company called SPTek has figured out a way to use an array of cameras to generate a 3-D “Around View Monitor” that can show you multiple different views of the outside of your car. Use a top-down view for tight parking spaces, a front view looking backward for highway lane changes, or a see-through rear view for pulling out into traffic. It’s not a video game; it’s the next step in safety.
Cross-Platform Debugger for Go — take the source code of a target program, insert debugging code between every line, then compile and run that instead. The result is a fully-functional debugger that is extremely portable. In fact, thanks to gopherjs, you can run it right here in your browser!
The Crazy-Tiny Next Generation of Computers — 1 cubic millimeter-sized sensors are coming. The only sound you might hear is a prolonged groan. That’s because these computers are just one cubic millimeter in size, and once they hit the floor, they’re gone. “We just lose them,” Dutta says. “It’s worse than jewelry.”
The New Science of Building Great Teams (Sandy Pentland) — fascinating discussion of MIT’s Human Dynamics lab’s research into how great teams function. The data also reveal, at a higher level, that successful teams share several defining characteristics: 1. Everyone on the team talks and listens in roughly equal measure, keeping contributions short and sweet. 2. Members face one another, and their conversations and gestures are energetic. 3. Members connect directly with one another—not just with the team leader. 4. Members carry on back-channel or side conversations within the team. 5. Members periodically break, go exploring outside the team, and bring information back.
Meet DJ Patil — “It was this kind of moment when you realize: ‘Oh, my gosh, I am that stupid,’” he said.
Interview with Bruce Sterling on the Convergence of Humans and Machines — If you are a human being, and you are doing computation, you are trying to multiply 17 times five in your head. It feels like thinking. Machines can multiply, too. They must be thinking. They can do math and you can do math. But the math you are doing is not really what cognition is about. Cognition is about stuff like seeing, maneuvering, having wants, desires. Your cat has cognition. Cats cannot multiply 17 times five. They have got their own umwelt (environment). But they are mammalian, you are a mammalian. They are actually a class that includes you. You are much more like your house cat than you are ever going to be like Siri. You and Siri converging, you and your house cat can converge a lot more easily. You can take the imaginary technologies that many post-human enthusiasts have talked about, and you could afflict all of them on a cat. Every one of them would work on a cat. The cat is an ideal laboratory animal for all these transitions and convergences that we want to make for human beings. (via Vaughan Bell)
DeepDive — DeepDive is targeted to help users extract relations between entities from data and make inferences about facts involving the entities. DeepDive can process structured, unstructured, clean, or noisy data and outputs the results into a database.
Running Kafka at Scale (LinkedIn Engineering) — This tiered infrastructure solves many problems, but it greatly complicates monitoring Kafka and assuring its health. While a single Kafka cluster, when running normally, will not lose messages, the introduction of additional tiers, along with additional components such as mirror makers, creates myriad points of failure where messages can disappear. In addition to monitoring the Kafka clusters and their health, we needed to create a means to assure that all messages produced are present in each of the tiers, and make it to the critical consumers of that data.
Facebook Biometrics Cache (Business Insider) — Facebook has been accused of violating the privacy of its users by collecting their facial data, according to a class-action lawsuit filed last week. This data-collection program led to its well-known automatic face-tagging service. But it also helped Facebook create “the largest privately held stash of biometric face-recognition data in the world,” the Courthouse News Service reports.
The Clustering of Time Series Sequences is Meaningless (PDF) — Clustering of time series subsequences is meaningless. More concretely, clusters extracted from these time series are forced to obey a certain constraint that is pathologically unlikely to be satisfied by any data set, and because of this, the clusters extracted by any clustering algorithm are essentially random. While this constraint can be intuitively demonstrated with a simple illustration and is simple to prove, it has never appeared in the literature. We can justify calling our claim surprising since it invalidates the contribution of dozens of previously published papers. We will justify our claim with a theorem, illustrative examples, and a comprehensive set of experiments on reimplementations of previous work. From 2003, warning against sliding window techniques.
Toolkits for the Mind (MIT TR) — Programming–language designer Guido van Rossum, who spent seven years at Google and now works at Dropbox, says that once a software company gets to be a certain size, the only way to stave off chaos is to use a language that requires more from the programmer up front. “It feels like it’s slowing you down because you have to say everything three times,” van Rossum says. Amen!
Robots Roam Earth’s Imperiled Oceans (Wired) — It’s six feet long and shaped like an airliner, with two wings and a tail fin, and bears the message, “OCEANOGRAPHIC INSTRUMENT PLEASE DO NOT DISTURB.” All caps considered, though, it’s a more innocuous epigram than the one on a drone I saw back at the dock: “Not a weapon — Science Instrument.”
A/A Testing — In an A/A test, you run a test using the exact same options for both “variants” in your test. That’s right, there’s no difference between “A” and “B” in an A/A test. It sounds stupid, until you see the “results.” (via Nelson Minar)
NSA Declares War on General-Purpose Computing (BoingBoing) — NSA director Michael S Rogers says his agency wants “front doors” to all cryptography used in the USA, so that no one can have secrets it can’t spy on — but what he really means is that he wants to be in charge of which software can run on any general purpose computer.
The Great Reversal in the Demand for Skill and Cognitive Tasks (PDF) — The only difference with more conventional models of skill-biased technological change is our modelling of the fruits of cognitive employment as creating a stock instead of a pure flow. This slight change causes technological change to generate a boom and bust cycle, as is common in most investment models. We also incorporated into this model a standard selection process whereby individuals sort into occupations based on their comparative advantage. The selection process is the key mechanism that explains why a reduction in the demand for cognitive tasks, which are predominantly filled by higher educated workers, can result in a loss of employment concentrated among lower educated workers. While we do not claim that our model is the only structure that can explain the observations we present, we believe it gives a very simple and intuitive explanation to the changes pre- and post-2000.
provinces — state and province lists for (some) countries.
Cultural Analytics — the use of computational and visualization methods for the analysis of massive cultural data sets and flows. Interesting visualisations as well as automated understandings.
The Code is Just the Symptom — The engineering culture was a three-layer cake of dysfunction, where everyone down the chain had to execute what they knew to be an impossible task, at impossible speeds, perfectly. It was like the games of Simon Says and Telephone combined to bad effect. Most engineers will have flashbacks at these descriptions. Trigger warning: candid descriptions of real immature software organisations.