Using Logs to Build Solid Data Infrastructure — (Martin Kleppmann) — For lack of a better term, I’m going to call this the problem of ‘data integration.’ With that, I really just mean ‘making sure that the data ends up in all the right places.’ Whenever a piece of data changes in one place, it needs to change correspondingly in all the other places where there is a copy or derivative of that data.
Project Advice (Daniel Bachhuber) — for leaders of distributed teams. Focus on clearing blockers above all else. Because you’re working on an open source project with contributors across many timezones, average time to feedback will optimistically be six hours. More likely, it will be 24-48 hours. This slow feedback cycle can kill progress on pull requests. As a project maintainer, prioritize giving feedback, clearing blockers, and making decisions.
YAGNI (Martin Fowler) — cost of build, cost of delay, cost of carry, cost of repair. Every product manager has felt these costs.
How to Turn a Liberal Hipster into a Global Capitalist (The Guardian) — In Zoe Svendsen’s play “World Factory at the Young Vic,” the audience becomes the cast. Sixteen teams sit around factory desks playing out a carefully constructed game that requires you to run a clothing factory in China. How to deal with a troublemaker? How to dupe the buyers from ethical retail brands? What to do about the ever-present problem of clients that do not pay? […] And because the theatre captures data on every choice by every team, for every performance, I know we were not alone. The aggregated flowchart reveals that every audience, on every night, veers toward money and away from ethics. I’m a firm believer that games can give you visceral experience, not merely intellectual knowledge, of an activity. Interesting to see it applied so effectively to business.
Why Are Eight Bits Enough for Deep Neural Networks? (Pete Warden) — It turns out that neural networks are different. You can run them with eight-bit parameters and intermediate buffers, and suffer no noticeable loss in the final results. This was astonishing to me, but it’s something that’s been re-discovered over and over again.
The Great Decoupling (HBR) — The Second Machine Age is playing out differently than the First Machine Age, continuing the long-term trend of material abundance but not of ever-greater labor demand.
OpenMP Support in LLVM — OpenMP enables Clang users to harness full power of modern multi-core processors with vector units. Pragmas from OpenMP 3.1 provide an industry standard way to employ task parallelism, while ‘#pragma omp simd’ is a simple yet flexible way to enable data parallelism (aka vectorization).
GM: That Car You Bought, We’re Really the Ones Who Own It — GM’s claim is all about copyright and software code, and it’s the same claim John Deere is making about their tractors. The TL;DR version of the argument goes something like this: cars work because software tells all the parts how to operate; the software that tells all the parts to operate is customized code; that code is subject to copyright; GM owns the copyright on that code and that software; a modern car cannot run without that software; it is integral to all systems; therefore, the purchase or use of that car is a licensing agreement; and since it is subject to a licensing agreement, GM is the owner and can allow/disallow certain uses or access. In the future, manufacturers own the secondary market.
Architectural Robots (Robohub) — The concept is named ‘Minibuilders.’ This is a group of robots each performing a specific task. The first robot layers a 15 cm (6 in) footprint or foundation, while a second and a third robot print the rest of the building by climbing over the structures they already printed and laying more material over them. This design is only possible at construction scale where printed layers are solid enough to support a robotic print head.
gigo — Fetching packages in golang can be difficult, especially when juggling multiple packages and private repositories. GIGO (Gigo Installer for Go) tries to solve that problem, effectively being the golang equivalent of Python’s pip.
Guide to Technical Development (Google) — This guide is a suggested path for university students to develop their technical skills academically and non-academically through self-paced, hands-on learning.
Immutable Infrastructure is the Future (Michael DeHaan) — The future of configuration management systems is in deploying cloud infrastructure that will later run immutable systems via an API level.
machinery — an asynchronous task queue/job queue based on distributed message passing.
Shadow Robot’s Dextrous Hand (Robohub) — The now infamous crab bisque recipe was prepared copying the exact movements of 2011 BBC MasterChef winner Tim Anderson. The robot switches on a hob, scrapes butter into the pan and adds prepared and measured ingredients – from a specific place on the worktop. But, it doesn’t do the chopping… That, says Rich, is an AI problem.
How We Do Agile Hardware Development at Meld — In every sprint we built both hardware and software. This doesn’t mean we had a fully fabricated new board rev once a week. […] We couldn’t build a complete new board every week, and early on we didn’t even know for sure what parts we wanted in our final BOM (Bill of Materials) so we used eval boards. These stories of how companies iterated fast will eventually build a set of best practices for hardware startups, similar to those in software.
Recording Time Series — if data arrives with variable latency, timestamps are really probabilistic ranges. How do you store your data for searches and calculations that reflect reality, and are not erroneous because you’re ignoring a simplification you made to store the data more conveniently?
Call Me Maybe, ElasticSearch 1.5.0 — To be precise, Elasticsearch’s transaction log does not put your data safety first. It puts it anywhere from zero to five seconds later. In this test we kill random Elasticsearch processes with kill -9 and restart them. In a datastore like Zookeeper, Postgres, BerkeleyDB, SQLite, or MySQL, this is safe: transactions are written to the transaction log and fsynced before acknowledgement. In Mongo, the fsync flags ensure this property as well. In Elasticsearch, write acknowledgement takes place before the transaction is flushed to disk, which means you can lose up to five seconds of writes by default. In this particular run, ES lost about 10% of acknowledged writes.
FIDO — Netflix’s open source system for automatically analyzing security events and responding to security incidents.
Disinformation Visualisation: How to Lie with Datavis — We don’t spread visual lies by presenting false data. That would be lying. We lie by misrepresenting the data to tell the very specific story we’re interested in telling. If this is making you slightly uncomfortable, that’s a good thing; it should. If you’re concerned about adopting this new and scary habit, well, don’t worry; it’s not new. Just open your CV to be reminded you’ve lied with truthful data before. This time, however, it will be explicit and visual. (via Regine Debatty)
Microtugs — a new type of small robot that can apply orders of magnitude more force than it weighs. This is in stark contrast to previous small robots that have become progressively better at moving and sensing, but lacked the ability to change the world through the application of human-scale loads.
Vault — a tool for securely managing secrets and encrypting data in-transit.
iSAX: Indexing and Mining Terabyte Sized Time Series (PDF) — Our approach allows both fast exact search and ultra-fast approximate search. We show how to exploit the combination of both types of search as sub-routines in data mining algorithms, allowing for the exact mining of truly massive real-world data sets, containing millions of time series. (via Benjamin Black)
Perfect Security (99% Invisible) — Since we lost perfect security in the 1850s, it has has remained elusive. Despite tremendous leaps forward in security technology, we have never been able to get perfect security back. History of physical security, relevant to digital security today.
keywhiz — a system for managing and distributing secrets. It can fit well with a service oriented architecture (SOA).
Call Me Maybe: MongoDB Stale Reads — a master class in understanding modern distributed systems. Kyle’s blog is consistently some of the best technical writing around today.
3rd Person Driving (IEEE) — A Taiwan company called SPTek has figured out a way to use an array of cameras to generate a 3-D “Around View Monitor” that can show you multiple different views of the outside of your car. Use a top-down view for tight parking spaces, a front view looking backward for highway lane changes, or a see-through rear view for pulling out into traffic. It’s not a video game; it’s the next step in safety.
Cross-Platform Debugger for Go — take the source code of a target program, insert debugging code between every line, then compile and run that instead. The result is a fully-functional debugger that is extremely portable. In fact, thanks to gopherjs, you can run it right here in your browser!