Mesa: Geo-Replicated, Near Real-Time, Scalable Data Warehousing (PDF) — paper by Googlers on the database holding G’s ad data. Trillions of rows, petabytes of data, point queries with 99th percentile latency in the hundreds of milliseconds and overall query throughput of trillions of rows fetched per day, continuous updates on the order of millions of rows updated per second, strong consistency and repeatable query results even if a query involves multiple datacenters, and no SPOF. (via Greg Linden)
Thumbstopping (Salon) — The prime goal of a Facebook ad campaign is to create an ad “so compelling that it would get people to stop scrolling through their news feeds,” reports the Times. This is known, in Facebook land, as a “thumbstopper.” And thus, the great promise of the digitial revolution is realized: The best minds of our generation are obsessed with manipulating the movement of your thumb on a smartphone touch-screen.
Microsoft’s Development Practices (Ars Technica) — they get the devops religion but call it “combined engineering”. They get the idea of shared code bases, but call it “open source”. At least when they got the agile religion, they called it that. Check out the horror story of where they started: a two-year development process in which only about four months would be spent writing new code. Twice as long would be spent fixing that code. MSFT’s waterfall was the equivalent of American football, where there’s 11 minutes of actual play in the average 3h 12m game.
Discussion Graph Tool (Microsoft Research) — simplifies social media analysis by making it easy to extract high-level features and co-occurrence relationships from raw data.
Superlinear Productivity in Collective Group Actions (PLoS ONE) — study of open source projects shows small groups exhibit non-linear productivity increases by size, which drop off at larger sizes. we document a size effect in the strength and variability of the superlinear effect, with smaller groups exhibiting widely distributed superlinear exponents, some of them characterizing highly productive teams. In contrast, large groups tend to have a smaller superlinearity and less variability.
coop — cheat sheet of the most common concurrency program flows in Go.
Tessera — set of open source tools around Hadoop, R, and visualization.
Dynamics of Correlated Novelties (Nature) — paper on “the adjacent possible”. Here we propose a simple mathematical model that mimics the process of exploring a physical, biological, or conceptual space that enlarges whenever a novelty occurs. The model, a generalization of Polya’s urn, predicts statistical laws for the rate at which novelties happen (Heaps’ law) and for the probability distribution on the space explored (Zipf’s law), as well as signatures of the process by which one novelty sets the stage for another. (via Steven Strogatz)
streisand — sets up a new server running L2TP/IPsec, OpenSSH, OpenVPN, Shadowsocks, Stunnel, and a Tor bridge. It also generates custom configuration instructions for all of these services. At the end of the run you are given an HTML file with instructions that can be shared with friends, family members, and fellow activists.
cockroachdb — a scalable, geo-replicated, transactional datastore from a team that includes the person who built Spanner for Google. Spanner requires atomic clocks, cockroach does not (which has corresponding performance consequences). (via Wired)
The Deep Convergence of Networks, Software, and People — as we wire up our digital products increasingly with interconnected networks, their nature is increasingly a product of the responses that come back from those networks. The experience cannot be wholly represented in mock prototypes that are coded to respond in predictable ways, or even using a set of preset random responses. The power of the application is seeing the emergent behaviour of the system, and recognizing that you are a participant in that emergent behaviour. (via Tim O’Reilly)
An Ethics Class for Inventors, via Sci-Fi — “Reading science fiction is kind of like ethics class for inventors,” says Brueckner. Traditionally, technology schools ask ‘how do we build it?’ This class asks a different question: ‘should we?’
Virtual Economies — new book from MIT Press on economics in games. The book will enable developers and designers to create and maintain successful virtual economies, introduce social scientists and policy makers to the power of virtual economies, and provide a useful guide to economic fundamentals for students in other disciplines.
The Wisdom of Smaller, Smarter Crowds — in domains in which some crowd members have demonstrably more skill than others, smart sub-crowds could possibly outperform the whole. The central question this work addresses is whether such smart subsets of a crowd can be identified a priori in a large-scale prediction contest that has substantial skill and luck components. (via David Pennock)
The Flipside of the Right To Be Forgotten (Business Insider) — deletion requests were granted for a former politician who wanted to remove links to a news article about his behavior when previously in office – so that he can have a clean slate when running for a new position – and a man who was convicted of possessing child sexual abuse imagery.
Chromecast Ultrasonic Comms — In the new system, Chromecast owners first allow support for nearby devices. A nearby device then requests access to the Chromecast, and the Chromecast plays an ultrasonic sound through the connected TV’s speakers. The sound is then picked up by the microphone in the device, which allows it to pair with the TV. (via Greg Linden)
The Financial Future of Game Developers (Raph Koster) — Today, a console is really just a hardware front end to a digital publisher/distribution network/storefront. […] Any structure that depends solely on blockbusters is not long for this world, because there is a significant component of luck in what drives popularity, so every release is literally a gamble. […] The median game uploaded to the App Store makes zero dollars. It starts great and just gets better. Koster is on fire! He scores again! GOOOOOOOOOOOOOOOAL!
Hardening Android for Security and Privacy — a brilliant project! prototype of a secure, full-featured, Android telecommunications device with full Tor support, individual application firewalling, true cell network baseband isolation, and optional ZRTP encrypted voice and video support. ZRTP does run over UDP which is not yet possible to send over Tor, but we are able to send SIP account login and call setup over Tor independently.
The Great Smartphone War (Vanity Fair) — “I represented [the Swedish telecommunications company] Ericsson, and they couldn’t lie if their lives depended on it, and I represented Samsung and they couldn’t tell the truth if their lives depended on it.” That’s the most catching quote, but interesting to see Samsung’s patent strategy described as copying others, delaying the lawsuits, settling before judgement, and in the meanwhile ramping up their own innovation. Perhaps the other glory part is the description of Samsung employee shredding and eating incriminating documents while stalling lawyers out front. An excellent read.
socketcluster — highly scalable realtime WebSockets based on Engine.io. They have screenshots of 100k messages/second on an 8-core EC2 m3.2xlarge instance.
Machine Learning on a Board — everything good becomes hardware, whether in GPUs or specialist CPUs. This one has a “Machine Learning Co-Processor”. Interesting idea, to package up inputs and outputs with specialist CPU, but I wonder whether it’s a solution in search of a problem. (via Pete Warden)