Dragon: A Distributed Graph Query Engine — Facebook describes its internal graph query engine. [T]he layout of these indices on storage is optimized based on a deeper understanding of query patterns (e.g., many queries are about friends), as opposed to accepting random sharding, which is common in these systems. Wisely, the system is tailored to the use cases they have and the patterns they see in access.
Almost Everyone Is Doing the API Economy Wrong (Techcrunch) — Redux: your API should help you make money when the API customer makes money, and you should set clear expectations for what’s acceptable and what’s not. But every developer should be forced to write 100 times: “if you build on a platform you don’t own, you’re building on a potential and probable future competitor.”
Traditional Economics Failed, Here’s a Blueprint — runs through the shifts happening in our thinking about the world and ourselves (simple to complex, independent to interdependent, rational calculator to irrational approximators, etc) and concludes: True self-interest is mutual interest. The best way to improve your likelihood of surviving and thriving is to make sure those around you survive and thrive. See above API note.
Blitzscaling (HBR) — as you move from village to city, functions are beginning to be differentiated; you’re really multithreading. I could write a thesis on the CAP theorem for business. And I have definitely worked for companies that have a “share nothing” approach to solving their threading issues.
Security Without Identification (PDF) — a David Chaum paper from 1985. Digital pseudonyms, handheld signing devices, Current systems emphasize the one-sided security of organizations attempting to protect themselves from individuals; the new approach allows all parties to protect their own interests. The new approach relies on individuals keeping secret keys from organizations and organizations devising other secret keys that are kept from individuals. During transactions, parties use these keys to provide each other with specially coded confirmation of the transaction details, which can be used as evidence.
Killing Slow Chrome Tabs (Medium) — There is one not-so-well known tool in Chrome, that allows you to analyse how much resources the individual tabs consume. It is called Task Manager and you can find it in Menu > More Tools > Task Manager.
Comments Off on Four short links: 22 February 2016
The Economics of Drone Delivery — The analysis is still mostly speculative. Keeney imagines that 6,000 operators who earn $50,000 per year will operate 30,000 to 40,000 drones. Each drone will make 30 deliveries per day. Her analysis ignores depreciation and questions like: ‘How will drones avoid airplanes and deliver packages in Manhattan?’ And there’s another core issue: $12.92 is the price UPS charges to consumers, but its actual marginal cost of delivering one more package along a route they are delivering to already is probably closer to $2. When push comes to shove, will drones be able to compete? (via Chris Anderson)
7 Ways Your Data is Telling You It’s a Graph — Network, tree, taxonomy, ancestry, structure – if people are using those words to talk about an organizational chart or reporting structure, they’re telling you that data and the relationships between that data are important.
Comments Off on Four short links: 25 December 2015
Economics Apparently Not Replicable (PDF) — We successfully replicate the key qualitative result of 22 of 67 papers (33%) without contacting the authors. Excluding the six papers that use confidential data and the two papers that use software we do not possess, we replicate 29 of 59 papers (49%) with assistance from the authors. Because we are able to replicate less than half of the papers in our sample even with help from the authors, we assert that economics research is usually not replicable.
26 Things I Learned in the Deep Learning Summer School — 20. When Frederick Jelinek and his team at IBM submitted one of the first papers on statistical machine translation to COLING in 1988, they got the following anonymous review: The validity of a statistical (information theoretic) approach to MT has indeed been recognized, as the authors mention, by Weaver as early as 1949. And was universally recognized as mistaken by 1950 (cf. Hutchins, MT – Past, Present, Future, Ellis Horwood, 1986, p. 30ff and references therein). The crude force of computers is not science. The paper is simply beyond the scope of COLING.
The Final Leaked TPP Text is All That We Feared (EFF) — If you dig deeper, you’ll notice that all of the provisions that recognize the rights of the public are non-binding, whereas almost everything that benefits rightsholders is binding.
Eric Brewer on Kubernetes — interesting not only for insights into Google’s efforts around Kubernetes but for: There’s so much excitement we can hardly handle all the pull requests. I think we’re committing, based on the GitHub log, something like 40 per day right now, and the demand is higher than that. Each of those takes reviews and, of course, there’s a wide variety of quality on those. Some are easy to review and some are quite hard to review. It’s a success problem, and we’re happy to have it. We did scale up the team to try and improve its velocity, but also just improve our ability to interact with all of the open source world that legitimately wants to contribute and has a lot to contribute. I’m very excited that the velocity is here, but it’s moving so fast it’s hard to even know all the things that change day to day. Makes a welcome change from the code dumps that are some of Google’s other high-profile projects.
We Don’t Sell Saddles Here — Stewart Butterfield, to his team, on product development and quality. Every word of this is true for every other product, too.
What is Privacy Worth? (PDF) — When endowed with the $10 untrackable card, 60.0% of subjects claimed they would keep it; however, when endowed with the $12 trackable card only 33.3% of subjects claimed they would switch to the untrackable card. […] This research raises doubts about individuals’ abilities to rationally navigate issues of privacy. From choosing whether or not to join a grocery loyalty program, to posting embarrassing personal information on a public website, individuals constantly make privacy-relevant decisions which impact their well-being. The finding that non-normative factors powerfully influence individual privacy valuations may signal the appropriateness of policy interventions.
The Great Reversal in the Demand for Skill and Cognitive Tasks (PDF) — The only difference with more conventional models of skill-biased technological change is our modelling of the fruits of cognitive employment as creating a stock instead of a pure flow. This slight change causes technological change to generate a boom and bust cycle, as is common in most investment models. We also incorporated into this model a standard selection process whereby individuals sort into occupations based on their comparative advantage. The selection process is the key mechanism that explains why a reduction in the demand for cognitive tasks, which are predominantly filled by higher educated workers, can result in a loss of employment concentrated among lower educated workers. While we do not claim that our model is the only structure that can explain the observations we present, we believe it gives a very simple and intuitive explanation to the changes pre- and post-2000.
provinces — state and province lists for (some) countries.
Cultural Analytics — the use of computational and visualization methods for the analysis of massive cultural data sets and flows. Interesting visualisations as well as automated understandings.
The Code is Just the Symptom — The engineering culture was a three-layer cake of dysfunction, where everyone down the chain had to execute what they knew to be an impossible task, at impossible speeds, perfectly. It was like the games of Simon Says and Telephone combined to bad effect. Most engineers will have flashbacks at these descriptions. Trigger warning: candid descriptions of real immature software organisations.
The Poisoned NUL Byte, 2014 Edition (Project Zero) — from Google’s public security efforts, this detailed public description of how an exploit was constructed from a found vulnerability. They’re helping. Kudos!
Myths About the Coming Robot Economy (Eric Sofge) — the entire discussion of the so-called robot economy, with its predictions of vast, permanent employment rates and glacial productivity gains, is nothing more than a wild guess. A strong pushback on the Pew Report (PDF): Frey and Osborne’s analysis is full of logical leaps, and far-reaching conclusions drawn from cursory observations about robots that have yet to replace humans.
Content for Sensitive Situations (Luke Wroblewski) — People have all kinds of feelings when interacting with your content. When someone’s needs are being met they may feel very different then when their needs are not being met. How can you meet people’s needs?
Urban Villages (Senseable City at MIT) — People who live in a larger town make more calls and call a larger number of different people. The scaling of this relation is ‘superlinear,’ meaning that on average, if the size of a town doubles, the sum of phone contacts in the city will more than double – in a mathematically predictable way. Surprisingly, however, group clustering (the odds that your friends mutually know one another) does not change with city size. It seems that even in large cities we tend to build tightly knit communities, or ‘villages,’ around ourselves. There is an important difference, though: if in a real village our connections might simply be defined by proximity, in a large city we can elect a community based on any number of factors, from affinity to interest to sexual preference. (via Flowing Data)
Oolite — open-source clone of Elite, the classic space trading game from the 80s.
Who Owns the Robots Rules The World (PDF) — interesting finding: As companies substitute machines and computers for human activity, workers need to own part of the capital stock that substitutes for them to benefit from these new “robot” technologies. Workers could own shares of the firm, hold stock options, or be paid in part from the profits. Without ownership stakes, workers will become serfs working on behalf of the robots’ overlords. Governments could tax the wealthy capital owners and redistribute income to workers, but that is not the direction societies are moving in. Workers need to own capital rather than rely on government income redistribution policies. (via Robotenomics)
Schrodinger’s Cat Video and the Death of Clear-Text (Morgan Marquis-Boire) — report, based on leaked information, about use of network injection appliances targeted unencrypted pages from major providers. Compromising a target becomes as simple as waiting for the user to view unencrypted content on the Internet.
Sensors and Citizens: Finding Balance in the New Urban Reality (Frog Design) — as the sensor systems themselves become capable of autonomous data collection and information creation, we will begin to encounter closed-loop spatial sensing networks capable not only of taking instructions, but also of taking action. When that happens, nearly every industry and government imaginable—and your daily life—will be deeply affected. This is exciting, scary, and inevitable.