"scale" entries

Four short links: 5 August 2014

Four short links: 5 August 2014

Discussion Graph Tool, Superlinear Productivity, Go Concurrency, and R Map/Reduce Tools

  1. Discussion Graph Tool (Microsoft Research) — simplifies social media analysis by making it easy to extract high-level features and co-occurrence relationships from raw data.
  2. Superlinear Productivity in Collective Group Actions (PLoS ONE) — study of open source projects shows small groups exhibit non-linear productivity increases by size, which drop off at larger sizes. we document a size effect in the strength and variability of the superlinear effect, with smaller groups exhibiting widely distributed superlinear exponents, some of them characterizing highly productive teams. In contrast, large groups tend to have a smaller superlinearity and less variability.
  3. coop — cheat sheet of the most common concurrency program flows in Go.
  4. Tessera — set of open source tools around Hadoop, R, and visualization.
Comment
Four short links: 4 August 2014

Four short links: 4 August 2014

Web Spreadsheet, Correlated Novelty, A/B Ethics, and Replicated Data Structures

  1. EtherCalcopen source web-based spreadsheet.
  2. Dynamics of Correlated Novelties (Nature) — paper on “the adjacent possible”. Here we propose a simple mathematical model that mimics the process of exploring a physical, biological, or conceptual space that enlarges whenever a novelty occurs. The model, a generalization of Polya’s urn, predicts statistical laws for the rate at which novelties happen (Heaps’ law) and for the probability distribution on the space explored (Zipf’s law), as well as signatures of the process by which one novelty sets the stage for another. (via Steven Strogatz)
  3. On The Media Interview with OKCupid CEO — relevant to the debate on ethics of A/B tests. I preferred this to Tim Carmody’s rant.
  4. CRDTs as Alternative to APIswhen using CRDTs to tie your system together, you don’t need to resort to using impoverished representations that simply never come anywhere near the representational power of the data structures you use in your programs at runtime. See also this paper on Convergent and Commutative Replicated Data Types.
Comment
Four short links: 29 July 2014

Four short links: 29 July 2014

Community Detection, Proven Kernel, Graph Processing on GPUs, and Browser Vision

  1. Online Community Detection for Large Complex Networks (PLosONE) — readable recount of earlier algorithms and inventions in the area, as well as a new algorithm with linear time complexity for large complex networks.
  2. sel4 — open source OS kernel (GPLv2, most userland is BSD) with end-to-end proof of implementation correctness and security enforcement. (For a discussion of what’s verified, see this blog post)
  3. mapgraph.ioMassively Parallel Graph processing on GPUs. (via Leo Meyerovich)
  4. tracking.js — browser framework and algorithms for computer vision algorithms and frameworks.
Comment: 1
Four short links: 28 July 2014

Four short links: 28 July 2014

Secure Server, Angular Style, Recursion History (see Recursion History), Aerospike Open Source

  1. streisandsets up a new server running L2TP/IPsec, OpenSSH, OpenVPN, Shadowsocks, Stunnel, and a Tor bridge. It also generates custom configuration instructions for all of these services. At the end of the run you are given an HTML file with instructions that can be shared with friends, family members, and fellow activists.
  2. Angular.js Style Guidemy opinionated styleguide for syntax, building and structuring Angular applications.
  3. How Recursion Got into ProgrammingCommittee member F.L. Bauer registered his protest by characterizing the addition of recursion to the language as an “Amsterdam plot”.
  4. aerospike — open source database server and client, with bold claims of performance.
Comment
Four short links: 22 July 2014

Four short links: 22 July 2014

English lint, Scalable Replicated Datastore, There's People in my Software, and Sci-Fi for Ethics

  1. write-gooda naive `lint’ for English prose.
  2. cockroachdba scalable, geo-replicated, transactional datastore from a team that includes the person who built Spanner for Google. Spanner requires atomic clocks, cockroach does not (which has corresponding performance consequences). (via Wired)
  3. The Deep Convergence of Networks, Software, and Peopleas we wire up our digital products increasingly with interconnected networks, their nature is increasingly a product of the responses that come back from those networks. The experience cannot be wholly represented in mock prototypes that are coded to respond in predictable ways, or even using a set of preset random responses. The power of the application is seeing the emergent behaviour of the system, and recognizing that you are a participant in that emergent behaviour. (via Tim O’Reilly)
  4. An Ethics Class for Inventors, via Sci-Fi“Reading science fiction is kind of like ethics class for inventors,” says Brueckner. Traditionally, technology schools ask ‘how do we build it?’ This class asks a different question: ‘should we?’
Comments: 2
Four short links: 8 July 2014

Four short links: 8 July 2014

Virtual Economies, Resource UAVs, Smarter Smaller Crowds, and Scaling Business

  1. Virtual Economies — new book from MIT Press on economics in games. The book will enable developers and designers to create and maintain successful virtual economies, introduce social scientists and policy makers to the power of virtual economies, and provide a useful guide to economic fundamentals for students in other disciplines.
  2. Resource Industry UAV Conference Presentations — collection of presentations from a recent resources industry conference. Includes UaaS: UAVs as a Service. (via DIY Drones)
  3. The Wisdom of Smaller, Smarter Crowdsin domains in which some crowd members have demonstrably more skill than others, smart sub-crowds could possibly outperform the whole. The central question this work addresses is whether such smart subsets of a crowd can be identified a priori in a large-scale prediction contest that has substantial skill and luck components. (via David Pennock)
  4. Larry and Sergey with Vinod (YouTube) — see transcription. I really liked Page’s point about scaling the number of things that companies do, and the constraints on such scaling.
Comment

Four short links: 4 July 2014

Deleted Transparency, Retro Theme, MPA Suckage, and Ultrasonic Comms

  1. The Flipside of the Right To Be Forgotten (Business Insider) — deletion requests were granted for a former politician who wanted to remove links to a news article about his behavior when previously in office – so that he can have a clean slate when running for a new position – and a man who was convicted of possessing child sexual abuse imagery.
  2. BOOTSTRA.386 — gorgeously retro theme for Bootstrap.
  3. Multi-Process Architectures Suck — detailed and painful look at the computational complexity and costs of multiprocess architectures.
  4. Chromecast Ultrasonic CommsIn the new system, Chromecast owners first allow support for nearby devices. A nearby device then requests access to the Chromecast, and the Chromecast plays an ultrasonic sound through the connected TV’s speakers. The sound is then picked up by the microphone in the device, which allows it to pair with the TV. (via Greg Linden)
Comment
Four short links: 15 May 2014

Four short links: 15 May 2014

Pervasive Monitoring, Mozilla DRM, Game Finances, and Distributed Systems

  1. Pervasive Monitoring is an Attack (Tim Bray) — if your ap­pli­ca­tion doesn’t sup­port pri­va­cy, that’s prob­a­bly a bug in your ap­pli­ca­tion.
  2. Reconciling Mozilla’s Mission and the W3C EME — essentially, “we don’t want to put a closed source bolus of evil into our open source unicorn, but you won’t be able to watch House of Cards with Firefox if we don’t.”
  3. The Financial Future of Game Developers (Raph Koster) — Today, a console is really just a hardware front end to a digital publisher/distribution network/storefront. […] Any structure that depends solely on blockbusters is not long for this world, because there is a significant component of luck in what drives popularity, so every release is literally a gamble. […] The median game uploaded to the App Store makes zero dollars. It starts great and just gets better. Koster is on fire! He scores again! GOOOOOOOOOOOOOOOAL!
  4. Notes on Distributed Systems for Young Bloods“It’s slow” is the hardest problem you’ll ever debug.
Comment
Four short links: 9 May 2014

Four short links: 9 May 2014

Hardening Android, Samsung Connivery, Scalable WebSockets, and Hardware Machine Learning

  1. Hardening Android for Security and Privacy — a brilliant project! prototype of a secure, full-featured, Android telecommunications device with full Tor support, individual application firewalling, true cell network baseband isolation, and optional ZRTP encrypted voice and video support. ZRTP does run over UDP which is not yet possible to send over Tor, but we are able to send SIP account login and call setup over Tor independently.
  2. The Great Smartphone War (Vanity Fair) — “I represented [the Swedish telecommunications company] Ericsson, and they couldn’t lie if their lives depended on it, and I represented Samsung and they couldn’t tell the truth if their lives depended on it.” That’s the most catching quote, but interesting to see Samsung’s patent strategy described as copying others, delaying the lawsuits, settling before judgement, and in the meanwhile ramping up their own innovation. Perhaps the other glory part is the description of Samsung employee shredding and eating incriminating documents while stalling lawyers out front. An excellent read.
  3. socketclusterhighly scalable realtime WebSockets based on Engine.io. They have screenshots of 100k messages/second on an 8-core EC2 m3.2xlarge instance.
  4. Machine Learning on a Board — everything good becomes hardware, whether in GPUs or specialist CPUs. This one has a “Machine Learning Co-Processor”. Interesting idea, to package up inputs and outputs with specialist CPU, but I wonder whether it’s a solution in search of a problem. (via Pete Warden)
Comment: 1
Four short links: 23 April 2014

Four short links: 23 April 2014

Mobile UX, Ideation Tools, Causal Consistency, and Intellectual Ventures Patent Fail

  1. Samsung UX (Scribd) — little shop of self-catalogued UX horrors, courtesy discovery in a lawsuit. Dated (Android G1 as competition) but rewarding to see there are signs of self-awareness in the companies that inflict unusability on the world.
  2. Tools for Ideation and Problem Solving (Dan Lockton) — comprehensive and analytical take on different systems for ideas and solutions.
  3. Don’t Settle for Eventual Consistency (ACM) — proposes “causal consistency”, prototyped in COPS and Eiger from Princeton.
  4. Intellectual Ventures Loses Patent Case (Ars Technica) — The Capital One case ended last Wednesday, when a Virginia federal judge threw out the two IV patents that remained in the case. It’s the first IV patent case seen through to a judgment, and it ended in a total loss for the patent-holding giant: both patents were invalidated, one on multiple grounds.
Comment