"data" entries

Four short links: 30 January 2015

Four short links: 30 January 2015

FAA Rules, Sports UAVs, Woodcut Data, and Concurrent Programming

  1. FAA to Regulate UAVs? (Forbes) — and the Executive Order will segment the privacy issues related to drones into two categories — public and private. For public drones (that is, drones purchased with federal dollars), the President’s order will establish a series of privacy and transparency guidelines. See also How ESPN is Shooting the X Games with Drones (Popular Mechanics)—it’s all fun and games until someone puts out their eye with a quadrocopter. The tough part will be keeping within the tight restrictions the FAA gave them. Because drones can’t be flown above a crowd, Calcinari says, “We basically had to build a 500-foot radius around them, where the public can’t go.” The drones will fly over sections of the course that are away from the crowds, where only ESPN production employees will be. That rule is part of why we haven’t seen drones at college football games.
  2. Milestones for SaaS Companies“Getting from $0-1m is impossible. Getting from $1-10m is unlikely. And getting from $10-100m is inevitable.” —Jason Lemkin, ex-CEO of Echosign. The article proposes some significant milestones, and they ring true. Making money is generally hard. The nature of the hard changes with the amount of money you have and the amount you’re trying to make, but if it were easy, then we’d structure our society on something else.
  3. Woodcut Data VisualisationRecently, I learned how to operate a laser cutter. It’s been a whole lot of fun, and I wanted to share my experiences creating woodcut data visualizations using just D3. I love it when data visualisations break out of the glass rectangle.
  4. Why is Concurrent Programming Hard?on the one hand there is not a single concurrency abstraction that fits all problems, and on the other hand the various different abstractions are rarely designed to be used in combination with each other. We are due for a revolution in programming, something to help us make sense of the modern systems made of more moving parts than our feeble grey matter can model and intuit about.
Comment
Four short links: 28 January 2015

Four short links: 28 January 2015

Note and Vote, Gaming Behaviour, Code Search, and Immutabilate All The Things

  1. Note and Vote (Google Ventures) — nifty meeting hack to surface ideas and identify popular candidates to a decision maker.
  2. Applying Psychology to Improve Online Behaviour — online game runs massive experiments (w/researchers to validate findings) to improve the behaviour of their players. Some of Riot’s experiments are causing the game to evolve. For example, one product is a restricted chat mode that limits the number of messages abusive players can type per match. It’s a temporary punishment that has led to a noticeable improvement in player behavior afterward —on average, individuals who went through a period of restricted chat saw 20 percent fewer abuse reports filed by other players. The restricted chat approach also proved 4 percent more effective at improving player behavior than the usual punishment method of temporarily banning toxic players. Even the smallest improvements in player behavior can make a huge difference in an online game that attracts 67 million players every month.
  3. Hound — open source code search tool from Etsy.
  4. Immutability Changes Everything (PDF) — This paper is simply an amuse-bouche on the repeated patterns of computing that leverage immutability.
Comment
Four short links: 23 January 2015

Four short links: 23 January 2015

Investment Themes, Python Web Mining, Code Review, and Sexist Brilliance

  1. 16 Andreessen-Horowitz Investment Areas — I’m struck by how they’re connected: there’s a cluster around cloud development, there are two maybe three on sensors …
  2. Patterna web mining module for the Python programming language. It has tools for data mining (Google, Twitter and Wikipedia API, a web crawler, a HTML DOM parser), natural language processing (part-of-speech taggers, n-gram search, sentiment analysis, WordNet), machine learning (vector space model, clustering, SVM), network analysis and <canvas> visualization.
  3. Code Review — FogCreek’s code review checklist.
  4. Expectations of Brilliance Underlie Gender Distributions Across Academic Disciplines (Science) — Surveys revealed that some fields are believed to require attributes such as brilliance and genius, whereas other fields are believed to require more empathy or hard work. In fields where people thought that raw talent was required, academic departments had lower percentages of women. (via WaPo)
Comment
Four short links: 7 January 2015

Four short links: 7 January 2015

Program Synthesis, Data Culture, Metrics, and Information Biology

  1. Program Synthesis ExplainedThe promise of program synthesis is that programmers can stop telling computers how to do things, and focus instead on telling them what they want to do. Inductive program synthesis tackles this problem with fairly vague specifications and, although many of the algorithms seem intractable, in practice they work remarkably well.
  2. Creating a Data-Driven Culture — new (free!) ebook from Hilary Mason and DJ Patil. The editor of that team is the luckiest human being alive.
  3. Ev Williams on Metrics — a master-class in how to think about and measure what matters. If what you care about — or are trying to report on — is impact on the world, it all gets very slippery. You’re not measuring a rectangle, you’re measuring a multi-dimensional space. You have to accept that things are very imperfectly measured and just try to learn as much as you can from multiple metrics and anecdotes.
  4. Nature, the IT Wizard (Nautilus) — a fun walk through the connections between information theory, computation, and biology.
Comment
Four short links: 5 January 2015

Four short links: 5 January 2015

3D Art Reuse, Faceted Data Browser, Robotics Roundup, and Social Signal Interpretation

  1. Lincoln Gallery Reuse — UK gallery placed 3D models of their works online and are sharing what people did with them. Some beautiful art in here! (via BoingBoing)
  2. Kesihif — open source browser for faceted data.
  3. 2014 Robotics IPOs, Acquisitions, and Failures (RoboHub) — good roundup of what happened in 2014.
  4. SSIan open source platform for social signal interpretation.
Comment
Four short links: 18 December 2014

Four short links: 18 December 2014

Manufacturer Rootkits, Dangerous Dongle, Physical Visualisation, and Cryptoed Comms

  1. Popular Chinese Android Smartphone Backdoored By ManufacturerCoolpad is the third largest smartphone builder in China, and ranks sixth worldwide with 3.7 percent global market share. It trails only Lenovo and Xiaomi in China and is the leader of China’s 4G market with 16 percent market share. Coolpad outsells Samsung and Apple in China, and has said it plans to expand globally with a goal of 60 million phones worldwide. For now, its high-end Halo Dazen phones are the only ones containing the backdoor, Palo Alto said. Backdoor enabled installation of other apps, dial numbers, send messages, and report back to the mothership. The manufacturer even ran the command-and-control nodes for the malware.
  2. USB Driveby — dongle that plugs into USB, and tries to root the box. Specifically, when you normally plug in a mouse or keyboard into a machine, no authorization is required to begin using them. The devices can simply begin typing and clicking. We exploit this fact by sending arbitrary keystrokes meant to launch specific applications (via Spotlight/Alfred/Quicksilver), permanently evade a local firewall (Little Snitch), install a reverse shell in crontab, and even modify DNS settings without any additional permissions.
  3. Physical Data Visualisationsa chronological list of physical visualizations and related artifacts. (via Flowing Data)
  4. Dissentan anonymous communication substrate intended primarily for applications built on a broadcast communication model: for example, bulletin boards, wikis, auctions, or voting. Users of an online group obtain cryptographic guarantees of sender and receiver anonymity, message integrity, disruption resistance, proportionality, and location hiding. And a pony.
Comment
Four short links: 11 December 2014

Four short links: 11 December 2014

Crowdsourcing Framework, Data Team Culture, Everybody Scrolls, and Honeypot Data

  1. Hive — open source crowdsourcing framework from NYT Labs.
  2. Prezi Data Team Culture — good docs on logging, metrics, etc. The vision is a great place to start.
  3. Scroll Behaviour Across the Web (Chartbeat) — nobody reads above the fold, they immediately scroll.
  4. threat_research (github) — shared raw data and stats from honeypots.
Comment
Four short links: 9 December 2014

Four short links: 9 December 2014

Tab Tool, Ad Manifesto, Cultural Heritage, and Software Sustainability

  1. tab — command-line tool for doing heavy lifting with tab-separated files.
  2. Acceptable Ads — manifesto from the makers of AdBlock Plus. (via Monday Note)
  3. Cultural Heritage of Humanity (Matt Webb) — Matt points to UNESCO’s Representative List of the Intangible Cultural Heritage of Humanity and comments: When the aliens land and set up shop and they’re like, “Guys, so what have you got?” And we’re all… “Uh, lasers? We’ll trade you lasers for a starship drive.” And the aliens will be: “Nope, what else?” Then we’ll say: “Tsiattista poetic duelling. Turkish coffee. Jazz.” Bingo. Kudos to UNESCO for prepping our inventory ahead of time.
  4. Apache (and Other Foundations) Considered Useful (Chris Aniszczyk) — have over a decade of experience being built for the sole purpose of allowing independent open source communities to flourish with fair governance models […] This is important because the incentives between individuals small companies, large companies, heavily funded companies and even academics are different and need to be accounted for in a fair open source governance structure. Sustainability of software commons is an unsolved problem, but foundations make it tractable.
Comments: 5

Signals from Strata + Hadoop World in Barcelona 2014

From the Internet of Things to data-driven fashion, here are key insights from Strata + Hadoop World in Barcelona 2014.

Experts from across the big data world came together for Strata + Hadoop World in Barcelona 2014. We’ve gathered insights from the event below.

#IoTH: The Internet of Things and Humans

“If we could start over with these capabilities we have now, how would we do it differently?” Tim O’Reilly continues to explore data and the Internet of Things through the lens of human empowerment and the ability to “use technology to give people superpowers.”

Read more…

Comment: 1
Four short links: 14 November 2014

Four short links: 14 November 2014

Completing Maps, ChatOps, Career Design, and Data Privacy

  1. Missing Maps Fill In the Blanks (New Scientist) — OpenStreetMap project to crowdmap slums around the world.
  2. Chatops — devops deployment chatter with Hubot.
  3. Alternatives to Tech Career Ladders — Spotify trying to figure out how to keep engineers challenged as they become more senior.
  4. Mozilla’s Data Privacy Principles — well-articulated and useful: without pre-defined principles, it’s so easy to accidentally collect or poorly protect data.
Comment