"metrics" entries

Computing Twitter Influence, Part 1: Arriving at a Base Metric

The subtle variables affecting a base metric

This post introduces a series that explores the problem of approximating a Twitter account’s influence. With the ubiquity of social media and its effects on everything from how we shop to how we vote at the polls, it’s critical that we be able to employ reasonably accurate and well-understood measurements for approximating influence from social media signals.

Unlike social networks such as LinkedIn and Facebook in which connections between entities are symmetric and typically correspond to a real world connection, Twitter’s underlying data model is fundamentally predicated upon asymmetric following relationships. Another way of thinking about a following relationship is to consider that it’s little more than a subscription to a feed about some content of interest. In other words, when you follow another Twitter user, you are expressing interest in that other user and are opting-in to whatever content it would like to place in your home timeline. As such, Twitter’s underlying network structure can be interpreted as an interest graph and mined for insights about the relative popularity of one user when compared to another.
Read more…


The True Cost of Lemonade

Learn to resist vanity metrics

One of the things we preach in Lean Analytics is that entrepreneurs should avoid vanity metrics—numbers that make you feel good, but ultimately, don’t change your behavior. Vanity metrics (such as “total visitors”) tend to go “up and to the right” but don’t tell you much about how you’re doing.

Many people find solace in graphs that go up and to the right. The metric “Total number of people who have visited my restaurant” will always increase; but on its own it doesn’t tell you anything about the health of the business. It’s just head-in-the-sand comforting.

A good metric is often a comparative rate or ratio. Consider what happens when you put the word “per” before or after a metric. “Restaurant visitors per day” is vastly more meaningful. Time is the universal denominator, since the universe moves inexorably forwards. But there are plenty of other good ratios. For example, “revenue per restaurant visitor” matters a lot, since it tells you what each diner contributes.

What’s an active user, anyway?

For many businesses, the go-to metric revolves around “active users.” In a mobile app or software-as-a-service business, only some percentage of people are actively engaged. In a media site, only some percentage uses the site each day. And in a loyalty-focused e-commerce company, only some buyers are active.

This is true of more traditional businesses, too. Only a percentage of citizens are actively engaged in local government; only a certain number of employees are using the Intranet; only a percentage of coffee shop patrons return daily.

Unfortunately, saying “measure active users” begs the question: What’s active, anyway?

To figure this out, you need to look at your business model. Not your business plan, which is a hypothetical projection of how you’ll fare, but your business model. If you’re running a lemonade stand, your business model likely has a few key assumptions:

  • The cost of lemonade;
  • The amount of foot traffic past your stand;
  • The percent of passers-by who will buy from you;
  • The price they are willing to pay.

Our Lean lemonade stand would then set about testing and improving each metric, running experiments to find the best street corner, or determine the optimal price.

Lemonade stands are wonderfully simple, so your business may have many other assumptions, but it is essential that you quantify them and state them so you can then focus on improving them, one by one, until your business model and reality align. In a restaurant, for example, these assumptions might be, “we will have at least 50 diners a day” or “diners will spend on average $20 a meal.”

The activity you want changes

We believe most new companies and products go through five distinct stages of growth:

  • Empathy, where you figure out what problem you’re solving and what solution people want;
  • Stickiness, where you measure how many people adopt your solution rather than trying it and leaving;
  • Virality, where you maximize word-of-mouth and references;
  • Revenue, where you pour some part of your revenues back into paid acquisition or advertising;
  • Scale, where you grow the business through automation, delegation, and process.

Read more…

Four short links: 11 April 2013

Four short links: 11 April 2013

Automating NES Games, Code Review Tool, SaaS KPIs, and No Free Lunch

  1. A General Technique for Automating NES Gamessoftware that learns how to play NES games and plays them automatically, using an aesthetically pleasing technique. With video, research paper, and code.
  2. rietveld — open source tool like Mondrian, Google’s code review tool. Developed by Guido van Rossum, who developed Mondrian. Still being actively developed. (via Nelson Minar)
  3. KPI Dashboard for Early-Stage SaaS Startups — as Google Docs sheet. Nice.
  4. Life Without Sleep — interesting critique of Provigil as performance-enhancing drug for information workers. It is very difficult to design a stimulant that offers focus without tunnelling – that is, without losing the ability to relate well to one’s wider environment and therefore make socially nuanced decisions. Irritability and impatience grate on team dynamics and social skills, but such nuances are usually missed in drug studies, where they are usually treated as unreliable self-reported data. These problems were largely ignored in the early enthusiasm for drug-based ways to reduce sleep. […] Volunteers on the stimulant modafinil omitted these feedback requests, instead providing brusque, non-question instructions, such as: ‘Exit West at the roundabout, then turn left at the park.’ Their dialogues were shorter and they produced less accurate maps than control volunteers. What is more, modafinil causes an overestimation of one’s own performance: those individuals on modafinil not only performed worse, but were less likely to notice that they did. (via Dave Pell)
Four short links: 14 January 2013

Four short links: 14 January 2013

Open Source Metrics, BitTorrent to TV, Tumblr Value, and Variable Fiction

  1. Open Source MetricsTalking about the health of the project based on a single metric is meaningless. It is definitely a waste of time to talk about the health of a project based on metrics like number of software downloads and mailing list activities. Amen!
  2. BitTorrent To Your TVThe first ever certified BitTorrent Android box goes on sale today, allowing users to stream files downloaded with uTorrent wirelessly to their television. The new set-top box supports playback of all popular video formats and can also download torrents by itself, fully anonymously if needed. (via Andy Baio)
  3. Tumblr URL Culture — the FOO.tumblr.com namespace is scarce and there’s non-financial speculation. People hoard and trade URLs, whose value is that they say “I’m cool and quirky”. I’m interested because it’s a weird largely-invisible Internet barter economy. Here’s a rant against it. (via Beta Knowledge)
  4. Design-Fiction Slider Bar of Disbelief (Bruce Sterling) — I love the list as much as the diagram. He lays out a sliding scale from “objective reality” to “holy relics” and positions black propaganda, 419 frauds, design pitches, user feedback, and software code on that scale (among many other things). Bruce is an avuncular Loki, pulling you aside and messing with your head for your own good.
Comments: 2
Four short links: 9 November 2012

Four short links: 9 November 2012

Civil Drones, Fencing the Public Domain, Quantified Spy, and Data Daemons for Fun and Metrics

  1. Helping Drones Play Nice With Other AviationThe U.S. airspace is quickly being filled with simultaneously flying drones. To such an extent, unmanned aircraft could soon become a nightmare for the ATC controllers. The ADS-B will improve Predator B’s crew situational awareness making the drone capable to operate more freely and safely in domestic and international airspace in accordance with civilian air traffic and airspace rules and regulations.
  2. Reclaiming NZ’s Digitised HeritageOut of a sample of 100 books: 50% of NZ Heritage Books (published before 1890) have been digitised; 90% of digitised texts are fully accessible; 98% of accessible texts are downloadable; Despite all works being in the public domain, only one did not have any licencing restrictions applied to its use. Most groups who digitise then go on to put restrictions around their use. [T]here are also many instances where arbitrary restrictions are being applied to the detriment of the public good.
  3. Self-Spy (GitHub) — Log everything you do on the computer, for statistics, future reference and all-around fun!
  4. statsd (GitHub) — Etsy’s data-gathering daemon, written up in an excellent blog post.
Four short links: 7 November 2012

Four short links: 7 November 2012

Relativity Toys, Removing Metrics, Parallel Open Source, and Text Karaoke

  1. A Slower Speed of Light — game where you control the speed of light and discover the wonders of relativity. (via Andy Baio)
  2. Facebook Demetricator — removes all statistics and numbers from Facebook’s chrome (“37 people like this” becomes “people like this”). (via Beta Knowledge)
  3. Rx — Microsoft open sources their library for composing asynchronous and event-based programs using observable sequences and LINQ-style query operators.
  4. Typing Karaoke — this is awesome. Practice typing to song lyrics. With 8-bit aesthetic for maximum quirk.
Comment: 1
Four short links: 30 August 2012

Four short links: 30 August 2012

Decoding ToS, Impact Factors are Nonsense, Crappy Open Source Code, and Data Mining History

  1. TOS;DR — terms of service rendered comprehensible. “Make the hard stuff easy” is a great template for good ideas, and this just nails it.
  2. Sick of Impact Factorstypically only 15% of the papers in a journal account for half the total citations. Therefore only this minority of the articles has more than the average number of citations denoted by the journal impact factor. Take a moment to think about what that means: the vast majority of the journal’s papers — fully 85% — have fewer citations than the average. The impact factor is a statistically indefensible indicator of journal performance; it flatters to deceive, distributing credit that has been earned by only a small fraction of its published papers. (via Sci Blogs)
  3. A Generation Lost in the Bazaar (ACM) — Today’s Unix/Posix-like operating systems, even including IBM’s z/OS mainframe version, as seen with 1980 eyes are identical; yet the 31,085 lines of configure for libtool still check if and exist, even though the Unixen, which lacked them, had neither sufficient memory to execute libtool nor disks big enough for its 16-MB source code. […] That is the sorry reality of the bazaar Raymond praised in his book: a pile of old festering hacks, endlessly copied and pasted by a clueless generation of IT “professionals” who wouldn’t recognize sound IT architecture if you hit them over the head with it. It is hard to believe today, but under this embarrassing mess lies the ruins of the beautiful cathedral of Unix, deservedly famous for its simplicity of design, its economy of features, and its elegance of execution. (Sic transit gloria mundi, etc.)
  4. History as Science (Nature) — Turchin and his allies contend that the time is ripe to revisit general laws, thanks to tools such as nonlinear mathematics, simulations that can model the interactions of thousands or millions of individuals at once, and informatics technologies for gathering and analysing huge databases of historical information.
Comments: 3

Joshua Bixby on the business of performance

Why businesses should care about speed.

In this Velocity Podcast, Strangeloop's Joshua Bixby discusses the business of speed and why web performance optimization is an institutional need.

Four short links: 20 March 2012

Four short links: 20 March 2012

jQuery Video Plugin, Open Source Data View, QR Insanity, and Measuring Citizen Science

  1. jPlayer — jQuery plugin for audio and video in HTML5. Dual-licensed MIT and GPL.
  2. Tesseract (Github) — Square has open sourced (Apache license) their Javascript library for filtering large multidimensional datasets in the browser. Tesseract supports extremely fast (<30ms) interaction with coordinated views, even with datasets containing a million or more records; we built it to power analytics for Square Register, allowing merchants to slice and dice their payment history fluidly.
  3. QR Code MadnessI recently received an MMS (multimedia text message) with a picture to a QR code. First, it’s bad enough advertising agencies still randomly text people ads. Second, what am I supposed to scan that with? My eyes? But check out the photo for maximum silliness.
  4. Galaxy Zoo: Crowdsourcing Citizen Scientists (Guardian) — yes, the headline is a collection of buzzwords but the Galaxy Zoo project remains fantastic. My eye was caught by Working 12 hours a day non-stop for a week, [Kevin] Schawinski had managed the not inconsiderable task of detailing the characteristics of 50,000 galaxies. He needed a pint. [… they built Galaxy Zoo in a day of two …] Within 24 hours of it being announced on Lintott’s website, Galaxy Zoo was receiving 70,000 classifications an hour. They still measure their hit-rate in “Kevin weeks” – a unit of 50,000. “Soon after that we were doing many Kevin weeks per hour,” Schawinski says. (via Roger Dennis)
Comment: 1

Mobile analytics unlock the what and the when

Flurry's Sean Byrnes on mobile metrics and tablet apps vs phone apps.

Flurry's CTO Sean Byrnes discusses app life cycles, the specifics of user engagement, and the difference between smartphone apps and tablet apps.