"Twitter" entries

Pattern-detection and Twitter’s Streaming API

In some key use cases a random sample of tweets can capture important patterns and trends

Researchers and companies who need social media data frequently turn to Twitter’s API to access a random sample of tweets. Those who can afford to pay (or have been granted access) use the more comprehensive feed (the firehose) available through a group of certified data resellers. Does the random sample of tweets allow you to capture important patterns and trends? I recently came across two papers that shed light on this question.

Systematic comparison of the Streaming API and the Firehose
A recent paper from ASU and CMU compared data from the streaming API and the firehose, and found mixed results. Let me highlight two cases addressed in the paper: identifying popular hashtags and influential users.

Of interest to many users is the list of top hashtags. Can one identify the “top n” hastags using data made available throughthe streaming API? The graph below is a comparison of the streaming API to the firehose: n (as in “top n” hashtags) vs. correlation (Kendall’s Tau). The researchers found that the streaming API provides a good list of hashtags when n is large, but is misleading for small n.

streaming api vs firehose

Read more…

Four short links: 31 May 2013

Four short links: 31 May 2013

  1. Modeling Users’ Activity on Twitter Networks: Validation of Dunbar’s Number (PLoSone) — In this paper we analyze a dataset of Twitter conversations collected across six months involving 1.7 million individuals and test the theoretical cognitive limit on the number of stable social relationships known as Dunbar’s number. We find that the data are in agreement with Dunbar’s result; users can entertain a maximum of 100–200 stable relationships. Thus, the ‘economy of attention’ is limited in the online world by cognitive and biological constraints as predicted by Dunbar’s theory. We propose a simple model for users’ behavior that includes finite priority queuing and time resources that reproduces the observed social behavior.
  2. Mary Meeker’s Internet Trends (Slideshare) — check out slide 24, ~2x month-on-month growth for MyFitnessPal’s number of API calls, which Meeker users as a proxy for “fitness data on mobile + wearable devices”.
  3. What I Learned as an Oompa Loompa (Elaine Wherry) — working in a chocolate factory, learning the differences and overlaps between a web startup and an more traditional physical goods business. It’s so much easier to build a sustainable organization around a simple revenue model. There are no tensions between ad partners, distribution sites, engineering, and sales teams. There are fewer points of failure. Instead, everyone is aligned towards a simple goal: make something people want.
  4. Augmented Reality Futures (Quartz) — wrap-up of tech in the works and coming. Instruction is the bit that interests me, scaffolding our lives: While it isn’t on the market yet, Inglobe Technologies just previewed an augmented reality app that tracks and virtually labels the components of a car engine in real time. That would make popping the hood of your car on the side of the road much less scary. The app claims to simplify tasks like checking oil and topping up coolant fluid, even for novice mechanics.

These are the top 20 investors to follow on Twitter? Really?

Finding the right people to follow for investment advice has very little to do with the extent of their social media following.

Business Insider really jumped the shark with their recent post entitled These Are The Top 20 Tech Investors You Should Follow On Twitter. It was clearly linkbait for social media rather than real advice for those looking for investment wisdom.  Ashton Kutcher (@aplusk) as the top investor to follow on Twitter?  Really?  When the greatest investor of all time, Warren Buffett (@WarrenBuffett), is also on Twitter?  Sure, Warren is new to Twitter and has only posted one link (to a fascinating article about why women are key to America’s prosperity), but when millions of investors hang on his every word, you’d think he’d get a mention. Ashton is great, but is he a better investor to pay attention to just because he has more “social media pull”?

This kind of story illustrates the vapidity of so much social media reporting.  What does someone’s social media following have to do with whether or not they are worth following for investment advice?

I’d prefer to follow investors who are good investors and who share their investment strategy!  That’s why I’d probably put Fred Wilson (@FredWilson) of Union Square Ventures (who was at an inexplicable number 19 on the Business Insider List) and his partners Brad Burnham (@BradUSV) and Alfred Wenger (@AlbertWenger) at the top.  Not only are they among the most successful tech investors active today (Twitter, Tumblr, Zynga, Foursquare, Etsy, Kickstarter, to name only a few of their investments), but they clearly explain their rationale for investing, their criteria, and their interests. Read more…

Four short links: 28 May 2013

Four short links: 28 May 2013

Geeky Primer, Visible CSS, Remote Working, and Raspberry Pi Sentiment Server

  1. My Little Geek — children’s primer with a geeky bent. A is for Android, B is for Binary, C is for Caffeine …. They have a Kickstarter for two sequels: numbers and shapes.
  2. Visible CSS RulesEnter a url to see how the css rules interact with that page.
  3. How to Work Remotely — none of this is rocket science, it’s all true and things we had to learn the hard way.
  4. Raspberry Pi Twitter Sentiment Server — step-by-step guide, and github repo for the lazy. (via Jason Bell)
Four short links: 25 April 2013

Four short links: 25 April 2013

iOS Package Manager, Designed Satire, API Fragility, and Retweeting WWI

  1. Alcatraz — package manager for iOS. (via Hacker News)
  2. Scarfolk Council — clever satire, the concept being a UK town stuck in 1979. Tupperware urns, “put old people down at birth”. The 1979 look is gorgeous. (via BoingBoing)
  3. Stop Designing Fragile Web APIsIt is possible to design your API in a manner that reduces its fragility and increases its resilience to change. The key is to design your API around its intent. In the SOA world, this is also referred to as business-orientation.
  4. @life100yearsago (Twitter) — account that tweets out fragments of New Zealand journals and newspapers and similar historic documents, as part of celebrating the surprising and the commonplace during WWI. My favourite so far: “Wizard” stones aeroplane. (via NDF)

Strata Week: Movers and shakers on the data journalism front

Reuters' Connected China, accessing Pew's datasets, Simon Rogers' move to Twitter, data privacy solutions, and Intel's shift away from chips.

Reuters launches Connected China, Pew instructs on downloading its data, and Twitter gets a data editor

Yue Qiu and Wenxiong Zhang took a look this week at a data journalism effort by Reuters, the Connected China visualization application. Qiu and Zhang report that “[o]ver the course of about 18 months, a dozen bilingual reporters based in Hong Kong dug into government websites, government reports, policy papers, Mainland major publications, English news reporting, academic texts, and think-tank reports to build up the database.”

Read more…

Finding and telling data-driven stories in billions of tweets

Twitter has hired Guardian Data editor Simon Rogers as its first data editor.

GD*15341872

Simon Rogers

Twitter has hired its first data editor. Simon Rogers, one of the leading practitioners of data journalism in the world, will join Twitter in May. He will be moving his family from London to San Francisco and applying his skills to telling data-driven stories using tweets. James Ball will replace him as the Guardian’s new data editor.

As a data editor, will Rogers keep editing and producing something that we’ll recognize as journalism? Will his work at Twitter be different than what Google Think or Facebook Stories delivers? Different in terms of how he tells stories with data? Or is the difference that Twitter has a lot more revenue coming in or sees data-driven storytelling as core to driving more business? (Rogers wouldn’t comment on those counts.)

Read more…

Four short links: 22 February 2013

Four short links: 22 February 2013

Indiepocalypse Continued, Unblockable p2p Twitter, Disposable Satellites, and iOS to HTML5

  1. Indiepocalypse: Harlem Shake Edition (Andy Baio) — “After four weeks topping the Billboard Hot 100, Macklemore and Ryan Lewis’s “Thrift Shop” was replaced this week by Baauer’s “Harlem Shake,” the song that inspired the Internet meme.”
  2. SplinterNet — an Android app designed to create an unblockable Twitter like network that uses no cellular or Internet communications. All messages are transmitted over Bluetooth between users, creating a true peer-to-peer messaging system. All messages are anonymous to prevent retaliation by government authorities. (via Ushahidi)
  3. Disposable Satellites (Forbes) — “tiny, near-disposable satellites for use in getting battlefield surveillance quickly […] launched from a jet into orbit, and within a few minutes […] provide soldiers on the ground with a zoomed-in, birds-eye view of the battlefield. Those image would be transmitted to current communications devices, and the company is working to develop a way to transmit them to smartphones, as well.”
  4. Native iOS to HTML5 Porting Tool (Intel) — essentially a source-to-source translator that can handle a number of conversions from Objective-C into JavaScript/HTML5 including the translation of APIs calls. A number of open source projects are used as foundation for the conversion including a modified version of Clang front-end, LayerD framework and jQuery Mobile for widgets rendering in the translated source code. A porting aid, not a complete translator but a lot of the dog work is done. Requires one convert to Microsoft tools, however. (via Kevin Marks)

Commerce Weekly: You can now buy stuff with tweets

AmEx now lets you buy with hashtags, 3D printing threats to retail, and PayPal comes to the gas pump.

American Express turns Twitter into an ecommerce platform

American Express announced an enhancement this week to its Sync with Twitter feature — users can now buy things with a tweet. Tricia Duryee reports at All Things Digital that all users will need to register to participate, even previous users of the sync feature, in order to provide a delivery address for purchased items. Once registration is complete, Duryee says, the purchasing process is pretty straightforward:

“For instance, participants will be able to buy a $25 American Express Gift Card for $15 … by tweeting #BuyAmexGiftCard25. American Express will reply via Twitter, asking the user to confirm the purchase in a tweet. All products will be shipped via free two-day shipping.”

Read more…

Exploring web standards for high data density visualizations

A sneak peek at an upcoming visualization session from the 2013 Strata Conference in Santa Clara, Calif.

Strata Editor’s Note: Over the next few weeks, the Strata Community Site will be providing sneak peeks of upcoming sessions at the Strata Conference in Santa Clara. Nicolas’ sneak peek is the first in this series. 

Last year was a great year for data visualization at Twitter. Our Analytics team expanded and created a dedicated data visualization team, and some of our projects were released publicly with great feedback.

Our first public interactive of 2012 was a fun way to expose how the Eurocup was experienced at Twitter. You can see in this organic visualization how people cheered for  their teams during each match, and how the tension and volume of  tweets increased towards the finals.

NB StrataSC 2013 image1

Read more…