- Web Traffic Visualization — Dots enter when transactions start and exit when completed. Their speed is proportional to client’s response time while their size reflects the server’s contribution to total time. Color comes from the specific request. (via Nelson Minar)
- Complete Guide to Being Interviewed on TV (Quartz) — good preparation for everyone who runs the risk of being quoted for 15 seconds.
- Harlan (GitHub) — new language for GPU programming. Simple examples in the announcement. (via Michael Bernstein)
- Open Fit — open source software that investigates several approaches to generating custom tailored pants patterns. Open Fit Lab is an attempt to use this software for on-the-spot generation and creation of custom clothes. (via Kaitlin Thaney)
ENTRIES TAGGED "visualization"
Web Traffic Visualisation, TV Interviews, GPU Programming, and Programmatic Pants Design
Facebook Pub/Sub, Space/Time Visualization, Sean That Matters, and Keyboard Control
- Wormhole — Facebook’s pub/sub system. Wormhole propagates changes issued in one system to all systems that need to reflect those changes – within and across data centers.
- Nanocubes — Fast Visualization of Large Spatiotemporal Datasets.
- Sean Gourley on Relevance (YouTube) — Is Silicon Valley really doing what it should be doing? he asks, 3m30 in. Good to see him pondering stuff that matters, back in 2011.
- Shortcat — a keyboard tool for Mac OS X that lets you “click” buttons and control your apps with a few keystrokes. Think of it as Spotlight for the user interface.
Velocity 2013 Speaker Series
Be honest, have you ever wanted to play Steve Souders for a day and pull some revealing stats or trends about some web sites of your choice? Or maybe dig around the HTTP archive? You can do that and more by setting up your own HTTP Archive.
httparchive.org is a fantastic tool to track, monitor, and review how the web is built. You can dig into trends around page size, page load time, content delivery network (CDN) usage, distribution of different mimetypes, and many other stats. With the integration of WebPagetest, it’s a great tool for synthetic testing as well.
You can download an HTTP Archive MySQL dump (warning: it’s quite large) and the source code from the download page and dissect a snapshot of the data yourself. Once you’ve set up the database, you can easily query anything you want.
You need MySQL, PHP, and your own webserver running. As I mentioned above, HTTP Archive relies on WebPagetest—if you choose to run your own private instance of WebPagetest, you won’t have to request an API key. I decided to ask Patrick Meenan for an API key with limited query access. That was sufficient for me at the time. If I ever wanted to use more than 200 page loads per day, I would probably want to set up a private instance of WebPagetest.
To find more details on how to set up an HTTP Archive instance yourself and any further advice, please check out my blog post.
Going back to the scenario I described above: the real motivation is that often you don’t want to throw your website(s) in a pile of other websites (e.g. not related to your business) to compare or define trends. Our digital property at the Canadian Broadcasting Corporation’s (CBC) spans over dozens of URLs that have different purposes and audiences. For example, CBC Radio covers most of the Canadian radio landscape, CBC News offers the latest breaking news, CBC Hockey Night in Canada offers great insights on anything related to hockey, and CBC Video is the home for any video available on CBC. It’s valuable for us to not only compare cbc.ca to the top 100K Alexa sites but also to verify stats and data against our own pool of web sites.
In this case, we want to use a set of predefined URLs that we can collect HTTP Archive stats for. Hence a private instance can come in handy—we can run tests every day, or every week, or just every month to gather information about the performance of the sites we’ve selected. From there, it’s easy to not only compare trends from httparchive.org to our own instance as a performance baseline, but also have a great amount of data in our local database to run queries against and to do proper performance monitoring and investigation.
The beautiful thing about having your own instance is that you can be your own master of data visualization: you can now create more charts in addition to the ones that came out of the box with the default HTTP Archive setup. And if you don’t like Google chart tools, you may even want to check out D3.js or Highcharts instead.
The image below shows all mime types used by CBC web properties that are captured in our HTTP archive database, using D3.js bubble charts for visualization.
3D Visualization, Printing On Any Surface, Rebuilding Reality, and Emotions as Data
- For Example — amazing discussion of 3D visualization techniques, full of examples using the D3.js library and bl.ocks.org example gist system. Gorgeous and informative.
- Anti-Gravity 3D Printer — uses strands to sculpt on any surface. (via Slashdot)
- How 3D Printing Will Rebuild Reality (BoingBoing) — But even though home 3D-printing has received substantial publicity of late, it is in the industrial sector where the technology will probably make its most significant near-term impact on the world both by manufacturing improved commercial products and by stimulating industry to develop next-generation fab methods and machines that could one day truly bring 3D-printing home to users in a real way.
- The Emotional Side of Big Data — Personal Democracy Forum 2013 talk by Sara Critchfield, on reframing emotion as data for decision-making. (via Quartz)
Stephen Malinowski's latest music visualization celebrates The Rite of Spring's 100th anniversary.
Stephen Malinowski’s hypnotic music visualizations have been quite a hit on YouTube — he has visualized a number of scores, from Debussy’s, Clair de lune to Chopin’s Nocturne in B Major, opus 32 no.1 to his own Fugue in A minor. Anastasia Tsioulcas reports at NPR that Malinowski’s visualizations have garnered more than 100 million page views. And just in time to celebrate the 100th anniversary of Stravinsky’s ballet The Rite of Spring at the end of May, Malinowski created a visualization of the score:
The New York Times and The Washington Post created visualizations using data released by The Centers for Medicare and Medicaid Services.
The Centers for Medicare and Medicaid Services released procedure billing data on more than 3,000 U.S. hospitals. The New York Times and The Washington Post have put together interactive visualizations to help consumers compare costs. The New York Times’ visualization compares costs on a per-hospital basis:Read more…
Inside NASDAQ's Failbook, SimAustralia, Distraction Attraction, and Big Brother Says "Wash Your Hands!"
- Facebook IPO Tech Post-Mortem (PDF) — SEC’s analysis of the failures that led to the NASDAQ kicking Facebook’s IPO in the NADSAQ. (via Quartz)
- Run That Town — SimCity for real cities, from the Australian Bureau of Statistics and using real census data. No mention of whether you can make your citizens shout “Aussie Aussie Aussie, Oi Oi Oi!” after three cans of lager at an Aussie Rules game. (via John Birmingham)
- Maintaining Focus (The Atlantic) — excellent Linda Stone interview. We may think that kids have a natural fascination with phones. Really, children have a fascination with what-ever Mom and Dad find fascinating. If they are fascinated by the flowers coming up in the yard, that’s what the children are going to find fascinating. And if Mom and Dad can’t put down the device with the screen, the child is going to think, That’s where it’s all at, that’s where I need to be! I interviewed kids between the ages of 7 and 12 about this. They said things like “My mom should make eye contact with me when she talks to me” and “I used to watch TV with my dad, but now he has his iPad, and I watch by myself.”
- Networked Motion Sensors in Hospital Bathrooms (NY Times) — At North Shore University Hospital on Long Island, motion sensors, like those used for burglar alarms, go off every time someone enters an intensive care room. The sensor triggers a video camera, which transmits its images halfway around the world to India, where workers are checking to see if doctors and nurses are performing a critical procedure: washing their hands. [...] the video monitoring program, run by a company called Arrowsight, has been adapted from the meat industry, where cameras track whether workers who skin animals — the hide can contaminate the meat — wash their hands, knives and electric cutters.
Geeky Primer, Visible CSS, Remote Working, and Raspberry Pi Sentiment Server
- My Little Geek — children’s primer with a geeky bent. A is for Android, B is for Binary, C is for Caffeine …. They have a Kickstarter for two sequels: numbers and shapes.
- Visible CSS Rules — Enter a url to see how the css rules interact with that page.
- How to Work Remotely — none of this is rocket science, it’s all true and things we had to learn the hard way.
- Raspberry Pi Twitter Sentiment Server — step-by-step guide, and github repo for the lazy. (via Jason Bell)
Weekly Highlights and Insights: May 19-25
In-memory Databases: A discussion of near real-time manipulation of massive datasets
Burning the Silos: Minimize boundaries to reduce product cycle times.
Dart Is Not the Language You Think It Is: Seth Ladd’s enthusiastic reintroduction of Dart prompts a voluminous discussion on Slashdot.
Is that the Google Glass?: The anthropology of an always-on society
Tracing the Disappeared: An interactive visualization of CIA rendition flights
Weekly Highlights and Insights: May 13-17
Google I/O: O’Reilly Editor Rachel Roumeliotis reports from the conference floor.
Big Data, Cool Kids: Fumbling toward the adolescence of big data tools.
Real-time World-wide Wikipedia Edits: Stephen LaPorte and Mahmoud Hashemi’s addictive visualization.
Future of Open Source: The quality, security, and community driving open source adoption.