- AIRPrint — prototype box scans a fingerprint from six feet away. (via Greg Linden)
- Squishy Circuits — teaching electronic circuits with conductive and insulating playdough. (via Hacker News)
- GraphLab — alternative take on Map-Reduce, called Update-Sync, where tasks run on connected sets of nodes rather than on one node at a time.
- Tower Bridge Closed — the @towerbridge account was a cute hack from Tom Armitage, whereby the public site for the London Tower Bridge was scraped and connected to Twitter, so you would see tweets like “I am closing after the MV Dixie has passed Upstream” and get a feel for the ambient activity in your city. Twitter turned over @towerbridge to the most tediously vomit-in-your-own-mouth-they’re-so-anodyne beige corporate tweets ever (account description: “Leading tourist attraction situated inside Tower Bridge”, sample tweet: “Looking for something to do it the City this weekend, check out http://www.visitthecity.co.uk/ and you’re always welcome at @TowerBridge”) and deleted the past history of tweets. Way to embrace the community of engaged passionate fans, guys! Welcome to Twitter, try not to step in your social media strategy as you cross the threshold–oh no, too late.
"data mining" entries
Four short links: 13 June 2011
Remote Fingerprint Scans, Playdough Circuits, Update-Sync, and Tweet Failage
Four short links: 3 June 2011
Distributed Drug Money, Science Game, Beautiful Machine Learning, and Stream Event Processing
- Silk Road (Gawker) — Tor-delivered “web” site that is like an eBay for drugs, currency is Bitcoins. Jeff Garzik, a member of the Bitcoin core development team, says in an email that bitcoin is not as anonymous as the denizens of Silk Road would like to believe. He explains that because all Bitcoin transactions are recorded in a public log, though the identities of all the parties are anonymous, law enforcement could use sophisticated network analysis techniques to parse the transaction flow and track down individual Bitcoin users. “Attempting major illicit transactions with bitcoin, given existing statistical analysis techniques deployed in the field by law enforcement, is pretty damned dumb,” he says. The site is viewable here, and here’s a discussion of delivering hidden web sites with Tor. (via Nelson Minar)
- Dr Waller — a big game using DC Comics characters where players end up crowdsourcing science on GalaxyZoo. A nice variant on the captcha/ESP-style game that Luis von Ahn is known for. (via BoingBoing)
- Machine Learning Demos — hypnotically beautiful. Code for download.
- Esper — stream event processing engine, GPLv2-licensed Java. (via Stream Event Processing with Esper and Edd Dumbill)
Four short links: 24 May 2011
Kindle List, Insider Knowledge, Google News Archive Archived, and Work Week in Video
- Delivereads — genius idea, a mailing list for Kindles. Yes, if you can send email then you can be a Kindle publisher. (via Sacha Judd)
- Abnormal Returns From the Common Stock Investments of Members of the U.S. House of Representatives — We measure abnormal returns for more than 16,000 common stock transactions made by approximately 300 House delegates from 1985 to 2001. Consistent with the study of Senatorial trading activity, we find stocks purchased by Representatives also earn significant positive abnormal returns (albeit considerably smaller returns). A portfolio that mimics the purchases of House Members beats the market by 55 basis points per month (approximately 6% annually). (via Ellen Miller)
- Google News Archive Ends — hypothesizes that old material was “too hard” to make sense of, but that seems unlikely to me. More likely is that it wasn’t useful enough to their machine learning efforts. Newspapers can have their scanned/OCRed content for free now the program is being closed.
- Week Report 310 — BERG’s first (that I’ve seen) video report of the week, and it’s a cracker. No newsreel, just some really clever evocation of the mood of the place and the nature of the projects. I continue to be impressed by the BERG crew’s conscious creation of culture.
Strata Week: Overcharging algorithms
Algorithms go awry on Amazon, the future of Hadoop at Yahoo, and the Supreme Court mulls data mining
In this Strata Week: Algorithm pricing on Amazon pushes the price of a biology book to astronomical levels, Yahoo weighs the future of Hadoop, and the Supreme Court hears arguments about a Vermont law restricting the data mining of prescription records.
Four short links: 25 February 2011
Banshee Bucks, Log Mining, Visualization Secrets, and Repression Tools
- Canonical’s New Plan for Banshee — Canonical prepare the Linux distribution Ubuntu. They will distribute the popular iTunes-alike Banshee, but instead of the standard Amazon store plugin (which generates much $ in affiliate revenue for the GNOME Foundation) they will have Canonical’s own Amazon store plugin and keep 75% of the revenue (25% going to the GNOME Foundation). They’re legally within their rights, and it underscores for me how the goal of providing freedom from control is incompatible with a goal of making money. Free and open source software gives self-destination with software, and that includes the right to replace your money pump with theirs.
- Oluolu — an open source query log mining tool which works on Hadoop. This tool provides resources to add new features to search engines. Concretely Oluolu supports automatic dictionary creation such as spelling correction, context queries or frequent query n-grams from query log data. The dictionaries are applied to search engines to add features such as ‘did you mean’ or ‘related keyword suggestion’ service in search engines. (via Matt Biddulph on Delicious)
- Information is Beautiful Process (David McCandless) — David’s process for creating his beautiful and moving visualizations.
- Facebook for Repressive Regimes — The purpose of this blog post is not to help repressive regimes use Facebook better, but rather to warn activists about the risks they face when using Facebook. (via Justine Sanderson on Delicious)
Four short links: 26 January 2011
Identifying Communities, Web Principles, Wiring Library, and Instapaper Interview
- Find Communities — algorithm for uncovering communities in networks of millions of nodes, for producing identifiable subgroups as in LinkedIn InMaps. (via Matt Biddulph’s Delicious links)
- Seven Ways to Think Like The Web (Jon Udell) — seven principles that will head off a lot of mistakes. They should be seared into the minds of anyone working in the web. 2. Pass by reference rather than by value. [pass URLs, not copies of data] […] Why? Nobody else cares about your data as much as you do. If other people and other systems source your data from a canonical URL that you advertise and control, then they will always get data that’s as timely and accurate as you care to make it.
- Wire It — an open-source javascript library to create web wirable interfaces for dataflow applications, visual programming languages, graphical modeling, or graph editors. (via Pete Warden)
- Interview with Marco Arment (Rands in Repose) — Most people assume that online readers primarily view a small number of big-name sites. Nearly everyone who guesses at Instapaper’s top-saved-domain list and its proportions is wrong. The most-saved site is usually The New York Times, The Guardian, or another major traditional newspaper. But it’s only about 2% of all saved articles. The top 10 saved domains are only about 11% of saved articles. (via Courtney Johnston’s Instapaper Feed)
Strata gems: What your inbox knows
Mining implicit data trails makes CRM more effective
One of the richest sources of data exhaust, email logs contain valuable information. When added to data from a traditional CRM, email analytics can provide a much fuller picture of your company's relationships and activity.
Four short links: 17 December 2010
Systems Programming, Peer Review, Web Mining, Facebook Design
- Down the ls(1) Rabbit Hole — exactly how ls(1) does what it does, from logic to system calls to kernel. This is the kind of deep understanding of systems that lets great programmers cut great code. (via Hacker News)
- Towards a scientific concept of free will as a biological trait: spontaneous actions and decision-making in invertebrates (Royal Society) — peer-reviewed published paper that was initially reviewed and improved in Google Docs and got comments there, in FriendFeed, and on his blog. The bitter irony: Royal Society charged him €2000 to make it available for free download. (via Fabiana Kubke)
- Bixo — an open source web mining toolkit. (via Matt Biddulph on Delicious)
- How Facebook Does Design — podcast (with transcript) with stories about how tweaking design improved the user activity on Facebook. One of the designers thought closing your account should be more like leaving summer camp (you know a place which has all your friends, and you don’t want to leave.) So he created this page above for deactivation which has all your friends waving good-bye to you as you deactivate. Give you that final tug of the heart before you leave. This reduced the deactivation rate by 7%.
Four short links: 3 December 2010
Snake Oil, JSON v XML, Pac Man, and the Full Stack
- Data is Snake Oil (Pete Warden) — data is powerful but fickle. A lot of theoretically promising approaches don’t work because there’s so many barriers between spotting a possible relationship and turning it into something useful and actionable. This is the pin of reality which deflates the bubble of inflated expectations. Apologies for the camel’s nose of rhetoric poking under the metaphoric tent.
- XML vs the Web (James Clark) — resignation and understanding from one of the markup legends. I think the Web community has spoken, and it’s clear that what it wants is HTML5, JavaScript and JSON. XML isn’t going away but I see it being less and less a Web technology; it won’t be something that you send over the wire on the public Web, but just one of many technologies that are used on the server to manage and generate what you do send over the wire. (via Simon Willison)
- Understanding Pac Man Ghost Behaviour — The ghosts’ AI is very simple and short-sighted, which makes the complex behavior of the ghosts even more impressive. Ghosts only ever plan one step into the future as they move about the maze. Whenever a ghost enters a new tile, it looks ahead to the next tile that it will reach, and makes a decision about which direction it will turn when it gets there. Really detailed analysis of just one component of this very successful game. (via Hacker News)
- The Full Stack (Facebook) — we like to think that programming is easy. Programming is easy, but it is difficult to solve problems elegantly with programming. I like to think that a CS education teaches you this kind of “full stack” approach to looking at systems, but I suspect it’s a side-effect and not a deliberate output. This is the core skill of great devops: to know what’s happening up and down the stack so you’re not solving a problem at level 5 that causes problems at level 3.
Four short links: 1 November 2010
Crap Phones, HTML Editors, Digital Rights Minimization, and Data Munging
- The Most Popular Phone in the World (Gizmodo) — I have a mate who does prototyping R&D type stuff at a telco and this is his phone. “Why’d you carry a crap phone like that?” “Because this is the most popular phone with our customers.” The Gizmodo article talks about an upcoming Nokia that looks very promising: full keyboard, camera, et al. for under $100. (via Andrew Hedges on Twitter)
- Aloha Editor — very nice open source (AGPL3) HTML5 text editor widget for web apps. (via Jessy Cowan-Sharp on Twitter)
- How Do We Solve a Problem Like Geographic Restrictions — if you’re building a new business in the US around ebooks, digital music, or digital video, then be aware that your international uptake will be absolutely buggerized by rights issues. YouTube is the only US media site that doesn’t suck for overseas users: don’t rave to us about Hulu, it’s inaccessible to the rest of the world. (via Liza Daly on Twitter)
- Needlebase — tool with AI-type smarts to help you merge, munge, and export data. Check out Thread, the query language, for an interesting way of querying graphs. Was made by ITA Software, now owned by Google. Wonder what it’ll be wrapped into or released as …