ENTRIES TAGGED "text analysis"

Four short links: 9 January 2012

Four short links: 9 January 2012

Apple Factories, Open Source Spy Drones, Mail Files, and Text Topic Extraction

  1. Mr Daisey and the Apple Factor (This American Life) — episode looking at the claims of human rights problems in Apple’s Chinese factories.
  2. OpenPilot — open source UAVs with cameras. Yes, a DIY spy drone on autopilot. (via Jim Stogdill)
  3. mbox — more technical information than you ever thought you’d need, to be saved for the time when you have to parse mailbox files. It’s a nightmare. (via Hacker News)
  4. Maui (Google Code) — Maui automatically identifies main topics in text documents. Depending on the task, topics are tags, keywords, keyphrases, vocabulary terms, descriptors, index terms or titles of Wikipedia articles. GPLv3.
Comment: 1
Four short links: 28 December 2011

Four short links: 28 December 2011

Text Search, Cloud Filesystem, Javascript Parser, and Twitter Templates

  1. Terrier IR — open source (Mozilla) text search engine, now with Hadoop support.
  2. s3ql — open source (GPLv3) Linux filesystem which stores its data on Google Storage, Amazon S3, or OpenStack. (via Adam Shand)
  3. Esprima — open source (BSD) fast Javascript parser in Javascript. (via Javascript Weekly)
  4. Hogan.js — open source (Apache) Javascript templating engine from Twitter. If it proves anywhere near as good as Bootstrap, it’ll be heavily used.
Comment
Four short links: 26 December 2011

Four short links: 26 December 2011

Text Analysis Bundle, Scala Probabilistic Modeling, Game Analytics, and Encouraging Writing

  1. Pattern — a BSD-licensed bundle of Python tools for data retrieval, text analysis, and data visualization. If you were going to get started with accessible data (Twitter, Google), the fundamentals of analysis (entity extraction, clustering), and some basic visualizations of graph relationships, you could do a lot worse than to start here.
  2. Factorie (Google Code) — Apache-licensed Scala library for a probabilistic modeling technique successfully applied to [...] named entity recognition, entity resolution, relation extraction, parsing, schema matching, ontology alignment, latent-variable generative models, including latent Dirichlet allocation. The state-of-the-art big data analysis tools are increasingly open source, presumably because the value lies in their application not in their existence. This is good news for everyone with a new application.
  3. Playtomic — analytics as a service for gaming companies to learn what players actually do in their games. There aren’t many fields untouched by analytics.
  4. Write or Die — iPad app for writers where, if you don’t keep writing, it begins to delete what you wrote earlier. Good for production to deadlines; reflective editing and deep thought not included.
Comment
Four short links: 22 December 2011

Four short links: 22 December 2011

Fuzzy Text, Big Data Crime, Map Visualization, and Attacking Server-Side Javascript

  1. Fuzzy String Matching in Python (Streamhacker) — useful if you’re to have a hope against the swelling dark forces powered by illiteracy and touchscreen keyboards.
  2. The Business of Illegal Data (Strata Conference) — fascinating presentation on criminal use of big data. “The more data you produce, the happier criminals are to receive and use it. Big data is big business for organized crime, which represents 15% of GDP.”
  3. Isarithmic Maps — an alternative to chloropleths for geodata visualization.
  4. Server-Side Javascript Injection (PDF) — a Blackhat talk about exploiting backend vulnerabilities with techniques learned from attacking Javascript frontends. Both this paper and the accompanying talk will discuss security vulnerabilities that can arise when software developers create applications or modules for use with JavaScript-based server applications such as NoSQL database engines or Node.js web servers. In the worst-case scenario, an attacker can exploit these vulnerabilities to upload and execute arbitrary binary files on the server machine, effectively granting him full control over the server.
Comment
Four short links: 9 December 2011

Four short links: 9 December 2011

Designing Ubicomp, Online Community, Design Examples, and Ranking Discussions

  1. Critically Making the Internet of Things (Anne Galloway) — session notes from a conference, see also part two. Good thoughts, hastily captured. For example, this from Bruce Sterling: RFID + Superglue + Object ≠ IoT and the talk I want to see: “A study of how broken, hacked and malfunctioning digital road signs subvert the physical space of roadways.”
  2. Conquering the CHAOS of Online Community at StackExchange — StackExchange is doing some thoughtful work analysing conversations and channeling dissent into a healthy construction to guide future productive discussion. “We taught the users that it was alright to disagree, and gave them a set of arguments they could reference without every thread degenerating into a fight.”
  3. Little Big Details — one small detail done right, every day.
  4. Ranking Live Streams of Data (LinkedIn) — behind the “interesting discussions” report.
Comment
Four short links: 6 December 2011

Four short links: 6 December 2011

Dispel Your Illusions, Simple Mac OS X Apps, Assisted Translation, and AutoTagging

  1. How to Dispel Your Illusions (NY Review of Books) — Freeman Dyson writing about Daniel Kahneman’s latest book. Only by understanding our cognitive illusions can we hope to transcend them.
  2. Appify-UI (github) — Create the simplest possible Mac OS X apps. Uses HTML5 for the UI. Supports scripting with anything and everything. (via Hacker News)
  3. Translation Memory (Etsy) — using Lucene/SOLR to help automate the translation of their UI. (via Twitter)
  4. Automatically Tagging Entities with Descriptive Phrases (PDF) — Microsoft Research paper on automated tagging. Under the hood it uses Map/Reduce and the Microsoft Dryad framework. (via Ben Lorica)
Comment
Four short links: 18 November 2011

Four short links: 18 November 2011

Quantified Learner, Text Extraction, Backup Flickr, and Multitouch UI Awesomeness

  1. Learning With Quantified Self — this CS grad student broke Jeopardy records using an app he built himself to quantify and improve his ability to answer Jeopardy questions in different categories. This is an impressive short talk and well worth watching.
  2. Evaluating Text Extraction AlgorithmsThe gold standard of both datasets was produced by human annotators. 14 different algorithms were evaluated in terms of precision, recall and F1 score. The results have show that the best opensource solution is the boilerpipe library. (via Hacker News)
  3. Parallel Flickr — tool for backing up your Flickr account. (Compare to one day of Flickr photos printed out)
  4. Quneo Multitouch Open Source MIDI and USB Pad (Kickstarter) — interesting to see companies using Kickstarter to seed interest in a product. This one looks a doozie: pads, sliders, rotary sensors, with LEDs underneath and open source drivers and SDK. Looks almost sophisticated enough to drive emacs :-)
Comment
Four short links: 15 November 2011

Four short links: 15 November 2011

Internet Asthma Care, C Fulltext, Citizen Science, and Mozilla

  1. Cost-Effectiveness of Internet-Based Self-Management Compared with Usual Care in Asthma (PLoSone) — Internet-based self-management of asthma can be as effective as current asthma care and costs are similar.
  2. Apache Lucy full-text search engine library written in C and targeted at dynamic languages. It is a “loose C” port of Apache Lucene™, a search engine library for Java.
  3. The Near Future of Citizen Science (Fiona Romeo) — near future of science is all about honing the division of labour between professionals, amateurs and bots. See Bryce’s bionic software riff. (via Matt Jones)
  4. Microsoft’s Patent Claims Against Android (Groklaw) — behold, citizen, the formidable might of Microsoft’s patents and how they justify a royalty from every Android device equal to that which you would owe if you built a Windows Mobile device: These Microsoft patents can be divided into several basic categories: (1) the ’372 and ’780 patents relate to web browsers; (2) the ’551 and ’233 patents relate to electronic document annotation and highlighting; (3) the ’522 patent relates to resources provided by operating systems; (4) the ’517 and ’352 patents deal with compatibility with file names once employed by old, unused, and outmoded operating systems; (5) the ’536 and ’853 patents relate to simulating mouse inputs using non-mouse devices; and (6) the ’913 patent relates to storing input/output access factors in a shared data structure. A shabby display of patent menacing.
Comment
Four short links: 14 November 2011

Four short links: 14 November 2011

Science Hack Days, YouTube Doggy Science, Antisocial Software, and Mind Reading with Wikipedia

  1. Science Hack Day SF Videos (justin.tv) — the demos from Science Hack Day SF. The journey of a thousand miles starts with a Hack Day.
  2. A Cross-Sectional Study of Canine Tail-Chasing and Human Responses to It, Using a Free Video-Sharing Website (PLoSone) — Approximately one third of tail-chasing dogs showed clinical signs, including habitual (daily or “all the time”) or perseverative (difficult to distract) performance of the behaviour. These signs were observed across diverse breeds. Clinical signs appeared virtually unrecognised by the video owners and commenting viewers; laughter was recorded in 55% of videos, encouragement in 43%, and the commonest viewer descriptors were that the behaviour was “funny” (46%) or “cute” (42%).
  3. RSS Died For Your Sins (Danny O’Brien) — if you have seven thousand people following you, a good six thousand of those are going to be people you don’t particularly like. The problem, as ever, is—how do you pick out the other thousand? Especially when they keep changing? I firmly believe that one of the pressing unsolved technological problems of the modern age is getting safely away from people you don’t like, without actually throttling them to death beforehand, nor somehow coming to the conclusion that they don’t exist, nor ending up turning yourself into a hateful monster.
  4. Generating Text from Functional Brain Images (Frontiers in Human Neuroscience) — We built a model of the mental semantic representation of concrete concepts from text data and learned to map aspects of such representation to patterns of activation in the corresponding brain image. Turns out that the clustering of concepts in Wikipedia is similar to how they’re clustered in the brain. They found clusters in Wikipedia, mapped to the brain activity for known words, and then used that mapping to find words for new images of brain activity. (via The Economist)
Comment
Four short links: 21 October 2011

Four short links: 21 October 2011

Mozilla's Projects, YouTube Insults, iPhone Ultrasound, RoR Intro

  1. What Mozilla is Up To (Luke Wroblewski) — notes from a talk that Brendan Eich gave at Web 2.0 Summit. The new browser war is between the Web and new walled gardens of native networked apps. Interesting to see the effort Mozilla’s putting into native-alike Web apps.
  2. YouTube Insult Generator (Adrian Holovaty) — mines YouTube for insults of a particular form.
  3. Ultrasound for iPhone (Geekwire) — this personal sensor is $8000 today, but bound to drop. I want personal ultrasound at least once a month. How long until it’s in the $200-500 range? (via BERG London)
  4. Web Applications Class at Stanford OpenClassroom — a Ruby on Rails class taught by John Ousterhout, creator of TCL/Tk and log-structured filesystems.
Comment: 1