ENTRIES TAGGED "data"

Four short links: 5 August 2014

Four short links: 5 August 2014

Discussion Graph Tool, Superlinear Productivity, Go Concurrency, and R Map/Reduce Tools

  1. Discussion Graph Tool (Microsoft Research) — simplifies social media analysis by making it easy to extract high-level features and co-occurrence relationships from raw data.
  2. Superlinear Productivity in Collective Group Actions (PLoS ONE) — study of open source projects shows small groups exhibit non-linear productivity increases by size, which drop off at larger sizes. we document a size effect in the strength and variability of the superlinear effect, with smaller groups exhibiting widely distributed superlinear exponents, some of them characterizing highly productive teams. In contrast, large groups tend to have a smaller superlinearity and less variability.
  3. coop — cheat sheet of the most common concurrency program flows in Go.
  4. Tessera — set of open source tools around Hadoop, R, and visualization.
Comment
Four short links: 4 August 2014

Four short links: 4 August 2014

Web Spreadsheet, Correlated Novelty, A/B Ethics, and Replicated Data Structures

  1. EtherCalcopen source web-based spreadsheet.
  2. Dynamics of Correlated Novelties (Nature) — paper on “the adjacent possible”. Here we propose a simple mathematical model that mimics the process of exploring a physical, biological, or conceptual space that enlarges whenever a novelty occurs. The model, a generalization of Polya’s urn, predicts statistical laws for the rate at which novelties happen (Heaps’ law) and for the probability distribution on the space explored (Zipf’s law), as well as signatures of the process by which one novelty sets the stage for another. (via Steven Strogatz)
  3. On The Media Interview with OKCupid CEO — relevant to the debate on ethics of A/B tests. I preferred this to Tim Carmody’s rant.
  4. CRDTs as Alternative to APIswhen using CRDTs to tie your system together, you don’t need to resort to using impoverished representations that simply never come anywhere near the representational power of the data structures you use in your programs at runtime. See also this paper on Convergent and Commutative Replicated Data Types.
Comment

Health games platforms mature in preparation for mainstream adoption

Business models and sustainability will drive success in the health games space.

SPARX_screenshot

SPARX, a behavioral therapy game for youths,
combines a fantasy setting with skills for life.

For the past several years, researchers have strived to create compelling games that improve behavior, reduce stress, or teach healthy responses to difficult life situations. Such healthy games tend to arise in research settings because of the need to demonstrate clinically that the games are effective. I have covered such efforts in my postings from the Games for Health conference in 2012 and 2013.

These efforts have born fruit, and clinical trials have shown the value of many such games. Ben Sawyer, who founded the Games for Health conference more than 10 years ago, is watching all the pieces fall into place for the widespread adoption of games. Business plans, platforms, and the general environment for the acceptance of games (and other health-related apps) are coming together.

Read more…

Comment
Four short links: 15 July 2014

Four short links: 15 July 2014

Data Brokers, Car Data, Pattern Classification, and Hogwild Deep Learning

  1. Inside Data Brokers — very readable explanation of the data brokers and how their information is used to track advertising effectiveness.
  2. Elon, I Want My Data! — Telsa don’t give you access to the data that your cars collects. Bodes poorly for the Internet of Sealed Boxes. (via BoingBoing)
  3. Pattern Classification (Github) — collection of tutorials and examples for solving and understanding machine learning and pattern classification tasks.
  4. HOGWILD! (PDF) — the algorithm that Microsoft credit with the success of their Adam deep learning system.
Comment
Four short links: 11 July 2014

Four short links: 11 July 2014

Curated Code, Hackable Browser, IoT Should Be Open, and Better Treemaps

  1. Awesome Awesomeness — list of curated collections of frameworks and libraries in various languages that do not suck. They solve the problem of “so, I’m new to (language) and don’t want to kiss a lot of frogs before I find the right tool for a particular task”.
  2. Breach — a hackable, modular web browser.
  3. The CompuServe of Things (Phil Windley) — How we build the Internet of Things has far-reaching consequences for the humans who will use—or be used by—it. Will we push forward, connecting things using forests of silos that are reminiscent the online services of the 1980′s, or will we learn the lessons of the Internet and build a true Internet of Things? (via Cory Doctorow)
  4. FoamTree — nifty treemap layouts and animations, in Javascript. (via Flowing Data)
Comment: 1
Four short links: 9 July 2014

Four short links: 9 July 2014

Developer Inequality, Weak Signals, Geek Feminism Wiki, and Reidentification Risks

  1. Developer Inequality (Jonathan Edwards) — The bigger injustice is that programming has become an elite: a vocation requiring rare talents, grueling training, and total dedication. The way things are today if you want to be a programmer you had best be someone like me on the autism spectrum who has spent their entire life mastering vast realms of arcane knowledge — and enjoys it. Normal humans are effectively excluded from developing software. (via Slashdot)
  2. Signals From Foo Camp (O’Reilly Radar) — useful for me (aka “the stuff I didn’t get to see”), hopefully useful to you too. Companies outside of Silicon Valley badly want to understand it and want to find ways to truly collaborate with it, but they’re worried that conversations can turn into competition. “Old industry” has incredible expertise and operates in very complex environments, and it has much to teach tech, if tech will listen. Silicon Valley isn’t an IT department for the world, it’s the competition.
  3. Feminist Point of View: Lessons from Running the Geek Feminism Wiki — deck from Alex’s OS Bridge session. Today’s awareness and actions around sexism in tech resulted from their actions, sometimes directly, sometimes indirectly.
  4. Big Data Should Not Be a Faith-Based Initiative (Cory Doctorow) — Re-identification is part of the Big Data revolution: among the new meanings we are learning to extract from huge corpuses of data is the identity of the people in that dataset. And since we’re commodifying and sharing these huge datasets, they will still be around in ten, twenty and fifty years, when those same Big Data advancements open up new ways of re-identifying — and harming — their subjects.
Comment

Roll-your-own database architecture

Making the case for blended architectures in the rapidly evolving universe of advanced analytics.

Kenny_Louie_Squares_Circles

Two years ago, most of the conversations around big data had a futuristic, theoretical vibe. That vibe has been replaced with a gritty sense of practically. Today, when big data or some surrogate term arises in conversation, the talk is likely to focus not on “what if,” but on “how do we get it done?” and “what will it cost?”

Real-time big data analytics and the increasing need for applications capable of handling mixed read/write workloads — as well as transactions and analytics on “hot” data — are putting new pressures on traditional data management architectures.

What’s driving the need for change? There are several factors, including a new class of apps for personalizing the Internet, serving dynamic content, and creating rich user experiences. These apps are data driven, which means they essentially feed on deep data analytics. You’ll need a steady supply of activity history, insights, and transactions, plus the ability to combine historical analytics with hot analytics and read/write transactions. Read more…

Comment

Extracting value from the IoT

Data from the Internet of Things makes an integrated data strategy vital.

Union_Pacific

Union Pacific uses infrared and audio sensors placed on its tracks to gauge the state of wheels and bearings as the trains pass by.

The Internet of Things (IoT) is more than a network of smart toasters, refrigerators, and thermostats. For the moment, though, domestic appliances are the most visible aspect of the IoT. But they represent merely the tip of a very large and mostly invisible iceberg.

IDC predicts by the end of 2020, the IoT will encompass 212 billion “things,” including hardware we tend not to think about: compressors, pumps, generators, turbines, blowers, rotary kilns, oil-drilling equipment, conveyer belts, diesel locomotives, and medical imaging scanners, to name a few. Sensors embedded in such machines and devices use the IoT to transmit data on such metrics as vibration, temperature, humidity, wind speed, location, fuel consumption, radiation levels, and hundreds of other variables. Read more…

Comment: 1

Where did the issue of health data exchange disappear to?

More visible at Health Privacy Summit than Health Datapalooza.

On the first morning of the biggest conference on data in health care–the Health Datapalooza in Washington, DC–newspapers reported a bill allowing the Department of Veterans Affairs to outsource more of its care, sending veterans to private health care providers to relieve its burdensome shortage of doctors.

There has been extensive talk about the scandals at the VA and remedies for them, including the political and financial ramifications of partial privatization. Republicans have suggested it for some time, but for the solution to be picked up by socialist Independent Senator Bernie Sanders clinches the matter. What no one has pointed out yet, however–and what makes this development relevant to the Datapalooza–is that such a reform will make the free flow of patient information between providers more crucial than ever.

Read more…

Comment
Four short links: 29 May 2014

Four short links: 29 May 2014

Modern Software Development, Internet Trends, Software Ethics, and Open Government Data

  1. Beyond the Stack (Mike Loukides) — tools and processes to support software developers who are as massively distributed as the code they build.
  2. Mary Meeker’s Internet Trends 2014 (PDF) — the changes on slide 34 are interesting: usage moving away from G+/Facebook-style omniblather creepware and towards phonebook-based chat apps.
  3. Introduction to Software Engineering Ethics (PDF) — amazing set of provocative questions and scenarios for software engineers about the decisions they made and consequences of their actions. From a course in ethics from SCU.
  4. Open Government Data Online: Impenetrable (Guardian) — Too much knowledge gets trapped in multi-page pdf files that are slow to download (especially in low-bandwidth areas), costly to print, and unavailable for computer analysis until someone manually or automatically extracts the raw data.
Comment