"open data" entries
Tracking the data storm around Hurricane Sandy
When natural disasters loom, public open government data feeds become critical infrastructure.
Just over fourteen months ago, social, mapping and mobile data told the story of Hurricane Irene. As a larger, more unusual late October storm churns its way up the East Coast, the people in its path are once again acting as sensors and media, creating crisis data as this “Frankenstorm” moves over them.
[Photo Credit: NASA}
In that context, it’s key to understand that it’s government weather data, gathered and shared from satellites high above the Earth, that’s being used by a huge number of infomediaries to forecast, predict and instruct people about what to expect and what to do. In perhaps the most impressive mashup of social and government data now online, an interactive Google Crisis Map for Hurricane Sandy pictured below predicts the future of the ‘Frankenstorm’ in real-time, including a NYC-specific version.
If you’re looking for a great example of public data for public good, these maps like the Weather Underground’s interactive are a canonical example of what’s possible.
San Francisco looks to tap into the open data economy
With revised legislation and a chief data officer, San Francisco is iterating on its platform goals.
As interest in open data continues to grow around the world, cities have become laboratories for participatory democracy. They’re also ground zero for new experiments in spawning civic startups that deliver city services or enable new relationships between the people and city government. San Francisco was one of the first municipalities in the United States to embrace the city as a platform paradigm in 2009, with the launch of an open data platform.
Years later, the city government is pushing to use its open data to accelerate economic development. On Monday, San Francisco announced revised open data legislation to enable that change and highlighted civic entrepreneurs who are putting the city’s data to work in new mobile apps.
City staff have already published the revised open data legislation on GitHub. (If other cities want to “fork” it, clone away.) David Chiu, the chairman of the San Francisco Board of Supervisors, the city’s legislative body, introduced the new version on Monday and submitted it on Tuesday. A vote is expected before the end of the year.
Speaking at the offices of the Hatchery in San Francisco, Chiu observed that, by and large, the data that San Francisco has put out showed the city in a positive light. In the future, he suggested, that should change. Chiu challenged the city and the smartest citizens of San Francisco to release more data, figure out where the city could take risks, be more entrepreneurial and use data to hold the city accountable. In his remarks, he said that San Francisco is working on open budgeting but is still months away from getting the data that they need. Read more…
Data from health care reviews could power “Yelp for health care” startups
Data-driven decision engines will need patient experience to complete the feedback loop.
Given where my work and health has taken me this year, I’ve been thinking much more about the relationship of the Internet and health data to accountability and patient-driven health care.
When I was looking for a place in Maine to go for care this summer, I went online to look at my options. I consulted hospital data from the government at HospitalCompare.HHS.gov and patient feedback data on Yelp, and then made a decision based upon proximity and those ratings. If I had been closer to where I live in Washington D.C., I would also have consulted friends, peers or neighbors for their recommendations of local medical establishments.
My brush with needing to find health care when I was far from home reminded me of the prism that collective intelligence can now provide for the treatment choices we make, if we have access to the Internet.
Patients today are sharing more of their health data and experiences online voluntarily, which in turn means that the Internet is shaping health care. There’s a growing phenomenon of “e-patients” and caregivers going online to find communities and information about illness and disability.
Aided by search engines and social media, newly empowered patients are discussing health conditions with others suffering from disease and sickness — and they’re taking that peer-to-peer health care knowledge into their doctors’ offices with them, frequently on mobile devices. E-patients are sharing their health data of their own volition because they have a serious health condition, want to get healthy, and are willing.
From the perspective of practicing physicians and hospitals, the trend of patients contributing to and consulting on online forums adds the potential for errors, fraud, or misunderstanding. And yet, I don’t think there’s any going back from a networked future of peer-to-peer health care, anymore than we can turn back the dial on networked politics or disaster response. Read more…
DataMarket charges up with open energy data
Want to build a business on open data? Add value by solving a problem for your users.
Hjalmar Gislason commented earlier this year that open data has been all about apps. In the future, it should be about much more than consumer-facing tools. “Think also about the less sexy cases that can help a few people save us millions of dollars in aggregate, generate new insights and improve decision making on various levels,” he suggested.
Today, the founder and CEO of DataMarket told the audience of the first White House Energy Datapalooza that his company would make energy data more discoverable and usable. In doing so, Datamarket will be be tapping into an emerging data economy of businesses using open government data.
“We are honored to have been invited to take part in this fantastic initiative,” said Gislason in a prepared statement. “At DataMarket we focus on doing one thing well: aggregating vast amounts of heterogeneous data to help business users with their planning and decision-making. Our new energy portal applies this know-how to the US government’s energy data, for the first time enabling these valuable resources to be searched, visualized and shared through one gateway and in combination with other domestic and worldwide open data sources.”
Energy.datamarket.com, which won’t go live officially until mid-October, will offer search for 10 thousand data sets, 2 million time series and 50 million energy facts. DataMarket.com is based upon data from thirteen different data providers including the U.S. Department of Energy’s Energy Information Agency (EIA), Oak Ridge National Laboratory, Energy Efficiency and Renewable Energy program, National Renewable Energy Laboratory, the Environmental Protection Agency (EPA), the Bureau of Transportation Statistics, the World Bank and United Nations.
Last week, I interviewed Gislason about his company and why they’re focusing on energy data.
Four key trends changing digital journalism and society
Commonalities between the Knight Foundation's News Challenge winners hint at journalism's networked future.
It’s not just a focus on data that connects the most recent class of Knight News Challenge winners. They all are part of a distributed civic media community that works on open source code, collects and improves data, and collaborates across media organizations.
These projects are “part of an infrastructure that helps journalists better understand and serve their communities through data,” commented Chris Sopher, Knight Foundation Journalism Program Associate, in an interview last week. To apply a coding metaphor, the Knight Foundation is funding the creation of patches for the source code of society. This isn’t a new focus: in 2011, Knight chose to help build the newsroom stack, from editorial search engines to data cleaning tools.
Following are four themes that jumped out when I looked across the winners of the latest Knight News Challenge round.
Networked accountability
An intercontinental project that bridged citizen science, open data, open source hardware, civic hacking and the Internet of things to monitor, share and map radiation data? Safecast is in its own category. Adapting the system to focus on air quality in Los Angeles — a city that’s known for its smog — will be an excellent stress test for seeing if this distributed approach to networked accountability can scale.
If it does — and hacked Chumbys, LED signs, Twitter bots, smartphone apps and local media reports start featuring the results — open data is going to be baked into how residents of Los Angeles understand their own atmosphere. If this project delivers on some of its promise, the value of this approach will be clearer.
If this project delivers on all of its potential, the air itself might improve. For that to happen, the people who are looking at the realities of air pollution will need to advocate for policy makers to improve it. In the future, the success or failure of this project will inform similar efforts that seek to enlist communities in data collection, including whether governments embrace “citizensourcing” beyond natural disasters and crises. The idea of citizens as sensors continues to have legs. Read more…
Knight winners are putting data to work
The common thread among the Knight Foundation's latest grants: practical application of open data.
Data, on its own, locked up or muddled with errors, does little good. Cleaned up, structured, analyzed and layered into stories, data can enhance our understanding of the most basic questions about our world, helping journalists to explain who, what, where, how and why changes are happening.
Last week, the Knight Foundation announced the winners of its first news challenge on data. These projects are each excellent examples of working on stuff that matters: they’re collective investments in our digital civic infrastructure. In the 20th century, civil society and media published the first websites. In the 21st century, civil society is creating, cleaning and publishing open data.
The grants not only support open data but validate its place in the media ecosystem of 2012. The Knight Foundation is funding data science, accelerating innovation in the journalism and media space to help inform and engage communities, a project that they consider “vital to democracy.”
Why? Consider the projects. Safecast creates networked accountability using sensors, citizen science and open source hardware. LocalData is a mobile method for communities to collect information about themselves and make sense of it. Open Elections will create a free, standardized database stream of election results. Development Seed will develop better tools to contribute to and use OpenStreetMap, the “Wikipedia of maps.” Pop Up Archive will develop an easier way to publish and archive multimedia data to the Internet. And Census.IRE.org will improve the ability of a connected nation and its data editors to access and use the work of U.S. Census Bureau.
The projects hint at a future of digital open government, journalism and society founded upon the principles that built the Internet and World Wide Web and strengthened by peer networks between data journalists and civil society. A river of open data flows through them all. The elements and code in them — small pieces, loosely joined by APIs, feeds and the social web — will extend the plumbing of digital democracy in the 21st century.
Want an NIH grant to build a better mobile health app? Connect your code to the research
The United States National Institutes of Health (NIH) wants to tie development of mobile health apps to evidence-based research, and it hopes to do that with a new grant program. The imperative to align developers with research is urgent, given the strong interest in health IT, mobile health and health data. There are significant challenges for the space, from consumer concerns over privacy and mobile applications to the broader question of balancing health data innovation with patient rights.
To learn more about what’s happening with mobile health apps, health data, behavioral change and cancer research, I recently interviewed Dr. Abdul Sheikh. Our interview, lightly edited for content and clarity, follows.
What led you to your current work at NIH?
Dr. Abdul Sheikh: I’ve always had a strong grounding in public health and population health, but I also have a real passion for technology and informatics. What’s beautiful is, in my current position here as a program director at the National Cancer Institute (NCI), I have a chance to meld these worlds of public health, behavior and communication science with my passion for technology and informatics. Some of the work I did before coming to the NIH was related to the early telemedicine and web-based health promotion efforts that the government of Canada was involved in.
At NCI, I direct a portfolio of research on technology-mediated communication. I’ve also had the chance to get involved and provide leadership on two very cool efforts. One of them is leadership for our division’s Small Business Innovation Research Program (SBIR). I’ve led the first NIH developer challenge competitions as well.
Strata Week: Big problems in the age of big data
Big data and big problems, open data monetization, Hortonworks' first year, and a new Hadoop Partner Ecosystem launches
Here are a few stories that caught my attention in the data space this week.
Big data, Big Brother, big problems
Adam Frank took a look at some of the big problems with big data this week over at NPR. Franks addresses issues in analyzing the sheer volume of complex information inherent in big data. Learning to sort through and mine vasts amounts of data to extrapolate meaning will be a “trick,” he writes, but it turns out the big problems with big data go deeper than volume.
Creating computer models to simulate complex systems with big data, Franks notes, ultimately creates something a bit different from reality: “the very act of bringing the equations over to digital form means you have changed them in subtle ways and that means you are solving a slightly different problem than the real-world version.” Analysis, therefore, “requires trained skepticism, sophistication and, remarkably, some level of intuition about the systems we study,” he writes.
Franks also raises the problem of big data becoming a threat to individuals within society:
“Everyday we are scattering ‘digital breadcrumbs’ into the data-verse. Credit card purchases, cell phone calls, Internet searches: Big Data means memory storage has become so cheap that all data about all those aspects of our lives can be harvested and put to use. And it’s exactly the use of all that harvested data that can pose a threat to society.”
The threat comes from the Big Brother aspect of being constantly monitored in ways we’ve never before imagined, and Franks writes, “It may also allows levels of manipulation that are new and truly unimaginable.” You can read more of Franks thoughts on what it means to live in the age of big data here. (We’ve covered related ethics issues with big data here on Strata.)
Congress launches Congress.gov in beta, doesn’t open the data
The Library of Congress launched a new website for a more mobile public to access legislative information
The Library of Congress is now more responsive — at least when it comes to web design. Today, the nation’s repository for its laws launched a new beta website at Congress.gov and announced that it would eventually replace Thomas.gov, the 17-year-old website that represented one of the first significant forays online for Congress. The new website will educate the public looking for information on their mobile devices about the lawmaking process, but it falls short of the full promise of embracing the power of the Internet. (More on that later).
Tapping into a growing trend in government new media, the new Congress.gov features responsive design, adapting to desktop, tablet or smartphone screens. It’s also search-centric, with Boolean search and, in an acknowledgement that most of its visitors show up looking for information, puts a search field front and center in the interface. The site includes member profiles for U.S. Senators and Representatives, with associated legislative work. In a nod to a mainstay of social media and media websites, the new Congress.gov also has a “most viewed bills” list that lets visitors see at a glance what laws or proposals are gathering interest online. (You can download a fact sheet on all the changes as a PDF).
On the one hand, the new Congress.gov is a dramatic update to a site that desperately needed one, particularly in a historic moment where citizens are increasingly connecting to the Internet (and one another) through their mobile devices.
On the other hand, the new Congress.gov beta has yet to realize the potential of Congress publishing bulk open legislative data. There is no application programming interface (API) for open government developers to build upon. In many ways, the new Congress.gov replicates what was already available to the public at sites like Govtrack.us and OpenCongress.org. Read more…
Breaking down the data barriers that hold back cancer research
Marcia Kean of Feinstein Kean Healthcare answered a request by the Institute of Medicine to design a better way to do cancer research, and helped write a proposal that led to the establishment of the Data Liquidity Coalition. In this video interview, Kean describes the benefits of data sharing for researchers, pharmaceutical companies, and patients. She describes the goals and initial activities of the coalition, and describes the effects it can have on research, patients, and physicians.
Highlights from the full video interview include:
- Reasons for the Data Liquidity Coalition. [Discussed at the 0:16 mark]
- Origins at the Institute of Medicine. [Discussed at the 1:20 mark]
- Value of data sharing for cancer researchers. [Discussed at the 2:30 mark]
- Value of data sharing for patients and their physicians. [Discussed at the 5:05 mark]
- Proprietary systems and open interfaces. [Discussed at the 6:45 mark]
- Getting patients involved and developing advocates for sharing. [Discussed at the 10:04 mark]
- Clinical studies in an ecosystem of large data sets. [Discussed at the 14:31 mark]
- 15:22 Roles for computer programmers, IT businesses. [Discussed at the 15:22 mark]
- 17:34 Why the time is ripe for data liquidity. [Discussed at the 17:34 mark]
- Role models for data sharing. [Discussed at the 20:03 mark]