- Infrastructure for Data Streams — describing the high-volume log data use case for Apache Kafka, and how it plays out in storage and infrastructure.
- Obama: Treat Broadband and Mobile as Utility (Ars Technica) — In short, Obama is siding with consumer advocates who have lobbied for months in favor of reclassification while the telecommunications industry lobbied against it.
- MozVR — a website, and the tools that made it, designed to be seen through the Oculus Rift.
- All Cameras are Police Cameras (James Bridle) — how the slippery slope is ridden: When the Wall was initially constructed, the public were informed that this [automatic license plate recognition] data would only be held, and regularly purged, by Transport for London, who oversee traffic matters in the city. However, within less than five years, the Home Secretary gave the Metropolitan Police full access to this system, which allowed them to take a complete copy of the data produced by the system. This permission to access the data was granted to the Police on the sole condition that they only used it when National Security was under threat. But since the data was now in their possession, the Police reclassified it as “Crime” data and now use it for general policing matters, despite the wording of the original permission. As this data is not considered to be “personal data” within the definition of the law, the Police are under no obligation to destroy it, and may retain their ongoing record of all vehicle movements within the city for as long as they desire.
The evolving marketplace is making new data applications and interactions possible.
Here’s a look at some options in the evolving, maturing marketplace of big data components that are making the new applications and interactions we’ve been looking at possible.
First used in social network analysis, graph theory is finding more and more homes in research and business. Machine learning systems can scale up fast with tools like Parameter Server, and the TitanDB project means developers have a robust set of tools to use.
Are graphs poised to take their place alongside relational database management systems (RDBMS), object storage, and other fundamental data building blocks? What are the new applications for such tools?
Inside the black box of algorithms: whither regulation?It’s possible for a machine to create an algorithm no human can understand. Evolutionary approaches to algorithmic optimization can result in inscrutable, yet demonstrably better, computational solutions.
If you’re a regulated bank, you need to share your algorithms with regulators. But if you’re a private trader, you’re under no such constraints. And having to explain your algorithms limits how you can generate them.
As more and more of our lives are governed by code that decides what’s best for us, replacing laws, actuarial tables, personal trainers, and personal shoppers, oversight means opening up the black box of algorithms so they can be regulated.
Years ago, Orbitz was shown to be charging web visitors who owned Apple devices more money than those visiting via other platforms, such as the PC. Only that’s not the whole story: Orbitz’s machine learning algorithms, which optimized revenue per customer, learned that the visitor’s browser was a predictor of their willingness to pay more. Read more…
Andreas Antonopoulos urges the Canadian Senate to resist the temptation to centralize bitcoin.
Editor’s note: our O’Reilly Radar Summit: Bitcoin & the Blockchain will take place on January 27, 2015, at Fort Mason in San Francisco. Andreas Antonopoulos, Vitalik Buterin, Naval Ravikant, and Bill Janeway are but a few of the confirmed speakers for the event. Learn more about the event and reserve your ticket here.
We recently announced a Radar summit on present and future applications of cryptocurrencies and blockchain technologies. In a webcast presentation one of our program chairs, Kieren James-Lubin, observed that we’re very much in the early days of these technologies. He also noted that the technologies are complex enough that most users will rely on service providers (like wallets) to securely store, transfer, and receive cryptocurrencies.
As some of these service providers reach a certain scale, they will start coming under the scrutiny of regulators. Certain tenets are likely to remain: currencies require continuous liquidity and large financial institutions need access to the lender of last resort.
There are also cultural norms that take time to change. Take the example of notaries, whose services seem amenable to being replaced by blockchain technologies. Such a wholesale change would entail adjusting rules and norms across localities, which means going up against the lobbying efforts of established incumbents.
One way to sway regulators and skeptics is to point out that the decentralized nature of the (bitcoin) blockchain can unlock innovation in financial services and other industries. Mastering Bitcoin author Andreas Antonopoulos did a masterful job highlighting this in his recent testimony before the Canadian Senate:
“Traditional models for financial payment networks and banking rely on centralized control in order to provide security. The architecture of a traditional financial network is built around a central authority, such as a clearinghouse. As a result, security and authority have to be vested in that central actor. The resulting security model looks like a series of concentric circles with very limited access to the center and increasing access as we move farther away from the center. However, even the most outermost circle cannot afford open access.
The tech entrepreneur turned legislator on open government, data, regulatory reform and his new foundation.
An interview with Congressman Darrell Issa (R-CA) on open government, personal data ownership, a digital Bill of Rights, Internet freedom, regulation, and more.
Open Source Implants, Gut Fungus, Closed Source Damage, and Microtask Framework
- When Code Can Kill or Cure (The Economist) — I’ve linked to the dangers of closed source devices before, but this caught my eye: “In the 1990s we developed an excellent radiation-therapy treatment-planning system and tried to give it away to other clinics,” says Dr Mackie. “But when we were told by the FDA that we should get our software approved, the hospital wasn’t willing to fund it.” He formed a spin-off firm specifically to get FDA approval. It took four years and cost millions of dollars. The software was subsequently sold as a traditional, closed-source product.
- Gut Fungus (Wired) — the microbiome of bacteria in your body is being studied, but now researchers have scoured the poop of different species and found different mycological populations in each, and linked them to diseases.
- Evaluating the Harm from Closed Source (Eric Raymond) — whether or not you argue with his ethics, you will appreciate the clear description of the things you’re trading off when you choose to use closed source software.
- PyBossa — a free, open-source, platform for creating and running crowd-sourcing applications that utilise online assistance in performing tasks that require human cognition, knowledge or intelligence such as image classification, transcription, geocoding and more! (via The Open Knowledge Foundation)
Almost anything can be claimed as a copyright violation if you don't have to defend the claim.
SOPA and Protect IP are proposing remedies to copyright violation that never come under the scrutiny of the legal system.
SOPA and PROTECT IP would harm innovation.
In a time when the American economy needs to catalyze innovation to compete in a global marketplace, members of the United States Congress have advanced legislation that could cripple the Internet industry, damage cybersecurity and harm freedom of expression online.