- Computer Software Archive (Jason Scott) — The Internet Archive is the largest collection of historical software online in the world. Find me someone bigger. Through these terabytes (!) of software, the whole of the software landscape of the last 50 years is settling in. (And documentation and magazines and …). Wow.
- 7 in 10 Doctors Have a Self-Tracking Patient — the most common ways of sharing data with a doctor, according to the physicians, were writing it out by hand or giving the doctor a paper printout. (via Richard MacManus)
- opsmezzo — open-sourced provisioning tools from the Nodejitsu team. (via Nuno Job)
- Hacking Secret Ciphers with Python — teaches complete beginners how to program in the Python programming language. The book features the source code to several ciphers and hacking programs for these ciphers. The programs include the Caesar cipher, transposition cipher, simple substitution cipher, multiplicative & affine ciphers, Vigenere cipher, and hacking programs for each of these ciphers. The final chapters cover the modern RSA cipher and public key cryptography.
Software is adding more and more value to machines. Could it completely commoditize them?
I’m a sucker for a good plant tour, and I had a really good one last week when Jim Stogdill and I visited K. Venkatesh Prasad at Ford Motor in Dearborn, Mich. I gave a seminar and we talked at length about Ford’s OpenXC program and its approach to building software platforms.
The highlight of the visit was seeing the scale of Ford’s operation, and particularly the scale of its research and development organization. Prasad’s building is a half-mile into Ford’s vast research and engineering campus. It’s an endless grid of wet labs like you’d see at a university: test tubes and robots all over the place; separate labs for adhesives, textiles, vibration dampening; machines for evaluating what’s in reach for different-sized people.
Prasad explained that much of the R&D that goes into a car is conducted at suppliers–Ford might ask its steel supplier to come up with a lighter, stronger alloy, for instance–but Ford is responsible for integrative research: figuring out how to, say, bond its foam insulation onto that new alloy.
In our more fevered moments, we on the software side of things tend to foresee every problem being reduced to a generic software problem, solvable with brute-force computing and standard machinery. In that interpretation, a theoretical Google car operating system–one that would drive the car and provide Web-based services to passengers–could commoditize the mechanical aspects of the automobile. Read more…
Hardware is back and makers are driving it. Here are some of the signals.
Chris Anderson wrote Makers and went from editor-in-chief of Wired to CEO of 3D Robotics, making his hobby his side job and then making it his main job.
A new executive at Motorola Mobility, a division of Google, said that Google seeks to “googlify” hardware. By that he meant that devices would be inexpensive, if not free, and that the data created or accessed by them would be open. Motorola wants to build a truly hackable cellphone, one that makers might have ideas about what to do with it.
Regular hardware startup meetups, which started in San Francisco and New York, are now held in Boston, Pittsburgh, Austin, Chicago, Dallas and Detroit. I’m sure there are other American cities. Melbourne, Stockholm and Toronto are also organizing hardware meetups. Hardware entrepreneurs want to find each other and learn from each other.
Hardware-oriented incubators and accelerators are launching on both coasts in America, and in China.
The market for personal 3D printers and 3D printing services has really taken off. 3D printer startups continue to launch, and all of them seem to have trouble keeping up with demand. MakerBot is out raising money. Shapeways raised $30 million in a new round of financing announced this week.
Learn to build event-driven client and server applications
I want to build a web server, a mail server, a BitTorrent client, a DNS server, or an IRC bot—clients and servers for a custom protocol in Python. And I want them to be cross-platform, RFC-compliant, testable, and deployable in a standardized fashion. What library should I use?
Twisted is a “batteries included” networking engine for writing, testing, and deploying event-driven clients and servers in Python. It comes with off-the-shelf support for popular networking protocols like HTTP, IMAP, IRC, SMTP, POP3, IMAP, DNS, FTP, and more.
To see just how easy it is to write networking services using Twisted, let’s run and discuss a simple Twisted TCP echo server:
from twisted.internet import protocol, reactor
def dataReceived(self, data):
def buildProtocol(self, addr):
With Twisted installed, if we save this code to echoserver.py and run it with python echoserver.py, clients can now connect to the service on port 8000, send it data, and get back their echoed results. Read more…
Software Archive, Self-Tracking, Provisioning, and Python Ciphers
Establishing an effective organization for large-scale growth
In the open source and free software movement, we always exalt community, and say the people coding and supporting the software are more valuable than the software itself. Few communities have planned and philosophized as much about community-building as ZeroMQ. In the following posting, Pieter Hintjens quotes from his book ZeroMQ, talking about how he designed the community that works on this messaging library.
There are, it has been said (at least by people reading this sentence out loud), two ways to make really large-scale software. Option One is to throw massive amounts of money and problems at empires of smart people, and hope that what emerges is not yet another career killer. If you’re very lucky and are building on lots of experience, have kept your teams solid, and are not aiming for technical brilliance, and are furthermore incredibly lucky, it works.
But gambling with hundreds of millions of others’ money isn’t for everyone. For the rest of us who want to build large-scale software, there’s Option Two, which is open source, and more specifically, free software. If you’re asking how the choice of software license is relevant to the scale of the software you build, that’s the right question.
The brilliant and visionary Eben Moglen once said, roughly, that a free software license is the contract on which a community builds. When I heard this, about ten years ago, the idea came to me—Can we deliberately grow free software communities?
Learn to be a better programmer by taking charge of your interests
If you want to be a better programmer, a good first step would be to choose an area of software development to take additional responsibility for. Now, when we say “responsibility,” we don’t mean the sort of “you’re to blame and you accept it” responsibility that is so commonly thought of. Instead, we mean the willingness to take charge or the willingness to do something about an area.
So select out some area of software development, and decide to be a bit more responsible for it than one was before. The “area” could be simply some additional part of the current project you work on, the whole project itself, some type of software development that you haven’t done before, some aspect of software development you’d like to know more about, etc. If you’re feeling adventurous, try deciding that you’re personally responsible for the quality of the entire software project you’re working on. If you do, you may be surprised at how much easier this makes your life. When you’re trying to maintain the quality of only a piece of a software project, it’s very difficult. You’re surrounded by complexity or confusion, and you have to fend it off at every turn. But when you look at the project as a whole instead and try to decide what should be done with it as a whole, the solution presents itself much more readily. Now, it may seem like quite a bit more work to resolve the problems of the whole project, and it is. But it’s considerably more satisfying, tremendously more effective, and will bring you up to seniority as a software developer much more quickly than just trying to solve the problems of your one particular area.
O'Reilly's new site for all things related to programming.
Welcome to O’Reilly Media’s Programming blog, our resource for all things related to programming. Whether you’re a professional developer, hardcore hacker, or student, I hope this site provides you with interesting ideas, ways to learn new skills, exposure to alpha geeks, and the opportunity to interact with our talented and unique editors. The best part of my days are conversations with our editors and authors, and I’d like you to benefit from the same exchange of ideas.
We’re building this blog to meet your needs with news, information, and analysis. Whether you work on front ends, back ends, or middleware, open source software or commercial, there’s something here for you. Over the coming weeks and months we’ll be publishing how-to information, interviews, and our opinions in addition to exposing you to O’Reilly’s vast products and services.
We’ve seen an explosion of interest in the creation of software over the last two years. Groups like Codecademy promise to teach anyone willing to code, even people like Mayor Bloomberg of New York; one of the most important factors in the legal fight between Google and Oracle was a judge who knew how to code; and groups like Code for America are transforming the civic landscape.
Open source remains one of the pillars of the programming community, but we’re building a very large tent — a tent that includes Windows developers, iOS developers, Oracle developers, and more. We’ll also pay attention to non-technical issues that affect programmers: jobs, developer culture, and occasional tangents into other obsessions (like food).
O’Reilly’s readers created the world we inhabit. Now, we need to bring a new generation into that world and expand it for those already making a difference. While we have a lot of ideas for the Programming blog, we want to hear your thoughts about what you want to see on the site and how you want to participate. Please let us know what you think through the comments or email me directly.
Our new research report outlines our vision for the coming-together of software and big machines.
The big machines that define modern life — cars, airplanes, furnaces, and so forth — have become exquisitely efficient, safe, and responsive over the last century through constant mechanical refinement. But mechanical refinement has its limits, and there are enormous improvements to be wrung out of the way that big machines are operated: an efficient furnace is still wasteful if it heats a building that no one is using; a safe car is still dangerous in the hands of a bad driver.
It is this challenge that the industrial internet promises to address by layering smart software on top of machines. The last few years have seen enormous advances in software and computing that can handle gushing streams of data and build nuanced models of complex systems. These have been used effectively in advertising and web commerce, where data is easy to gather and control is easy to exert, and marketers have rejoiced.
Thanks to widespread sensors, pervasive networks, and standardized interfaces, similar software can interact with the physical world — harvesting data, analyzing it in context, and making adjustments in real-time. The same data-driven approach that gives us dynamic pricing on Amazon and customized recommendations on Foursquare has already started to make wind turbines more efficient and thermostats more responsive. It may soon obviate humans as drivers and help blast furnaces anticipate changes in electricity prices. Read more…
Being both liberal and safe in programming is hard
Recent discoveries of security vulnerabilities in Rails and MongoDB led me to thinking about how people get to write software.
In engineering, you don’t get to build a structure people can walk into without years of study. In software, we often write what the heck we want and go back to clean up the mess later. It works, but the consequences start to get pretty monumental when you consider the network effects of open source.
You might think it’s a consequence of the tools we use—running fast and loose with scripting languages. I’m not convinced. Unusually among computer science courses, my alma mater taught us programming 101 with Ada. Ada is a language that more or less requires a retinal scan before you can use the compiler. It was a royal pain to get Ada to do anything you wanted: the philosophical inverse of Perl or Ruby. We certainly came up the “hard way.”
I’m not sure that the hard way was any better: a language that protects you from yourself doesn’t teach you much about the problems you can create.
But perhaps we are in need of an inversion of philosophy. Where Internet programming is concerned, everyone is quick to quote Postel’s law: “Be conservative in what you do, be liberal in what you accept from others.”
The fact of it is that being liberal in what you accept is really hard. You basically have two options: look carefully for only the information you need, which I think is the spirit of Postel’s law, or implement something powerful that will take care of many use cases. This latter strategy, though seemingly quicker and more future-proof, is what often leads to bugs and security holes, as unintended applications of powerful parsers manifest themselves.
My conclusion is this: use whatever language makes sense, but be systematically paranoid. Be liberal in what you accept, but conservative about what you believe.
Ford's OpenXC platform opens up real-time drivetrain data.
OpenXC (Ford Motor) — Ford has taken a significant step in turning its cars into platforms for innovative developers. OpenXC goes beyond the Ford Developer Program, which opens up audio and navigation features, and lets developers get their hands on drivetrain and auto-body data via the on-board diagnostic port. Once you’ve built the vehicle interface from open-source parts, you can use outside intelligence — code running on an Android device — to analyze vehicle data.
Of course, as outside software gets closer to the drivetrain, security becomes more important. OpenXC is read-only at the moment, and it promises “proper hardware isolation to ensure you can’t ‘brick’ your $20,000 investment in a car.”
Still, there are plenty of sophisticated data-machine tieups that developers could build with read-only access to the drivetrain: think of apps that help drivers get better fuel economy by changing their acceleration or, eventually, apps that optimize battery cycles in electric vehicles.
Drivers with Full Hands Get a Backup: The Car (New York Times) — John Markoff takes a look at automatic driver aides — tools like dynamic cruise control and collision-avoidance warnings that represent something of a middle ground between driverless cars and completely manual vehicles. Some features like these have been around for years, many of them using ultrasonic proximity sensors. But some of these are special, and illustrative of an important element of the industrial Internet: they rely on computer vision like Google’s driverless car. Software is taking over some kinds of machine intelligence that had previously resided in specialized hardware, and it’s creating new kinds of intelligence that hadn’t existed in cars at all. Read more…