- How Bad Software Leads to Bad Science — 21% of scientists who write software have never received training in software development.
- Roaring Bitmaps — compressed bitmaps which tend to outperform conventional compressed bitmaps such as WAH, EWAH or Concise. In some instances, they can be hundreds of times faster and they often offer significantly better compression.
- Two Eras of the Internet: From Pull to Push (Chris Dixon) — in which the consumer becomes the infinite sink for an unending and constant stream of updates, media, and social mobile local offers to swipe right on brands near you.
- Graphical Perception: Theory, Experimentation, and Application to the Development of Graphical Methods (PDF) — research on how well people decode visual cues. In order: Position along a common scale e.g. scatter plot; Position on identical but nonaligned scales e.g. multiple scatter plots; Length e.g. bar chart; Angle & Slope (tie) e.g. pie chart; Area e.g. bubbles; Volume, density, and color saturation (tie) e.g. heatmap; Color hue e.g. newsmap. (via Flowing Data)
A closer look at the forces causing demand
Buzzwords in the software industry arise and then die off with startling frequency. Ambiguous terms such as “growth hacker”, “sales engineer” and “rockstar developer” trip a developer’s spidey sense that the person saying them is just handwaving. However, occasionally a new term is created to articulate a programming skill set based on demand due to changes in the software development industry.
In 2013 the search term “full stack developer” took off on Google Trends and began appearing in numerous tech startup job postings. In this term’s case there are several real trends driving developers to invest in learning and identifying as full stack developers.
The usage of the full stack developer term is driven by several larger trends in software development. Read more…
Software and hardware are moving together, and the combined result is a new medium.
An updated version of this essay was published in December 2014.
The result is an entirely new medium that’s just beginning to emerge. We can see it in Ars Electronica Futurelab’s Spaxels, which use drones to render a three-dimensional pixel field; in Baxter, which layers emotive software onto an industrial robot so that anyone can operate it safely and efficiently; in OpenXC, which gives even hobbyist-level programmers access to the software in their cars; in SmartThings, which ties Web services to light switches.
The new medium is something broader than terms like “Internet of Things,” “Industrial Internet,” or “connected devices” suggest. It’s an entirely new discipline that’s being built by software developers, roboticists, manufacturers, hardware engineers, artists, and designers. Read more…
Business analytics projects: Using decisions as a basis to prioritize and identify requirements
Most normal people don’t look at data sets just for fun. They study views of the data to make decisions about what to do, be it a decision to take some specific action or a decision to do nothing at all. The main purpose of business analytics projects is to develop systems that turn large and often highly complex data sets into meaningful information from which decisions can be made.
The decisions that people make using business analytics systems can be strategic, operational, or tactical. For example, an executive might look at his sales team’s global performance dashboard to decide who to promote (tactical), which products need different marketing strategies (operational), or which products to target by markets (strategic). Generally speaking, all software systems that include an analytics component should enable users to make decisions that improve organizational performance in some dimension.
The App Store model has increased the uncertainty of the software release process
The recent unavailability of the Apple Developer’s Portal just underscores how increasingly dependent developers have become on third parties during the software lifecycle. For those who are not following the fun and games, the developer.apple.com sites, which include much of the functionality needed to develop Mac and iOS applications, has been unavailable for more than a week as of this writing. Although iTunes Connect, the portal used to actually deploy apps to the App Stores, has remained available, the remainder of the site territory has been off-limits. This is all thanks to a security intrusion (evidently by an over-zealous researcher.)
The App Store model has fundamentally changed how software is distributed, mostly for the better (IMHO), but it has also removed some of the control of the release process from the hands of the developers and companies they work for. As I have spelled out previously in my book on iOS enterprise development, the fact that Apple has the final say on if and when software goes into the store has required more conservative release timelines. If you want to release on the first of September, you need to count back at least two weeks for “gold master”, because you need to upload the app, potentially go through a round of rejection from Apple, and then upload a fixed version.
Android apps don’t suffer from this lag, because most of the Android stores don’t do any significant checking of the applications uploaded to them. The Devil’s Deal that Apple developers have made with Apple is that in return for the longer wait time to get apps in the store (and having to follow Apple’s rules), they get a de facto seal of approval from Apple. In other words, it is assumed that apps in the iTunes store are more stringently policed and less likely to crash or do harm (deliberately or else-wise.)
The current downtime has brought that deal into question, however. Suddenly, developers who need new provisioning certificates, passbook certificates, or push notification certificates find themselves with nowhere to go. Even if iTunes Connect is available, it doesn’t do you any good if you can’t get a distribution certificate to sign your app for the store. I’m sure that there are developers at this moment who have had their finely tuned release strategies thrown into disarray by the in-availability of the developer portal.
Being essentially at the mercy of Apple’s whims (or Google’s, for that matter) can’t be a pleasant sensation for a company or individual trying to get a new piece of software out the door. The question that the developer community will have to answer is if the benefits of the App Store model make it worth the hassles, in the long run.
Passion isn't just for romance novels.
Graduates, parents, guests, members of the faculty of
<%= college.collegeName %>. I am honored today to have the opportunity to speak with you, as you move out of the cloistered environment of higher education, and into “the real world.” Except for those of you moving on to postgraduate degrees, of course. You will get to enjoy a life uncluttered by 401Ks and team building exercises for a few more blessed years.
But, for the rest of you, today marks your first step into a journey that will last the rest of your life, unless you’re able to cash in on your equity in some startup, in which case I’m sure you’ll be hearing from the
<%= college.collegeName %> alumni office before the check settles from your brokerage.
In my 35 years of experience in the software field, I’ve met a lot of developers, young and old. And the one thing that separated the truly successful ones from the crowd is passion. Now passion is an overused and abused term these days. Too often people take it to mean a passion for being successful, for achieving a personal goal in their life. When I talk about passion, I mean love. I’ve been in love with computers since I was 14 years old, and I’d be playing with them even if I didn’t get paid for it. If software engineering is merely a means to an end, you’re not going to be happy in the long term working in this field, because much of it is God-awful boring unless you have a passion for it.
Being passionate about software is critical to being successful, because the field is a constantly moving target. What will net you $130K today will be done by junior programmers in five years, and unless you’re constantly adding new tools to your belt, you’re going to find yourself priced out of the market. Many of the best projects I’ve ever worked on came to me because I had already gained the skill-set on my own. Play around with new technologies, contribute to open source projects, and you may find yourself with an opportunity to apply those skills on the job, and get them into your resume. You are rarely going to get an opportunity to have your current employer pay for you to learn things, so learn them on your own and be in a position to leverage the skills when a new project comes along. But if you have a passion for technology, you’ll already be doing it, and enjoying it without needing me to tell you to. That’s why passionate people have a leg up.
People in their 20s tend to jump into small, fledgeling companies, and that’s one of the best things you can do. A junior developer at Fidelity or Akamai is going to work on one thing for long periods of time, while at a start-up you’ll get a chance to jump all over the place, learning many different aspects of the field. But don’t fall into the trap of trading long hours and happiness for the gold ring of equity. You are never going to be in better shape, less constrained by responsibilities, or have more energy than you will right now. Burning 80-hour weeks grinding code is a terrible waste of that gift. Most companies crash and burn, and that equity they gave you will be just toilet paper. It won’t pay for the time you sacrificed eating take-out pizza in front of a glowing tube rather than enjoying the best years of your life.
If you’re passionate, you’ll do the job you’re required to do, and more, but don’t let your employer abuse your enthusiasm. One of my tenets of life has always been that “a lack of planning on your part does not constitute an emergency on my part.” There will be times in your life when you have to step in and fix genuine, unforeseen emergencies, and burn the midnight oil. But if you’re being asked to do it regularly, just because your company didn’t allocate enough resources to see the job through, you’re being played for a patsy.
Consider the benefits of loyalty to your employer. More than just about any other industry, software suffers from a nomadic workforce, where spending five years at a single company is rare. Remember that in a good work environment, you make friends as well as money, and as much as you may say you will, you never keep in touch with them when you move on. But don’t be afraid to jump ship if you are truly unhappy where you are, either.
Software is adding more and more value to machines. Could it completely commoditize them?
I’m a sucker for a good plant tour, and I had a really good one last week when Jim Stogdill and I visited K. Venkatesh Prasad at Ford Motor in Dearborn, Mich. I gave a seminar and we talked at length about Ford’s OpenXC program and its approach to building software platforms.
The highlight of the visit was seeing the scale of Ford’s operation, and particularly the scale of its research and development organization. Prasad’s building is a half-mile into Ford’s vast research and engineering campus. It’s an endless grid of wet labs like you’d see at a university: test tubes and robots all over the place; separate labs for adhesives, textiles, vibration dampening; machines for evaluating what’s in reach for different-sized people.
Prasad explained that much of the R&D that goes into a car is conducted at suppliers–Ford might ask its steel supplier to come up with a lighter, stronger alloy, for instance–but Ford is responsible for integrative research: figuring out how to, say, bond its foam insulation onto that new alloy.
In our more fevered moments, we on the software side of things tend to foresee every problem being reduced to a generic software problem, solvable with brute-force computing and standard machinery. In that interpretation, a theoretical Google car operating system–one that would drive the car and provide Web-based services to passengers–could commoditize the mechanical aspects of the automobile. Read more…
Hardware is back and makers are driving it. Here are some of the signals.
Chris Anderson wrote Makers and went from editor-in-chief of Wired to CEO of 3D Robotics, making his hobby his side job and then making it his main job.
A new executive at Motorola Mobility, a division of Google, said that Google seeks to “googlify” hardware. By that he meant that devices would be inexpensive, if not free, and that the data created or accessed by them would be open. Motorola wants to build a truly hackable cellphone, one that makers might have ideas about what to do with it.
Regular hardware startup meetups, which started in San Francisco and New York, are now held in Boston, Pittsburgh, Austin, Chicago, Dallas and Detroit. I’m sure there are other American cities. Melbourne, Stockholm and Toronto are also organizing hardware meetups. Hardware entrepreneurs want to find each other and learn from each other.
Hardware-oriented incubators and accelerators are launching on both coasts in America, and in China.
The market for personal 3D printers and 3D printing services has really taken off. 3D printer startups continue to launch, and all of them seem to have trouble keeping up with demand. MakerBot is out raising money. Shapeways raised $30 million in a new round of financing announced this week.
Learn to build event-driven client and server applications
I want to build a web server, a mail server, a BitTorrent client, a DNS server, or an IRC bot—clients and servers for a custom protocol in Python. And I want them to be cross-platform, RFC-compliant, testable, and deployable in a standardized fashion. What library should I use?
Twisted is a “batteries included” networking engine for writing, testing, and deploying event-driven clients and servers in Python. It comes with off-the-shelf support for popular networking protocols like HTTP, IMAP, IRC, SMTP, POP3, IMAP, DNS, FTP, and more.
To see just how easy it is to write networking services using Twisted, let’s run and discuss a simple Twisted TCP echo server:
from twisted.internet import protocol, reactor
def dataReceived(self, data):
def buildProtocol(self, addr):
With Twisted installed, if we save this code to echoserver.py and run it with python echoserver.py, clients can now connect to the service on port 8000, send it data, and get back their echoed results. Read more…