"programmingblog" entries
3 topologies driving IoT networking standards
The importance of network architecture on the Internet of Things
This article is part of a series exploring the role of networking in the Internet of Things.
There are a lot of moving parts in the networking for the Internet of Things; a lot to sort out between WiFi, WiFi LP, Bluetooth, Bluetooth LE, Zigbee, Z-Wave, EnOcean and others. Some standards are governed by open, independent standards bodies, while others are developed by a single company and are being positioned as defacto standards. Some are well established, others are in the early adoption stage. All were initially developed to meet unique application-specific requirements such as range, power consumption, bandwidth, and scalability. Although these are familiar issues, they take on a new urgency in IoT networks.
To begin establishing the right networking technology for your application, it is important to first understand the network architecture, or the network topology, that is supported by each technology standard. The networking standards being used today in IoT can be categorized into three basic network topologies; point-to-point, star, and mesh. Read more…
Facebook’s Hack, HHVM, and the future of PHP
What is Hack and what does it mean for the future of PHP?
Facebook recently released Hack, a new programming language that looks and acts like PHP. Underneath the hood, however, are a ton of features like static typing, generics, native collections, and many more features for which PHP developers have long been asking. Syntax aside, Hack is not PHP. Hack runs only on Facebook’s HipHop virtual machine (HHVM), a competitor to the traditional PHP Zend Engine.
Why did Facebook build Hack?
Much of Facebook’s internal code is first written with PHP. Facebook can onboard new developers quickly with PHP because the language is notoriously easy to learn and use. Granted, much of Facebook’s PHP code is likely converted to a C derivative before being pushed into production. The point is Facebook depends strongly on the PHP language to attract new talent and increase developer efficiency.
Strict Typing
Unfortunately, PHP may not perform as well as possible at Facebook’s scale. PHP is a loosely typed language and type-related errors may not be recognized until runtime. This means Facebook must write more tests early to enforce type checking, or spend more time refactoring runtime errors after launch. To solve this problem, Facebook added strict typing and runtime-enforcement of return types to Hack. Strict typing nullifies the need for a lot of type-related unit tests and encourages developers to catch type-related errors sooner in the development process.
Instantaneous Type Checking
To make the development process and error-catching process even easier, Facebook includes a type-checking server with its HHVM engine. This server runs locally and monitors Hack code as it is written. Developers’ code editors and IDEs can use this type-checking server to immediately report syntax or type-related errors during code development.
Read more…
Transforming the web (through transformation)
Decorating content may no longer be enough
Thousands of people invented it independently. Millions use it without thinking about a broader context. It’s time to name it so we can talk about it.
Transformation is changing the way we look at the balance between clients and servers, our approach to formatting and layout, and our expectations of what’s possible on the Web. As applications shift from transformation on the server toward transformation on arrival on the client, transformation’s central role becomes more visible.
“Templating” doesn’t quite capture what’s happening here. While templates are often a key tool, describing that tool doesn’t explain the shift from server to client. Templating also misses the many cases where developers are using plain JavaScript to insert, delete, and modify the document tree in response to incoming data.
These practices have been emerging for a long time, in many different guises:
- In the Dynamic HTML days, scripts might tinker with the DOM tree as well as modify CSS presentation.
- Transformation was supposed to be a regular and constant thing in the early XSLT plans. Stylesheets on the client would generate presentation from clean blocks of XML content.
- Ajax opened the door to shell pages, apps that set up a UI, but get most of their content elsewhere, using JavaScript to put it in place.
- New data format options evolved at about the same time that Ajax emerged. JSON offered a more concise set of programmer-friendly content tools. Many apps include a ‘bind JSON to HTML before showing it to the user’ step.
- Template systems now run on the client as well as the server. In many systems, templates on the server feed data to the client, which applies other templates to that data before presenting it to users.
- The HTTP powering Ajax still created a long slow cycle of interaction. WebSockets and WebRTC now offer additional approaches for collecting content with less overhead, making it easier to create many more small transformations.
- Some developers and designers have long thought of the document tree as a malleable collection of layout boxes rather than a deliberately coherent base layer. Separation of concerns? A dead horse, apparently. Recent debates over CSS Regions highlighted these issues again.
Formulating Elixir
Simon St. Laurent and Jose Valim explore a new functional programming language
I was delighted to sit down with Jose Valim, the creator of Elixir, earlier this month at Erlang Factory. He and Dave Thomas had just given a brave keynote exploring the barriers that keep people from taking advantage of Erlang’s many superpowers, challenging the audience with reminders that a programming environment must have reach as well as power to change the world.
Elixir itself is a bold effort to bring Erlang’s strengths to a broader group of developers, adding new strengths, notably metaprogramming, along the way.
Watching Elixir grow has been a unique experience for me. I’ve seen other languages (JavaScript and XSLT) emerge, but Elixir combines solid foundations on prior (Erlang) work with a remarkably open conversation about how to structure the language. Jose tries things and asks for feedback, takes suggestions well, and values questions about how best to make the language accessible. Even without a standards organization, the process has remained open, stable, and productive.
Whether you’re interested in Elixir itself or just in the challenges of creating a new combination in a world filled with past experiments, it’s well worth listening to Jose Valim.
- We’ve had functional programming since 1959 – why the burst of interest now? [2:10]
- Moving from Ruby to Erlang “making Rails thread-safe, that was my personal pain-point” [3:13]
- “Every time I got to study more about the VM, the tooling and everything it provides, my mind gets blown.” [6:12]
- Why Elixir started, and how it’s changed as Jose learned more. [10:08]
- Integrating new Erlang features (R17 maps) into Elixir. [15:43]
- When can you use Elixir in production? [18:07]
I’m looking forward to seeing a lot more Elixir, even as I need to catch up on updating Introducing Elixir. I’m not sure it will conquer the world immediately, but it will certainly leave its mark.
The Case for Test-Driven Development
An interview with O'Reilly author Harry Percival
Harry Percival, author of Test-Driven Web Development with Python, discusses how he got into TDD, why you should too, and shares some tips. In the podcast above, listen to Harry talk candidly about the types of tests that make sense, what and what not to test, and at what point a program becomes complex enough to warrant testing. Below is a mostly matching text version of the same interview. Let us know your thoughts on TDD in the comments—your own war stories and what convinced you (or didn’t!).
Why write tests? How do you know it’s not a waste of time?
The theory is that it’s an investment—the time you spend writing tests will get paid back in time you don’t have to spend debugging. Also, the theory goes that tests should help you to write code that’s easier to work with, as well as code with less defects. Because having tests encourages you to refactor, and to think about design, your code should end up cleaner and better architected, and so it should be easier to work with, and your investment pays off because you’re more productive in future as well.
So that’s the theory. But the problem is that there’s delayed gratification—it’s hard to really believe this when the reward is so far off and the time required is now. So in practice, what was it that convinced me?
I first learned about testing from a book called “Dive Into Python”—it’s a popular book, maybe some of the people listening will have read it too? They may remember that Mark Pilgrim introduces testing, in fact he introduces TDD, in chapter 10. He uses the classic TDD example, which is a Roman Numeral calculator, and he shows how, by writing the tests before we even start writing the code, we can really get some help in how we implement our calculator. So he writes his tests, I should be 1 and II should be 2 and IV should be 4, and so on, and he shows how it helps us to build a really neat implementation of a Roman numeral calculator.
Just Enough Arel
Replacing hand-coded SQL with object oriented programming
If you were a web developer prior to ActiveRecord, you probably remember rolling your own SQL and being specific about which fields you retrieved, writing multiple queries to handle “upserts” (update or insert), and getting frustrated with how difficult it was to generate SQL dynamically.
Then Ruby on Rails came along with its ActiveRecord API, and all of that pain seemed to melt away into distant memory. Now we no longer have to worry about which fields to retrieve, “upserts” can be handled with a single method, and scopes allow us to be as dynamic as we want.
Unfortunately, ActiveRecord introduced new pains: the “WHERE” clause (i.e. the predicate) only allows connecting conditions with “AND”s, and the only comparisons allowed are “=” and “in”. To get around this, we often concede ActiveRecord’s shortcomings and revert back to raw SQL. But in Rails 3.0, we were introduced to another way.
Arel
The Arel relational algebra library is an abstract syntax tree (AST) manager and has the goals of “1) Simplif[ying] the generation of complex SQL queries; and 2) Adapt to various RDBMSes”. It’s the “engine” ActiveRecord uses to change method calls into actual SQL. To be clear, Arel only generates SQL, it doesn’t access the database and retrieve data.
With Arel, we are now capable of fully utilizing the power of SQL, without the mess of hand-coding it. ActiveRecord uses Arel to transform queries like this:
Post.joins(:comments).
where(:published => true).
where(:comments => {:flagged => true})Into SQL like this:
SELECT "posts".*
FROM "posts" INNER JOIN "comments" ON "posts"."id" = "comments"."post_id"
WHERE "posts"."published" = true
AND "comments"."flagged" = trueYes, CSS is code
Learning from "The Humble Border-Radius"
One of the best things I overheard at the Fluent Conference was (more or less):
“CSS live coding? I was like, that isn’t code. But then it was.”
Lea Verou had changed the mind of a skeptic.
CSS is not Turing complete (Update: Lea points out that I’m wrong about that), and I’ve long been grateful that it wasn’t designed as an imperative programming language. It can be too much code for designers, too little code for programmers, and yet it’s code.
I think about the graphics work I tried to do with shapes and sprites and assorted kinds of vectors over the years, and then watch this keynote. Even with the glitches and coming improvements she notes, it’s clear that whatever tools I was using at the time, they weren’t declarative enough. (I’d also just seen this JSFest talk on CSS box-shadow by Vince Allen, so I was looking forward to more CSS graphics magic.)
Six Degrees of Kevin Bacon in six languages
20 years of efficiently computing Bacon numbers
The Oracle at Delphi spoke just one language, a cryptic one that priests “compiled” into ancient Greek. The Oracle of Bacon—the website that plays the Six Degrees of Kevin Bacon game for you—has, in its 20-year existence, been written in six languages. Read on for the history and the reasons why.
1995-1999: C
The original version of the Oracle of Bacon, written by Brett Tjaden in 1995, was all C. The current version, my stewardship of it, and my revision control history only go back to 1999, so that’s where I’ll start. In 1999, I rewrote the Oracle… still entirely in C. Expensive shortest-path and spell-check algorithms? Definitely in C. String processing to build the database? Also C! Presentation layer to parse CGI parameters and generate HTML? C here, too!
Why C? The rationale for the algorithmic component was straightforward: the Oracle of Bacon ran on a slow, shared Unix machine that other people were using to get actual work done. Minimizing CPU and memory resource requirements was the polite thing to do. I needed a compiled language that let me optimize time and space extensively. The loops all counted down, not up, because comparing against zero was fractionally faster on SPARC. It had to be C.
But why were the offline string processing and the CGIs in C? Mostly, I think, to reuse code from the other parts of the code base and from previous projects I’d written when C was the only language I knew.
2004-2005: Perl
As the site added features, I got tired of writing code to generate HTML in C. I wrote new CGIs, then rewrote existing CGIs, in Perl. Simply put, writing the CGIs in an interpreted language made me more productive. I had hash tables and vectors built into the language and CGI support a simple “use” statement away. I didn’t have to compile on one server and then deploy to another—I could edit the CGIs right there on the web server. Good deployment practices it wasn’t, but it made me more productive as a programmer, and the performance of the CGIs didn’t matter all that much.
Upward Mobility: Avoid This Simple App Store Bounce
Large files in your app documents folder can earn you a rejection
There’s nothing worse than submitting your first app to the iTunes app store, and having it get rejected. Well, OK, there are plenty of things that are worse, but it still isn’t pleasant. Bounces can happen for a variety of reasons, from Apple not liking your app’s functionality to poorly written product descriptions and everything in between. But one recently new reason for rejections may catch you off guard, I know I was.
Now that iCloud can be used to back up app contents, Apple has become sensitive to apps that produce large amounts of documents in the app document folder. After all, if you dump 50MB of files into that folder, iCloud is going to try to back it all up, and it may suck up the user’s data plan usage. So, if you’re downloading and storing content as part of your app, you have to make sure that you mark it as not requiring backup to iCloud.
Here’s a nice little function that will do just that, courtesy of Apple:
- (BOOL)addSkipBackupAttributeToItemAtURL:(NSURL *)URL
{
assert([[NSFileManager defaultManager] fileExistsAtPath: [URL path]]);
NSError *error = nil;
BOOL success = [URL setResourceValue: [NSNumber numberWithBool: YES]
forKey: NSURLIsExcludedFromBackupKey error: &error];
if(!success){
NSLog(@"Error excluding %@ from backup %@", [URL lastPathComponent], error);
}
return success;
}
Back to Top
NSURLIsExcludedFromBackupKey flags the file as not requiring backup to iCloud. One important set of files that you should make sure you flag this way is any Core Data SQLite files that store significant amounts of data. For example, if you are storing images as blobs in your database, the file can get pretty big pretty fast.
Tailoring for performance
One source does not fit all
Like a lot of web teams, O’Reilly’s web group has increased its focus on using global components to better scale maintenance and optimize workflow. From a load-time measurement perspective, our performance ratings stay near benchmarks. However, after a recent analysis, using metrics other than load time, we found that our global efforts may have sacrificed performance on a handful of highly visible and heavily visited web pages.
Identifying the popular pages, we sought to improve the use of global components with server side logic, regex, and asynchronous loading. After re-measuring these popular pages, we arrived at faster load times with improved perception of speed. Read more…