- More Tools for Managing and Reproducing Complex Data Projects (Ben Lorica) — As I survey the landscape, the types of tools remain the same, but interfaces continue to improve, and domain specific languages (DSLs) are starting to appear in the context of data projects. One interesting trend is that popular user interface models are being adapted to different sets of data professionals (e.g. workflow tools for business users).
- Graphical Linear Algebra — or “Graphical The-Subject-That-Kicked-Nat’s-Butt” as I read it.
- Consistent Hashing: A Guide and Go Implementation — easy-to-follow article (and source).
- NoTCP Manifesto — a nice summary of the reasons to build custom protocols over UDP, masquerading as church-nailed heresy. Today’s heresy is just the larval stage of tomorrow’s constricting orthodoxy.
An introduction to multi-level caching.
Since a content delivery network (CDN) is essentially a cache, you might be tempted not to make use of the cache in the browser, to avoid complexity. However, each cache has its own advantages that the other does not provide. In this post I will explain what the advantages of each are, and how to combine the two for the most optimal performance of your website.
Why use both?
While CDNs do a good job of delivering assets very quickly, they can’t do much about users who are out in the boonies and barely have a single bar of reception on their phone. As a matter of fact, in the US, the 95th percentile for the round trip time (RTT) to all CDNs is well in excess of 200 milliseconds, according to Cedexis reports. That means at least 5% of your users, if not more, are likely to have a slow experience with your website or application. For reference, the 50th percentile, or median, RTT is around 45 milliseconds.
So why bother using a CDN at all? Why not just rely on the browser cache?
Outsourcing your DNS is not a magic bullet.
There is frequently a tendency toward letting one’s guard down when it comes to threats to your IT systems. Absent an immediate “hair-on-fire” situation, we may relax and assume all is well. Yet malicious activity such as hacking, phishing, malware, and DDoS attacks never stop accelerating in terms of frequency and intensity.
So it’s important to have a “Plan B” DNS solution in place and ready before a crisis hits. That way, even if you’re taken off guard, you still have a backup plan and can respond appropriately.
DNS is one of those things nobody really thinks about, until it stops working. The first time easyDNS went off the air on April 15, 2003, it induced a type of existential crisis in me. That summer, after meditating intensely on the situation, I came away with the conclusion that the centralized managed DNS model, as we understood it then, was doomed.
My response at the time was a proposal to pivot to a DNS appliance with decentralized deployments, but centralized monitoring and management. That concept was promptly shot down my co-founders and we’ve kept on with the centralized, hosted DNS model to this day.
The core problem is this: there are many reasons to elect to outsource your DNS to a managed DNS provider. Those reasons include:
How much do you need to know?
I expected that CSSDevConf would be primarily a show about front-end work, focused on work in clients and specifically in browsers. I kept running into conversations, though, about the challenges of moving between the front and back end, the client and the server side. Some were from developers suddenly told that they had to become “full-stack developers” covering the whole spectrum, while others were from front-end engineers suddenly finding a flood of back-end developers tinkering with the client side of their applications. “Full-stack” isn’t always a cheerful story.
In the early days of the Web, “full-stack” was normal. While there were certainly people who focused on running web servers or designing sites as beautiful as the technology would allow, there were lots of webmasters who knew how to design a site, write HTML, manage a server, and maybe write some CGI code for early applications.
Even as the bust set in, specialization remained the trend because Web projects — especially on the server side — had grown far more complicated. They weren’t just a server and a few scripts, but a complete stack, including templates, logic, and usually a database. Whether you preferred the LAMP stack, a Microsoft ASP stack, or perhaps Java servlets and JSP, the server side rapidly became its own complex arena. Intranet development in particular exploded as a way to build server-based applications that could (cheaply) connect data sources to users on multiple platforms. Writing web apps was faster and cheaper than writing desktop apps, with more tolerance for platform variation.
Range, power consumption, scalability, and bandwidth dominate technology decisions.
Three types of networking topologies are utilized in the Internet-of-Things: point-to-point, star, and mesh networking. To provide a way to explore the attributes and capabilities of each of these topologies, we defined a hypothetical (but realistic) application in the building monitoring and energy management space and methodically defined its networking requirements.
Let’s pull it all together to make a network selection for our building monitoring application. As described previously, the application will monitor, analyze, and optimize energy usage throughout the user’s properties. To accomplish this, monitoring and control points need to be deployed throughout each building, including occupancy and temperature sensors. Sensor data will be aggregated back to a central building automation panel located in each building. A continuous collection of data will provide a higher resolution of temperature and occupancy information, thus rendering better insight into HVAC performance and building utilization patterns. Comparison of energy utilization throughout the portfolio of properties allows lower performing buildings to be flagged.
A suitable network topology for building automation.
Editor’s note: this article is part of a series exploring the role of networking in the Internet of Things.
Today we are going to consider the attributes of wireless mesh networking, particularly in the context of our building monitoring and energy application.
A host of new mesh networking technologies came upon the scene in the mid-2000s through start-up ventures such as Millennial Net, Ember, Dust Networks, and others. The mesh network topology is ideally suited to provide broad area coverage for low-power, low-data rate applications found in application areas like industrial automation, home and commercial building automation, medical monitoring, and agriculture.
When to use a star network.
This article is part of a series exploring the role of networking in the Internet of Things.
In my previous post we evaluated a point-to-point networking technology, specifically Bluetooth, to determine its applicability to our building monitoring and energy application. In this post, we will evaluate the use of a star networking technology to meet our application needs.
A star network consists of one central hub that establishes a point-to-point network connection with all other nodes in the network (e.g. sensor nodes). This central hub acts as a common connection point for all nodes in the network. All peripheral nodes may therefore communicate with all others by transmitting to, and receiving from, the central hub only.
Today, Wi-Fi is by far the most commonly used wireless star topology. It is deployed widely throughout many environments, providing near ubiquitous internet access in facilities such as schools, campuses, office buildings, lodging, residential homes and so on. The term Wi-Fi is not a standard, but a term trademarked by The Wi-Fi Alliance and covering a number of IEEE 802.11 standards along with details of implementation.
As in past posts, let’s take a closer look at the technology and evaluate WI-Fi’s capabilities against the nine key application attributes that characterized our building monitoring and energy management application.
Toward unifying customer behavior and operations metrics.
For the last ten years I’ve had a foot in both the development and operations worlds. I stumbled into the world of IT operations as a result of having the most UNIX skills in the team shortly after starting at ThoughtWorks. I was fortunate enough to do so at a time when many of my ThoughtWorks colleagues and I where working on the ideas which were captured so well in Jez Humble and Dave Farley’s Continuous Delivery (Addison-Wesley).
During this time, our focus was on getting our application into production as quickly as possible. We were butting up against the limits of infrastructure automation and IaaS providers like Amazon were only in their earliest form.
Recently, I have spent time with operations teams who are most concerned with the longer-term challenges of looking after increasingly complex ecosystems of systems. Here the focus is on immediate feedback and knowing if they need to take action. At a certain scale, complex IT ecosystems can seem to exhibit emergent behavior, like an organism. The operations world has evolved a series of tools which allow these teams to see what’s happening *right now* so we can react, keep things running, and keep people happy.
At the same time, those of us who spend time thinking about how to quickly and effectively release our applications have become preoccupied with wanting to know if that software does what our customers want once it gets released. The Lean Startup movement has shown us the importance of putting our software in front of our customers, then working out how they actually use it so we can determine what to do next. In this world, I was struck by the shortcomings of the tools in this space. Commonly used web analytics tools, for example, might only help me understand tomorrow how my customers used my site today.