"programmingblog" entries
DevOps hiring
A holistic approach to hiring for in-demand positions
Traditional recruiting is broken. The biggest problem is that goals and incentives of candidates, hiring managers, and recruiters are often misaligned. For instance, taken to the extreme, a headhunter’s best candidate is one who survives at a new job just long enough to collect the finder’s fee before being placed at another company. Clearly, this puts the recruiter at odds with both the candidate and the hiring manager.
The current unemployment rate for “computer and mathematical occupations” is 2.9%. It’s been dropping, and is now lower than the 4% level that is considered “full employment.” This means that every software engineer, database administrator, and “DevOps engineer” who wants to work—and, apparently, some who don’t—are all gainfully employed. Not surprisingly, salaries are also going up.
Given this information, it’s easy to get caught up in the hysteria of the so-called “war for talent,” with its violent language of “headhunting” and “poaching.” A natural reaction to this situation might be to have more recruiters send even more unsolicited emails to candidates through LinkedIn. However, this default recruiting tactic has diminishing returns: the higher the volume of recruiter spam on LinkedIn, the more engineers will ignore it or delete their profiles altogether.
Is there another way?
Ohai, new Ohai plugins!
Extending Chef
When you start to use the Chef configuration management system, you will quickly encounter a tool it ships with called Ohai which collects information about the underlying system to expose to the Chef Client as attributes during its run. These attributes allows you to easily incorporate system-specific behavior into your Chef cookbooks, for example allowing you to create a single “package” resource in your cookbook which automatically detects whether to install packages from Yum when running on a Redhat based Linux distribution or from Apt when running on a Debian based distribution. Read more…
What’s New in Java 8: Lambdas
A hands-on introduction to Java 8's most exciting new feature
Java 8 is here, and, with it, come lambdas. Although long overdue, lambdas are a remarkable new feature that could make us rethink our programming styles and strategies. In particular, they offer exciting new possibilities for functional programming.
While lambdas are the most prominent addition to Java 8, there are many other new features, such as functional interfaces, virtual methods, class and method references, new time and date API, JavaScript support, and so on. In this post, I will focus mostly on lambdas and their associated features, as understanding this feature is a must for any Java programmer on-boarding to Java 8.
All of the code examples mentioned in this post can be found in this github repo.
What are lambdas?
Lambdas are succinctly expressed single method classes that represent behavior. They can either be assigned to a variable or passed around to other methods just like we pass data as arguments.
You’d think we’d need a new function type to represent this sort of expression. Instead, Java designers cleverly used existing interfaces with one single abstract method as the lambda’s type.
Before we go into detail, let’s look at a few examples.
Example Lambda Expressions
Here are a few examples of lambda expressions:
// Concatenating strings
(String s1, String s2) -> s1+s2;
// Squaring up two integers
(i1, i2) -> i1*i2;
// Summing up the trades quantity
(Trade t1, Trade t2) -> {
t1.setQuantity(t1.getQuantity() + t2.getQuantity());
return t1;
};
// Expecting no arguments and invoking another method
() -> doSomething();Have a look at them once again until you familiarize yourself with the syntax. It may seem a bit strange at first. We will discuss the syntax in the next section.
You might wonder what the type is for these expressions. The type of any lambda is a functional interface, which we discuss below.
Read more…
Simplifying Django
Lightweight Django by example
The following comes to you from Julia Elman and Mark Lavin. Julia is a a hybrid designer/developer who has been working her brand of web skills since 2002; and Mark is the Development Director at Caktus Consulting Group in Carrboro, NC where he builds scalable web applications with Django. Together, they are working on Lightweight Django, a book due out later this year that explores bringing Django into modern web practices.
Despite Django’s popularity and maturity, some developers believe that it is an outdated web framework made primarily for “content-heavy” applications. Since the majority of modern web applications and services tend not to be rich in their content, this reputation leaves Django seeming like a less than optimal choice as a web framework.
Let’s take a moment to look at Django from the ground up and get a better idea of where the framework stands in today’s web development practices.
Plain and Simple Django
A web framework’s primary purpose is to help to generate the core architecture for an application and reuse it on other projects. Django was built on this foundation to rapidly create web applications. At its core, Django is primarily a Web Server Gateway Interface (WSGI) application framework that provides HTTP request utilities for extracting and returning meaningful HTTP responses. It handles various services with these utilities by generating things like URL routing, cookie handling, parsing form data and file uploads.
Also, when it comes to building those responses Django provides a dynamic template engine. Right out of the box, you are provided with a long list of filters and tags to create dynamic and extensible templates for a rich web application building experience.
By only using these specific pieces, you easily see how you can build a plain and simple micro-framework application inside a Django project.
We do know that there are some readers who may enjoy creating or adding their own utilities and libraries. We are not trying to take away from this experience, but show that using something like Django allows for fewer distractions. For example, instead of having to decide between Jinja2, Mako, Genshi, Cheetah, etc, you can simply use the existing template language while you focus on building out other parts. Fewer decisions up front make for a more enjoyable application building process.
Health IT is a growth area for programmers
New report covers areas of innovation and their difficulties
O’Reilly recently released a report I wrote called The Information Technology Fix for Health: Barriers and Pathways to the Use of Information Technology for Better Health Care. Along with our book Hacking Healthcare, I hope this report helps programmers who are curious about Health IT see what they need to learn and what they in turn can contribute to the field.
Computers in health are a potentially lucrative domain, to be sure, given a health care system through which $2.8 trillion, or $8.915 per person, passes through each year in the US alone. Interest by venture capitalists ebbs and flows, but the impetus to creative technological hacking is strong, as shown by the large number of challenges run by governments, pharmaceutical companies, insurers, and others.
Some things you should consider doing include:
- Join open source projects
- Numerous projects to collect and process health data are being conducted as free software; find one that raises your heartbeat and contribute. For instance, the most respected health care system in the country, VistA from the Department of Veterans Affairs, has new leadership in OSEHRA, which is trying to create a community of vendors and volunteers. You don’t need to understand the oddities of the MUMPS language on which VistA is based to contribute, although I believe some knowledge of the underlying database would be useful. But there are plenty of other projects too, such as the OpenMRS electronic record system and the projects that cooperate under the aegis of Open Health Tools.
Can we extend the web cleanly?
The DOM and human nature create some challenges
“Design by Committee” is rarely a compliment. Can the Web shift away from that model, retaining some order without falling into troublesome chaos?
The Manifesto
Part of the excitement around the Extensible Web Manifesto was that it wanted to move the Web to more of an evolutionary model:
We aim to tighten the feedback loop between the editors of web standards and web developers.
Today, most new features require months or years of standardization, followed by careful implementation by browser vendors, only then followed by developer feedback and iteration. We prefer to enable feature development and iteration in JavaScript, followed by implementation in browsers and standardization.
Ruby: The Unit Test-Friendly Language
Accelerate your tests with test doubles
Let’s say you have a Ruby class that retrieves the contents of web pages, and you need to write a unit test for it…
class Spider
attr_accessor :address, :path
def get_response
response = Net::HTTP.get_response(@address, @path)
end
def get_body
get_response.body
end
end
You’ve tested the get_response method, and now you need to test get_body. You’re using MiniTest, a unit testing framework that comes standard with Ruby.
class TestSpider < MiniTest::Unit::TestCase
def test_get_body
spider = Spider.new
spider.address = 'programming.oreilly.com'
spider.path = '/2014/02/why-ruby-blocks-exist.html'
assert spider.get_body == '<h1>Hi!'
end
end
You create a Spider instance, assign it a page to retrieve, and at the end of the test, assert that the HTTP response matches your expected value. Simple enough.
Two problems
But there’s two problems with this test:
- Your
Spiderinstance is making a real network request for the page, slowing the test and incurring network overhead for both you and the host you visit. - The page could easily change on the remote side. You can use the current page contents in the test assertion, but it might need to be changed in the future.
In fact, the latter issue is causing your test to fail right now, because the real page doesn’t match the simplified HTML in your test.
Finished tests in 3.767214s, 0.2654 tests/s, 0.2654 assertions/s. 1) Failure: test_get_body(TestSpider) [-:26]: Failed assertion, no message given.
Building an Activity Feed System with Storm
One of many wonderfully functional recipes from the Clojure Cookbook
Editor’s Note: The Clojure Cookbook is a recently published book by experienced Clojurists Luke VanderHart and Ryan Neufeld. It seeks to be a practical collection of tasks for intermediate Clojure programmers. In addition to providing their own recipes, Ryan and Luke accepted contributions from a number of people in the community. One of those contributors was Travis Vachon–in this excerpt from the Cookbook, Travis gives you a tried and true recipe for working with Clojure and Storm.
Problem
You want to build an activity stream processing system to filter and aggregate the raw event data generated by the users of your application.
Solution
Streams are a dominant metaphor for presenting information to users of the modern Internet. Used on sites like Facebook and Twitter and mobile apps like Instagram and Tinder, streams are an elegant tool for giving users a window into the deluge of information generated by the applications they use every day.
As a developer of these applications, you want tools to process the firehose of raw event data generated by user actions. They must offer powerful capabilities for filtering and aggregating data and must be arbitrarily scalable to serve ever-growing user bases. Ideally they should provide high-level abstractions that help you organize and grow the complexity of your stream-processing logic to accommodate new features and a complex world.
Clojure offers just such a tool in Storm, a distributed real-time computation system that aims to be for real-time computation what Hadoop is for batch computation. In this section, you’ll build a simple activity stream processing system that can be easily extended to solve real-world problems.
First, create a new Storm project using its Leiningen template:
$ lein new cookbook-storm-project feeds
In the project directory, run the default Storm topology (which the lein template has generated for you):
$ cd feeds
$ lein run -m feeds.topology/run!
Compiling feeds.TopologySubmitter
...
Emitting: spout default [:bizarro]
Processing received message source: spout:4, stream: default, id: {}, [:bizarro]
Emitting: stormy-bolt default ["I'm bizarro Stormy!"]
Processing received message source: stormy-bolt:5,
stream: default, id: {}, [I'm bizarro Stormy!]
Emitting: feeds-bolt default ["feeds produced: I'm bizarro Stormy!"]
This generated example topology just babbles example messages incoherently, which probably isn’t what you want, so begin by modifying the “spout” to produce realistic events.
The power of HTML
Acknowledging the source of the Web's strength
For a growing number of developers, “web” means “JavaScript”. Programmers like to focus on programming languages, but the Web’s basic power comes from its support for communications, not programming.
I asked Jen Simmons, host of the Web Ahead podcast, to keynote Fluent after seeing her give a talk on responsive layouts at the ARTIFACT conference last year. It wasn’t just a great talk on responsive layout, but an exploration of the Web foundations that make responsive layout possible. Responsive layout’s foundations are deep in HTML, which contained those multi-device values from the beginning.
At Fluent, Simmons delivered A Love Letter to HTML, exploring HTML’s origins. The goals driving a memo written 25 years ago gave the Web strengths that developers need to study today.
Tailoring CSS for performance
Rethinking CSS delivery
In my last article, I demonstrated how improved performance and a lower PageSpeed Insights score were accomplished by removing unnecessary external JavaScript and CSS requests. YepNope was also used to manage the asynchronous loading of external requests.
After the improvements, I thought it was time to move on but PageSpeed Insights advised there was more work to do.