"github" entries

Introducing “A Field Guide to the Distributed Development Stack”

Tools to develop massively distributed applications.

Editor’s Note: At the Velocity Conference in Barcelona we launched “A Field Guide to the Distributed Development Stack.” Early response has been encouraging, with reactions ranging from “If I only had this two years ago” to “I want to give a copy of this to everyone on my team.” Below, Andrew Odewahn explains how the Guide came to be and where it goes from here.


As we developed Atlas, O’Reilly’s next-generation publishing tool, it seemed like every day we were finding interesting new tools in the DevOps space, so I started a “Sticky” for the most interesting-looking tools to explore.

field-guide-sticky

At first, this worked fine. I was content to simply keep a list, where my only ordering criteria was “Huh, that looks cool. Someday when I have time, I’ll take a look at that,” in the same way you might buy an exercise DVD and then only occasionally pull it out and think “Huh, someday I’ll get to that.” But, as anyone who has watched DevOps for any length of time can tell you, it’s a space bursting with interesting and exciting new tools, so my list and guilt quickly got out of hand.

Read more…

Four short links: 30 October 2014

Four short links: 30 October 2014

Security and Privacy, ISP Measurement, Github for Education, and Mobile Numbers

  1. A Critique of the Balancing Metaphor in Privacy and SecurityThe arguments presented by this paper are built on two underlying assertions. The first is that the assessment of surveillance measures often entails a judgement of whether any loss in privacy is legitimised by a justifiable increase in security. However, one fundamental difference between privacy and security is that privacy has two attainable end-states (absolute privacy through to the absolute absence of privacy), whereas security has only one attainable end-state (while the absolute absence of security is attainable, absolute security is a desired yet unobtainable goal). The second assertion, which builds upon the first, holds that because absolute security is desirable, new security interventions will continuously be developed, each potentially trading a small measure of privacy for a small rise in security. When assessed individually each intervention may constitute a justifiable trade-off. However, when combined together, these interventions will ultimately reduce privacy to zero. (via Alistair Croll)
  2. ISP Interconnection and its Impact on Consumer Internet Performance (Measurement Lab) — In researching our report, we found clear evidence that interconnection between major U.S. access ISPs (AT&T, Comcast, CenturyLink, Time Warner Cable, and Verizon) and transit ISPs Cogent, Level 3, and potentially XO was correlated directly with degraded consumer performance throughout 2013 and into 2014 (in some cases, ongoing as of publication). Degraded performance was most pronounced during peak use hours, which points to insufficient capacity and congestion as a causal factor. Further, by noting patterns of performance degradation for access/transit ISP pairs that were synchronized across locations, we were able to conclude that in many cases degradation was not the result of major infrastructure failures at any specific point in a network, but rather connected with the business relationships between ISPs.
  3. The Emergence of Github as Collaborative Platform for Education (PDF) — We argue that GitHub can support much of what traditional learning systems do, as well as go beyond them by supporting collaborative activities.
  4. Mobile is Eating the World (A16Z) — mobile becoming truly ubiquitous, bringing opportunities to use the construct “X is eating Y.”

Just fork it

Making forking the norm

Brian Kardell (вкαя∂εℓℓ)tweeted:

Kip Hamptonreplied:

Free and Open Source software licenses make forking legal. Git makes forking easy. GitHub makes it easy to fork sociably. Can we just make this normal?

The most visible recent fork – LibreSSL‘s blunt forking of OpenSSL – was widely reported as conflict. It’s certainly not a polite break, but the OpenSSL’s Apache-style license means it’s legal.

Meanwhile, in a reminder that specifications can fork too, Ian Hickson put his objections to the W3C forking a WHATWG spec on www-archive to make sure his complaints of plagiarism would be part of the permanent record. WHATWG specs are licensed CC0, so once again, it’s legal.

It seems to be a common pattern to want to grant rights, but only want other people to use those rights if they acknowledge our control. (I sometimes have similar tendencies, granted.) We hope that people will contribute to our works while recognizing our power and our ownership over those works. Even the fact that we have to choose licenses at the start of a project gives us a sense of ownership and control, often hiding the (excellent) lack of control that comes once those licenses are applied.

Read more…

Four short links: 14 April 2014

Four short links: 14 April 2014

dategrep, Agile Signoff, Feedback Speed, and Modern Dev

  1. dategrepprint lines matching ranges of dates. Genius!
  2. Business Case Guidance in Agile Projects (gov.uk) — how the UK govt signs off on Agile projects, which normally governments have no clue over how to handle properly.
  3. Hyper Growth Done Right“While I was at Oracle, it took a month before a new engineer would get any code in,” Agarwal says. “It sent this implicit message that it’s okay to take a month to write some code.” First time I’d heard this wise point articulated: slow feedback loops send the message that progress can be slow.
  4. Docker + Github + Jenkins — clever integration of the three tools to get repeatable continuous integration. The modern dev environment has workflow built on git, VMs, and glue.
Four short links: 2 April 2014

Four short links: 2 April 2014

Fault-Tolerant Resilient Yadda Yadda, Tour Tips, Punch Cards, and Public Credit

  1. Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing (PDF) — Berkeley research paper behind Apache Spark. (via Nelson Minar)
  2. Angular Tour — trivially add tour tips (“This is the widget basket, drag and drop for widget goodness!” type of thing) to your Angular app.
  3. Punchcard — generate Github-style punch card charts “with ease”.
  4. Where Credit Belongs for Hack (Bryan O’Sullivan) — public credit for individual contributors in a piece of corporate open source is a sign of confidence in your team, that building their public reputation isn’t going to result in them leaving for one of the many job offers they’ll receive. And, of course, of caring for your individual contributors. Kudos Facebook.

Technical Tests: You’re Doing it Wrong, Part 2

Alternatives and suggestions for candidates and companies to avoid tests

In part one we covered types of technical tests—their relative costs, and why organizations need to understand the costs to the candidates if they want to attract the right type of candidates, and at what point in the process to test.

We haven’t covered alternatives yet. Larger organizations have the benefit of HR or recruitment divisions to bear the brunt of the early cost of the recruitment process—they can call candidates individually to check out interpersonal skills and to make the candidates feel wanted. Smaller organizations don’t necessarily have this, but they do have the benefit of being more flexible. If the brunt of the recruitment process falls on developers (or tech leads, CTOs, or other people in the technical organization) then obviously these organizations are trying to keep the time costs down—every hour invested in recruitment is an hour not spent on coding the company’s money-making product. But these techies are also in a much better position to be able to judge a candidate, and don’t always need to rely on one channel (the technical test, for example) to perform this judgment. They have other alternatives.

Read more…

Restructuring the Web with Git

Can version control manage content?

Web designers? Git? Github? Aren’t those for programmers? At Artifact, Christopher Schmitt showed designers how much their peers are already doing with Github, and what more they can do. Github (and the underlying Git toolset) changes the way that all kinds of people work together.

Sharing with Git

As amazing as Linux may be, I keep thinking that Git may prove to be Linus Torvalds’ most important contribution to computing. Most people think of it, if they think of it at all, as a tool for managing source code. It can do far more, though, providing a drastically different (and I think better) set of tools for managing distributed projects, especially those that use text.

Git tackles an unwieldy problem, managing the loosely structured documents that humans produce. Text files are incredibly flexible, letting us store everything from random notes to code of all kinds to tightly structured data. As awesome as text files are—readable, searchable, relatively easy to process—they tend to become a mess when there’s a big pile of them.

Read more…

Mining the social web, again

If you want to engage with the data that's surrounding you, Mining the Social Web is the best place to start.

When we first published Mining the Social Web, I thought it was one of the most important books I worked on that year. Now that we’re publishing a second edition (which I didn’t work on), I find that I agree with myself. With this new edition, Mining the Social Web is more important than ever.

While we’re seeing more and more cynicism about the value of data, and particularly “big data,” that cynicism isn’t shared by most people who actually work with data. Data has undoubtedly been overhyped and oversold, but the best way to arm yourself against the hype machine is to start working with data yourself, to find out what you can and can’t learn. And there’s no shortage of data around. Everything we do leaves a cloud of data behind it: Twitter, Facebook, Google+ — to say nothing of the thousands of other social sites out there, such as Pinterest, Yelp, Foursquare, you name it. Google is doing a great job of mining your data for value. Why shouldn’t you?

There are few better ways to learn about mining social data than by starting with Twitter; Twitter is really a ready-made laboratory for the new data scientist. And this book is without a doubt the best and most thorough approach to mining Twitter data out there. Read more…

Four short links: 25 September 2013

Four short links: 25 September 2013

Scaling Systems, Code Reviews in Github, Humane Javascript, and Privacy in Identity

  1. Salesforce ArchitectureOur search tier runs on commodity Linux hosts, each of which is augmented with a 640 GiB PCI-E flash drive which serves as a caching layer for search requests. These hosts get their data from a shared SAN array via an NFS file system. Search indexes are stored on the flash drive to enable greater performance for search throughput. Architecture porn.
  2. Gerrit Code Review (Github) — tool for doing code reviews on Github codebases. (via Chris Aniszczyk)
  3. Humanize (Github) — Javascript to turn “first” into a list position, format numbers, generate plurals in English, etc. (via Pete Warden)
  4. Users vs Apps (Tim Bray) — the wrong thing being shared with the wrong people, even once, can ruin a trust relationship forever. Personally, I’m pretty hard-line about this one. I’m currently refusing to update the Android app from my bank, CIBC, because it wants access to my contacts. You know what the right amount of “social” content is in my relationship with my bank? Zero, that’s what.