"distributed systems" entries

Four short links: 20 February 2015

Four short links: 20 February 2015

Robotic Garden, Kids Toys, MSFT ML, and Twitter Scale

  1. The Distributed Robotic Garden (MIT) — We consider plants, pots, and robots to be systems with different levels of mobility, sensing, actuation, and autonomy. (via Robohub)
  2. CogniToys Leverages Watson’s Brain to Befriend, Teach Your Kids (IEEE) — Through the dino, Watson’s algorithms can get to know each child that it interacts with, tailoring those interactions to the child’s age and interests.
  3. How Machine Learning Ate Microsoft (Infoworld) — Azure ML didn’t merely take the machine learning algorithms MSR had already handed over to product teams and stick them into a drag-and-drop visual designer. Microsoft has made the functionality available to developers who know the R statistical programming language and Python, which together are widely used in academic machine learning. Microsoft plans to integrate Azure ML closely with Revolution Analytics, the R startup it recently acquired.
  4. Handling Five Billion Sessions a Day in Real Time (Twitter) — infrastructure porn.
Comment: 1

Variations in event-driven architecture

Find out if mediator or broker topology is right for you.

Editor’s note: this is an advance excerpt from Chapter 2 of the forthcoming Software Architecture Patterns by Mark Richards. This report looks at the patterns that define the basic characteristics and behavior of highly scalable and highly agile applications, and will be made available to download in advance of our Software Architecture Conference happening March 16-19 in Boston.

Coming soon.

Available now!

The event-driven architecture pattern is a popular distributed asynchronous architecture pattern used to produce highly scalable applications. It is also highly adaptable and can be used for both small and large, complex applications. The pattern is made up of highly decoupled, single-purpose event processing components that asynchronously receive and process events. 

The event-driven architecture pattern consists of two main topologies, the mediator and the broker. The mediator topology is commonly used when you need to orchestrate multiple steps within an event through a central mediator, whereas the broker topology is used when you want to chain events together without the use of a central mediator. Because the architecture characteristics and implementation strategies differ between these two topologies, it is important to understand each one to know which is best suited for your particular situation.

Read more…

Comment
Four short links: 9 February 2015

Four short links: 9 February 2015

iBeacon at Scale, Product Teams, Progress Bars, and Distributed Fallacies

  1. The Realities of Installing iBeacon at Scale (Brooklyn Museum) — death by a thousand mundane little real-world pains.
  2. How We Build Software (Intercom) — rare to see descriptions of how to build product teams.
  3. MProgress.js — Material Design progress bars for the web.
  4. Eight Fallacies of Distributed Computing — your network is unreliable, slow, congested, insecure, changing, expensive, and inconsistent. And that’s on a good day.
Comment
Four short links: 4 February, 2015

Four short links: 4 February, 2015

CLR Open Source, Cluster Management, Workplace Bias, and So Much Chrome

  1. CoreCLR Open Sourced (MSDN) — garbage collection, compilation to machine code, and other bits of the CLR.
  2. Kafka Manager — Yahoo open sources a cluster management tool.
  3. Unconscious Bias at Work — useful talk on workplace biases.
  4. All the Chrome (Quirksmode) — Google Chrome is not the default browser on Android 4.3+. There are now at least eight Chromium-based Android default browsers, and they are all subtly, though not wildly, different. The number of Chromium family members has recently risen from nine to eleven with the addition of HTC and LG Chromium, default browsers for modern HTC and LG high-end devices.
Comment
Four short links: 28 January 2015

Four short links: 28 January 2015

Note and Vote, Gaming Behaviour, Code Search, and Immutabilate All The Things

  1. Note and Vote (Google Ventures) — nifty meeting hack to surface ideas and identify popular candidates to a decision maker.
  2. Applying Psychology to Improve Online Behaviour — online game runs massive experiments (w/researchers to validate findings) to improve the behaviour of their players. Some of Riot’s experiments are causing the game to evolve. For example, one product is a restricted chat mode that limits the number of messages abusive players can type per match. It’s a temporary punishment that has led to a noticeable improvement in player behavior afterward —on average, individuals who went through a period of restricted chat saw 20 percent fewer abuse reports filed by other players. The restricted chat approach also proved 4 percent more effective at improving player behavior than the usual punishment method of temporarily banning toxic players. Even the smallest improvements in player behavior can make a huge difference in an online game that attracts 67 million players every month.
  3. Hound — open source code search tool from Etsy.
  4. Immutability Changes Everything (PDF) — This paper is simply an amuse-bouche on the repeated patterns of computing that leverage immutability.
Comment
Four short links: 14 January 2015

Four short links: 14 January 2015

IoT and Govt, Exactly Once, Random Database Subset, and UX Checking

  1. Internet of Things: Blackett Review — the British Government’s review of Internet of Things opportunities around government. Government and others can use expert commissioning to encourage participants in demonstrator programmes to develop standards that facilitate interoperable and secure systems. Government as a large purchaser of IoT systems is going to have a big impact if it buys wisely. (via Matt Webb)
  2. Exactly Once Semantics with Kafka — designing for failure means it’s easier to ensure that things get done than it is to ensure that things get done exactly once.
  3. rdbms-subsetter — open source tool to generate a random sample of rows from a relational database that preserves referential integrity – so long as constraints are defined, all parent rows will exist for child rows. (via 18F)
  4. UXcheck — a browser extension to help you do a quick UX check against Nielsen’s 10 principles.
Comment
Four short links: 24 December 2014

Four short links: 24 December 2014

DRMed Objects, Eventual Consistency, Complex Systems, and Machine Learning Papers

  1. DRMed Cat Litter Box — the future is when you don’t own what you buy, and it’s illegal to make it work better. (via BoingBoing)
  2. Are We Consistent Yet? — the eventuality of consistency on different cloud platforms.
  3. How Complex Systems Fail (YouTube) — Richard Cook’s Velocity 2012 keynote.
  4. Interesting papers from NIPS 2014 — machine learning holiday reading.
Comment

Why the data center needs an operating system

It’s time for applications — not servers — to rule the data center.

1214-missing-operating-system-620

Developers today are building a new class of applications. These applications no longer fit on a single server, but instead run across a fleet of servers in a data center. Examples include analytics frameworks like Apache Hadoop and Apache Spark, message brokers like Apache Kafka, key-value stores like Apache Cassandra, as well as customer-facing applications such as those run by Twitter and Netflix.

These new applications are more than applications, they are distributed systems. Just as it became commonplace for developers to build multithreaded applications for single machines, it’s now becoming commonplace for developers to build distributed systems for data centers.

But it’s difficult for developers to build distributed systems, and it’s difficult for operators to run distributed systems. Why? Because we expose the wrong level of abstraction to both developers and operators: machines. Read more…

Comments: 15
Four short links: 28 November 2014

Four short links: 28 November 2014

Material Design Inspiration, Event Processing, Launch Infrastructure, Remote Work

  1. Material Up — material design inspiration. MD is a physics engine for UI.
  2. Flafka (Cloudera) — Flume plus Kafka, offers sub-second-latency event processing without the need for dedicated infrastructure. (via Abishek Tiwari)
  3. terraform.io — open source package providing a common configuration to launch infrastructure, from physical and virtual servers to email and DNS providers.
  4. Remote Work: An Engineering Leader’s PerspectiveEven proponents of remote work seem to think that you should either have a distributed team from the get go, or stick to a traditional on-site team. Our experience shows that this is incorrect…
Comment
Four short links: 26 November 2014

Four short links: 26 November 2014

Metastable Failures, Static Python Analysis, Material Desktop, and AWS Scale Numbers

  1. Metastable Failure State (Facebook) — very nice story about working together to discover the cause of one of those persistently weird problems.
  2. Bandit — static security analysis of Python code.
  3. Quantum OS — Linux desktop based on Google’s Material Design. UI guidelines fascinate me: users love consistency, designers and brands hate that everything works the same.
  4. Inside AWSEvery day, AWS installs enough server infrastructure to host the entire Amazon e-tailing business from back in 2004, when Amazon the retailer was one-tenth its current size at $7 billion in annual revenue. “What has changed in the last year,” Hamilton asked rhetorically, and then quipped: “We have done it 365 more times.” That is another way of saying that in the past year AWS has added enough capacity to support a $2.55 trillion online retailing operation, should one ever be allowed to exist.
Comment