"deep learning" entries

Four short links: 18 March 2015

Four short links: 18 March 2015

Moonshots, Decacorns, Leadership, and Deep Learning

  1. How to Make Moonshots (Astro Teller) — Expecting a person to be a reliable backup for the [self-driving car] system was a fallacy. Once people trust the system, they trust it. Our success was itself a failure. We came quickly to the conclusion that we needed to make it clear to ourselves that the human was not a reliable backup — the car had to always be able to handle the situation. And the best way to make that clear was to design a car with no steering wheel — a car that could drive itself all of the time, from point A to point B, at the push of a button.
  2. Billion-Dollar Math (Bloomberg) — There’s a new buzzword, “decacorn,” for those over $10 billion, which includes Airbnb, Dropbox, Pinterest, Snapchat, and Uber. It’s a made-up word based on a creature that doesn’t exist. “If you wake up in a room full of unicorns, you are dreaming,” Todd Dagres, a founding partner at Spark Capital, recently told Bloomberg News. Not just cute seeing our industry explained to the unwashed, but it’s the first time I’d seen decacorn. (The weather’s just dandy in my cave, thanks for asking).
  3. What Impactful Engineering Leadership Looks Like — aside from the ugliness of “impactful,” notable for good advice. “When engineering management is done right, you’re focusing on three big things,” she says. “You’re directly supporting the people on your team; you’re managing execution and coordination across teams; and you’re stepping back to observe and evolve the broader organization and its processes as it grows.”
  4. cxxnet“a fast, concise, distributed deep learning framework” that scales beyond a single GPU.
Comment
Four short links: 12 March 2015

Four short links: 12 March 2015

Billion Node Graphs, Asynchronous Systems, Deep Learning Hardware, and Vision Resources

  1. Mining Billion Node Graphs: Patterns and Scalable Algorithms (PDF) — slides from a CMU academic’s talk at C-BIG 2012.
  2. There Is No NowOne of the most important results in the theory of distributed systems is an impossibility result, showing one of the limits of the ability to build systems that work in a world where things can fail. This is generally referred to as the FLP result, named for its authors, Fischer, Lynch, and Paterson. Their work, which won the 2001 Dijkstra Prize for the most influential paper in distributed computing, showed conclusively that some computational problems that are achievable in a “synchronous” model in which hosts have identical or shared clocks are impossible under a weaker, asynchronous system model.
  3. Deep Learning Hardware GuideOne of the worst things you can do when building a deep learning system is to waste money on hardware that is unnecessary. Here I will guide you step by step through the hardware you will need for a cheap high performance system.
  4. Awesome Computer Vision — curated list of computer vision resources.
Comment
Four short links: 24 February 2015

Four short links: 24 February 2015

Open Data, Packet Dumping, GPU Deep Learning, and Genetic Approval

  1. Wiki New Zealand — open data site, and check out the chart builder behind the scenes for importing the data. It’s magic.
  2. stenographer (Google) — open source packet dumper for capturing data during intrusions.
  3. Which GPU for Deep Learning?a lot of numbers. Overall, I think memory size is overrated. You can nicely gain some speedups if you have very large memory, but these speedups are rather small. I would say that GPU clusters are nice to have, but that they cause more overhead than the accelerate progress; a single 12GB GPU will last you for 3-6 years; a 6GB GPU is plenty for now; a 4GB GPU is good but might be limiting on some problems; and a 3GB GPU will be fine for most research that looks into new architectures.
  4. 23andMe Wins FDA Approval for First Genetic Test — as they re-enter the market after FDA power play around approval (yes, I know: one company’s power play is another company’s flouting of safeguards designed to protect a vulnerable public).
Comment
Four short links: 19 January 2015

Four short links: 19 January 2015

Going Offline, AI Ethics, Human Risks, and Deep Learning

  1. Reset (Rowan Simpson) — It was a bit chilling to go back over a whole years worth of tweets and discover how many of them were just junk. Visiting the water cooler is fine, but somebody who spends all day there has no right to talk of being full.
  2. Google’s AI Brain — on the subject of Google’s AI ethics committee … Q: Will you eventually release the names? A: Potentially. That’s something also to be discussed. Q: Transparency is important in this too. A: Sure, sure. Such reassuring.
  3. AVA is now Open Source (Laura Bell) — Assessment, Visualization and Analysis of human organisational information security risk. AVA maps the realities of your organisation, its structures and behaviors. This map of people and interconnected entities can then be tested using a unique suite of customisable, on-demand, and scheduled information security awareness tests.
  4. Deep Learning for Torch (Facebook) — Facebook AI Research open sources faster deep learning modules for Torch, a scientific computing framework with wide support for machine learning algorithms.
Comment

Cheap sensors, fast networks, and distributed computing

The history of computing has been a constant pendulum — that pendulum is now swinging back toward distribution.

Editor’s note: this is an excerpt from our new report Data: Emerging Trends and Technologies, by Alistair Croll. You can download the free report here.

The trifecta of cheap sensors, fast networks, and distributing computing are changing how we work with data. But making sense of all that data takes help, which is arriving in the form of machine learning. Here’s one view of how that might play out.

Clouds, edges, fog, and the pendulum of distributed computing

The history of computing has been a constant pendulum, swinging between centralization and distribution.

The first computers filled rooms, and operators were physically within them, switching toggles and turning wheels. Then came mainframes, which were centralized, with dumb terminals.

As the cost of computing dropped and the applications became more democratized, user interfaces mattered more. The smarter clients at the edge became the first personal computers; many broke free of the network entirely. The client got the glory; the server merely handled queries.

Once the web arrived, we centralized again. LAMP (Linux, Apache, MySQL, PHP) buried deep inside data centers, with the computer at the other end of the connection relegated to little more than a smart terminal rendering HTML. Load-balancers sprayed traffic across thousands of cheap machines. Eventually, the web turned from static sites to complex software as a service (SaaS) applications.

Then the pendulum swung back to the edge, and the clients got smart again. First with AJAX, Java, and Flash; then in the form of mobile apps, where the smartphone or tablet did most of the hard work and the back end was a communications channel for reporting the results of local action. Read more…

Comment
Four short links: 15 December 2014

Four short links: 15 December 2014

Transferable Learning, At-Scale Telemetry, Ugly DRM, and Fast Packet Processing

  1. How Transferable Are Features in Deep Neural Networks? — (answer: “very”). A final surprising result is that initializing a network with transferred features from almost any number of layers can produce a boost to generalization that lingers even after fine-tuning to the target dataset. (via Pete Warden)
  2. Introducing Atlas: Netflix’s Primary Telemetry Platform — nice solution to the problems that many have, at a scale that few have.
  3. The Many Facades of DRM (PDF) — Modular software systems are designed to be broken into independent pieces. Each piece has a clear boundary and well-defined interface for ‘hooking’ into other pieces. Progress in most technologies accelerates once systems have achieved this state. But clear boundaries and well-defined interfaces also make a technology easier to attack, break, and reverse-engineer. Well-designed DRMs have very fuzzy boundaries and are designed to have very non-standard interfaces. The examples of the uglified DRM code are inspiring.
  4. DPDKa set of libraries and drivers for fast packet processing […] to: receive and send packets within the minimum number of CPU cycles (usually less than 80 cycles); develop fast packet capture algorithms (tcpdump-like); run third-party fast path stacks.
Comment
Four short links: 8 December 2014

Four short links: 8 December 2014

Systemic Improvement, Chinese Trends, Deep Learning, and Technical Debt

  1. Reith Lectures — this year’s lectures are by Atul Gawande, talking about preventable failure and systemic improvement — topics of particular relevance to devops cultural devotees. (via BoingBoing)
  2. Chinese Mobile App UI Trends — interesting differences between US and China. Phone number authentication interested me: You key in your number and receive a confirmation code via SMS. Here, all apps offer this type of phone number registration/login (if not prefer it). This also applies to websites, even those without apps. (via Matt Webb)
  3. Large Scale Deep Learning (PDF) — Jeff Dean from Google. Starts easy! Starts.
  4. Machine Learning: The High-Interest Credit Card of Technical Debt (PDF) — Google research paper on the ways in which machine learning can create problems rather than solve them.
Comment: 1
Four short links: 30 September 2014

Four short links: 30 September 2014

Continuous Testing, Programmable Bees, Deep Learning on GPUs, and Silk Road Numbers

  1. Continuously Testing Infrastructure — “infrastructure as code”. I can’t figure out whether what I feel are thrills or chills.
  2. Engineer Sees Big Possibilities in Micro-robots, Including Programmable Bees (National Geographic) — He and fellow researchers devised novel techniques to fabricate, assemble, and manufacture the miniature machines, each with a housefly-size thorax, three-centimeter (1.2-inch) wingspan, and weight of just 80 milligrams (.0028 ounces). The latest prototype rises on a thread-thin tether, flaps its wings 120 times a second, hovers, and flies along preprogrammed paths. (via BoingBoing)
  3. cuDNN — NVIDIA’s library of primitives for deep neural networks (on GPUS, natch). Not open source (registerware).
  4. Analysing Trends in Silk Road 2.0If, indeed every sale can map to a transaction, some vendors are doing huge amounts of business through mail order drugs. While the number is small, if we sum up all the product reviews x product prices, we get a huge number of USD $20,668,330.05. REMEMBER! This is on Silk Road 2.0 with a very small subset of their entire inventory. A peek into a largely invisible economy.
Comment
Four short links: 26 September 2014

Four short links: 26 September 2014

Good Communities, AI Games, Design Process, and Web Server Library

  1. 15 Lessons from 15 Years of Blogging (Anil Dash) — If your comments are full of assholes, it’s your fault. Good communities don’t just happen by accident.
  2. Replicating DeepMind — open source attempt to build deep learning network that can play Atari games. (via RoboHub)
  3. ToyTalk — fantastic iterative design process for the product (see the heading “A Bit of Trickery”)
  4. h2oan optimized HTTP server implementation that can be used either as a standalone server or a library.
Comment
Four short links: 19 September 2014

Four short links: 19 September 2014

Deep Learning Bibliography, Go Playground, Tweet-a-Program, and Memory Management

  1. Deep Learning Bibliographyan annotated bibliography of recent publications (2014-) related to Deep Learning.
  2. Inside the Go Playground — on safely offering a REPL over the web to strangers.
  3. Wolfram Tweet-a-Program — clever marketing trick, and reminiscent of Perl Golf-style “how much can you fit into how little” contests.
  4. Memory Management Reference — almost all you ever wanted to know about memory management.
Comment