- Influx DB — open-source, distributed, time series, events, and metrics database with no external dependencies.
- Omega (PDF) — flexible, scalable schedulers for large compute clusters. From Google Research.
- GraspJS — Search and replace your JavaScript code based on its structure rather than its text.
- Amazon Mines Its Data Trove To Bet on TV’s Next Hit (WSJ) — Amazon produced about 20 pages of data detailing, among other things, how much a pilot was viewed, how many users gave it a 5-star rating and how many shared it with friends.
"amazon" entries
Four short links: 5 November 2013
Time Series Database, Cluster Schedulers, Structural Search-and-Replace, and TV Data
Four short links: 3 October 2013
USB in Cars, Capture Presentations, Amazon Redshift, and Polytweeting
- Hyundia Replacing Cigarette Lighters with USB Ports (Quartz) — sign of the times. (via Julie Starr)
- Freeseer — free, open source, cross-platform application that captures or streams your desktop—designed for capturing presentations. Would you like freedom with your screencast?
- Amazon Redshift: What You Need to Know — good write-up of experience using Amazon’s column database.
- GroupTweet — Allow any number of contributors to Tweet from a group account safely and securely. (via Jenny Magiera)
Four short links: 1 October 2013
Ploughbot, Amazon Warehouses, Kickstarting Safety, and The Island of Dr Thoreau
- Farmbot Wiki — open-source, scalable, automated precision farming machines.
- Amazon’s Chaotic Storage — photos from inside an Amazon warehouse. At the heart of the operation is a sophisticated database that tracks and monitors every single product that enters/leaves the warehouse and keeps a tally on every single shelf space and whether it’s empty or contains a product. Software-optimised spaces, for habitation by augmented humans.
- Public Safety Codes of the World — Kickstarter project to fund the release of public safety codes.
- #xoxo Thoreau Talk (Maciej Ceglowski) — exquisitely good talk by the Pinboard creator, on success, simplicity, and focus.
Four short links: 5 September 2013
Bezos on Business, CS Ratios, Easier Hadoopery, and AWS CLI
- Bezos at the Post (Washington Post) — “All businesses need to be young forever. If your customer base ages with you, you’re Woolworth’s,” added Bezos.[…] “The number one rule has to be: Don’t be boring.” (via Julie Starr)
- How Carnegie-Mellon Increased Women in Computer Science to 42% — outreach, admissions based on potential not existing advantage, making CS classes practical from the start, and peer support.
- Summingbird (Github) — Twitter open-sourced library that lets you write streaming MapReduce programs that look like native Scala or Java collection transformations and execute them on a number of well-known distributed MapReduce platforms like Storm and Scalding.
- aws-cli (Github) — commandline for Amazon Web Services. (via AWS Blog)
Podcast: ratings, rankings, and the advantage of being born lucky
A conversation with Sean Taylor, Hilary Mason, and John Myles White about how ratings affect our thinking
Researchers randomly upvoted some posts and downvoted others on a popular message board. The upvoted posts became substantially more popular over the long run. Image via Sean Taylor.
Is popularity just a matter of simple luck–of some early advantage compounded by human preference for things that are already popular? A paper published today in Science offers some insight into the way that popularity emerges in online ratings. Lev Muchnik, Sinan Aral, and Sean Taylor were able to set up a randomized experiment on a popular Reddit-like message board in which they gave some posts a one-point upvote on publication and others a one-point downvote. Posts that were “born lucky” ended up with 25% higher scores on average than those without modification.
In our latest podcast, Renee DiResta and I are joined by Sean Taylor, Hilary Mason and John Myles White to talk about Sean’s findings and about ratings, rankings and reviews in general. Bits and pieces that come up in the podcast:
- Anchoring and adjustment
- Daniel Kahneman’s Thinking, Fast and Slow; his Nobel Prize lecture is worth watching, too
- Amazon reviews both satirical and just poorly informed
- Health inspection results can be predicted from online reviews
- Restaurant grades are less effective in the age of Yelp
- Speaking of Yelp:
Subscribe to the O’Reilly Radar Podcast through iTunes, SoundCloud, or directly through our podcast’s RSS feed.
Four short links: 14 June 2014
UK Gov 2.0, Remote Work, Git + App Engine, and Amazon Sells 3D Printer Goodies
- How Geeks Opened up the UK Government (Guardian) — excellent video introduction to how the UK is transforming its civil service to digital delivery. Most powerful moment for me was scrolling through various depts’ web sites and seeing consistent visual design.
- Tools for Working Remotely — Braid’s set of tools (Trello, Hackpad, Slingshot, etc.) for remote software teams.
- Git Push to Deploy on Google App Engine — Enabling this feature will create a remote Git repository for your application’s source code. Pushing your application’s source code to this repository will simultaneously archive the latest the version of the code and deploy it to the App Engine platform.
- Amazon’s 3D Printer Store — printers and supplies. Deeply underwhelming moment of it arriving on the mainstream.
What Is the Risk That Amazon Will Go Down (Again)?
Velocity 2013 Speaker Series
Why should we at all bother about notions such as risk and safety in web operations? Do web operations face risk? Do web operations manage risk? Do web operations produce risk? Last Christmas Eve, Amazon had an AWS outage affecting a variety of actors, including Netflix, which was a service included in many of the gifts shared on that very day. The event has introduced the notion of risk into the discourse of web operations, and it might then be good timing for some reflective thoughts on the very nature of risk in this domain.
What is risk? The question is a classic one, and the answer is tightly coupled to how one views the nature of the incident occurring as a result of the risk.
One approach to assessing the risk of Amazon going down is probabilistic: start by laying out the entire space of potential scenarios leading to Amazon going down, calculate their probability, and multiply the probability for each scenario by their estimated severity (likely in terms of the costs connected to the specific scenario depending on the time of the event). Each scenario can then be plotted in a risk matrix showing their weighted ranking (to prioritize future risk mitigation measures) or calculated as a collective sum of the risks for each scenario (to judge whether the risk for Amazon going down is below a certain acceptance criterion).
This first way of answering the question of what the risk is for Amazon to go down is intimately linked with a perception of risk as energy to be kept contained (Haddon, 1980). This view originates from more recent times of increased development of process industries in which clearly graspable energies (fuel rods at nuclear plants, the fossil fuels at refineries, the kinetic energy of an aircraft) are to be kept contained and safely separated from a vulnerable target such as human beings. The next question of importance here becomes how to avoid an uncontrolled release of the contained energy. The strategies for mitigating the risk of an uncontrolled release of energy are basically two: barriers and redundancy (and the two combined: redundancy of barriers). Physically graspable energies can be contained through the use of multiple barriers (called “defenses in depth”) and potentially several barriers of the same kind (redundancy), for instance several emergency-cooling systems for a nuclear plant.
Using this metaphor, the risk of Amazon going down is mitigated by building a system of redundant barriers (several server centers, backup, active fire extinguishing, etc.). This might seem like a tidy solution, but here we run into two problems with this probabilistic approach to risk: the view of the human operating the system and the increased complexity that comes as a result of introducing more and more barriers.
Controlling risk by analyzing the complete space of possible (and graspable) scenarios basically does not distinguish between safety and reliability. From this view, a system is safe when it is reliable, and the reliability of each barrier can be calculated. However there is one system component that is more difficult to grasp in terms of reliability than any other: the human. Inevitably, proponents of the energy/barrier model of risk end up explaining incidents (typically accidents) in terms of unreliable human beings not guaranteeing the safety (reliability) of the inherently safe (risk controlled by reliable barriers) system. I think this problem—which has its own entire literature connected to it—is too big to outline in further detail in this blog post, but let me point you towards a few references: Dekker, 2005; Dekker, 2006; Woods, Dekker, Cook, Johannesen & Sarter, 2009. The only issue is these (and most other citations in this post) are all academic tomes, so for those who would prefer a shorter summary available online, I can refer you to this report. I can also reassure you that I will get back to this issue in my keynote speech at the Velocity conference next month. To put the critique short: the contemporary literature questions the view of humans as the unreliable component of inherently safe systems, and instead advocates a view of humans as the only ones guaranteeing safety in inherently complex and risky environments.
Read more…
Four short links: 23 May 2013
Amazon Slash Slashed, Indies Out, Printing for Peace, Massively Online Orthographic Build System
- Kindle Worlds Fine Print — Amazon’s fanfic publishing system has a few flaws: no pr0n, no crossovers, no slash, and Amazon Publishing will acquire all rights to your new stories, including global publication rights, for the term of copyright. I can’t see this attracting pinboard’s most passionate users.
- XBox One Won’t Allow Indies to Self-Publish Games — When it comes to self-publishing, Microsoft is the odd man out. Both Sony and Nintendo allow developers to publish their own games onto PlayStation Network and Nintendo Network, respectively. Microsoft’s position stands in stark contrast to Sony, which has been aggressively pursuing indie content for PS4. (via Andy Baio)
- 3D Printers for Peace Competition (Michigan Tech) — We are challenging the 3D printing community to design things that advance the cause of peace. This is an open-ended contest, but if you’d like some ideas, ask yourself what Mother Theresa, Martin Luther King, or Ghandi would make if they’d had access to 3D printing. (via BoingBoing)
- covim — Collaborative editing for vim. My dream of massively multiplayer troff can finally be realised.
Commerce Weekly: Amazon patent indicates its interest in the payments space
Amazon patent may address payment privacy concerns, Warby Parker outfits store with sensors, and Alipay launches sound wave payments.
Editor’s note: This will be the final installment of our Commerce Weekly series.
Mobile payments security, privacy concerns rise; Amazon may have a solution
The race is on to democratize mobile payments, to create a solution that improves the payment experience for consumers and merchants to the extent that it will replace traditional payment methods. Some experts, however, are concerned that technology developments are failing to address increasing concerns with security and privacy.
Kirk Ladendorf reports this week that smartphone security software company NQ Mobile noted a rise in worldwide phone malware threats from 24,000 in 2011 to 65,000 in 2012. In an interview with Ladendorf, Gavin Kim, chief commercial officer at NQ Mobile, warned that “[s]martphone sales are booming, and they are becoming a much more targeted device by hackers.”
Brent Warrington, CEO of online and mobile payment company SecureNet, disagreed, telling Ladendorf that he’s “comfortable and confident in the level of security of [payment] transactions” through his company, noting that the transaction information is “encrypted from end to end.”
While security concerns may be getting addressed, Ladendorf says that privacy advocates don’t see the same attention being given to privacy concerns and the “potential misuse of growing mountains of electronic data tied to the spending patterns of individual consumers.” Ladendorf notes that the same advances that make mobile payments more enjoyable and convenient also make it easier for companies to mine consumer data. He quotes from a McClatchy newspaper interview with Jeff Chester, executive director of the Center for Digital Democracy, in which Chester said, “[Mobile payment] is about exposing your financial behavior to a daisy chain of financial and other marketers who have a very detailed understanding of where you are, how you spend your time and what you buy.”
Commerce Weekly: Wal-Mart looks outside the (big) box to battle Amazon
Wal-Mart and Google pursue speedy delivery. Elsewhere, more reasons for retailers to fear smartphones, and mobile may be eBay's best bet.
Wal-Mart wants to crowdsource delivery, while Google chases same-day
On the heels of launching its in-store delivery locker program to compete with Amazon Locker, Wal-Mart has announced it’s toying with the idea of having in-store customers deliver online orders to speed delivery times. Reporting on the news at Reuters, Alistair Barr and Jessica Wohl note that, in essence, Wal-Mart would be experimenting with the growing crowdsourcing trend that works well in so many other areas, so why not for Wal-Mart delivery? They write:
“A plethora of start-ups now help people make money by renting out a spare room, a car, or even a cocktail dress, and Wal-Mart would in effect be inviting people to rent out space in their vehicle and their willingness to deliver packages to others.”
Barr and Wohl mention a few of the “why nots” — numerous legal, regulatory and privacy obstacles — but report that Joel Anderson, chief executive of Walmart.com, believes it to be a viable plan. “This is at the brain-storming stage,” he says, “but it’s possible in a year or two.”
At Bloomberg’s Businessweek, Susan Berfield points to the bigger picture: “Even if the idea never moves past the hypothetical, the fact that Anderson is even talking about it signals how serious a threat Walmart considers Amazon.” Wired’s Laura Heller agrees, noting that though there are “far too many unattractive variables” for this program to become a reality, “it shows the retailer is thinking outside of the box when it comes to competing with its online competition, Amazon.”