- Update on indie.vc — We’ve worked with the team at Cooley to create an investment instrument that has elements of both debt and equity. Debt in that we will not be purchasing equity initially, but, unlike debt, there is no maturity date, no collateralization of assets and no recourse if it’s never paid back. The equity element will only become a factor if the participating company chooses to raise a round of financing or sell out to an acquiring company. We don’t have a clever acronym or name for this instrument yet, but I’m sure we’ll come up with something great.
- How Nathan Barley Came True (Guardian) — if you haven’t already seen Nathan Barley, you should. It’s by the guy who did Black Mirror, and it’s both awful and authentic and predictive and retro and … painfully accurate about the horrors of our Internet/New Media industry. (via BoingBoing)
- Trust Engineers (Radio Lab) — Facebook has a created a laboratory of human behavior the likes of which we’ve never seen. We peek into the work of Arturo Bejar and a team of researchers who are tweaking our online experience, bit by bit, to try to make the world a better place. Radio show of goodness. (via Flowing Data)
- DARPA’S Haptix Project — The goal of the HAPTIX program is to provide amputees with prosthetic limb systems that feel and function like natural limbs, and to develop next-generation sensorimotor interfaces to drive and receive rich sensory content from these limbs. Today it’s prosthetic limbs for amputees, but within five years it’ll be augmented ad-driven realities for virtual currency ambient social recommendations.
Networks graphs can be used as primary visual objects with conventional charts used to supply detailed views
With Network Science well on its way to being an established academic discipline, we’re beginning to see tools that leverage it. Applications that draw heavily from this discipline make heavy use of visual representations and come with interfaces aimed at business users. For business analysts used to consuming bar and line charts, network visualizations take some getting used. But with enough practice, and for the right set of problems, they are an effective visualization model.
In many domains, networks graphs can be the primary visual objects with conventional charts used to supply detailed views. I recently got a preview of some dashboards built using Financial Network Analytics (FNA). Read more…
Analytic services are tailoring their solutions for specific problems and domains
In relatively short order Amazon’s internal computing services has become the world’s most successful cloud computing platform. Conceived in 2003 and launched in 2006, AWS grew quickly and is now the largest web hosting company in the world. With the recent addition of Kinesis (for stream processing), AWS continues to add services and features that make it an attractive platform for many enterprises.
A few other companies have followed a similar playbook: technology investments that benefit a firm’s core business, is leased out to other companies, some of whom may operate in the same industry. An important (but not well-known) example comes from finance. A widely used service provides users with clean, curated data sets and sophisticated algorithms with which to analyze them. It turns out that the world’s largest asset manager makes its investment and risk management systems available to over 150 pension funds, banks, and other institutions. In addition to the $4 trillion managed by BlackRock, the company’s Aladdin Investment Management system is used to manage1 $11 trillion in additional assets from external managers.
"Modelers have a bigger responsibility now than ever before."
People come to data science in all sorts of ways. I happen to be someone who came via finance. Trained as a mathematician, I worked first at a hedge fund and then a financial risk software company, each for about two years, starting in June 2007 and ending in February 2011. If you look at those dates again, you’ll realize I had a front row seat for the financial crisis.
I worked on a few projects in algorithmic trading with Larry Summers at the hedge fund and was invited, along with the other quants at Shaw, to see him discuss the impending doom one evening with Alan Greenspan and Robert Rubin. It honestly kind of surprised and shocked me to see how little they seemed to know, or at least admitted to knowing, about the true situation in the markets. These guys were supposed to be the experts, after all.
A math band-aid will distract us from fixing the problems that so desperately need fixing.
This piece originally appeared on Mathbabe. We’re also including Jordan Ellenberg’s counter-point to Cathy’s original post as well as Cathy’s response to Jordan. All of these pieces are republished with permission.
I just finished reading Nate Silver’s newish book, The Signal and the Noise: Why so many predictions fail – but some don’t.
The good news
First off, let me say this: I’m very happy that people are reading a book on modeling in such huge numbers – it’s currently eighth on the New York Times best seller list and it’s been on the list for nine weeks. This means people are starting to really care about modeling, both how it can help us remove biases to clarify reality and how it can institutionalize those same biases and go bad.
As a modeler myself, I am extremely concerned about how models affect the public, so the book’s success is wonderful news. The first step to get people to think critically about something is to get them to think about it at all.
Moreover, the book serves as a soft introduction to some of the issues surrounding modeling. Silver has a knack for explaining things in plain English. While he only goes so far, this is reasonable considering his audience. And he doesn’t dumb the math down.
In particular, Silver does a nice job of explaining Bayes’ Theorem. (If you don’t know what Bayes’ Theorem is, just focus on how Silver uses it in his version of Bayesian modeling: namely, as a way of adjusting your estimate of the probability of an event as you collect more information. You might think infidelity is rare, for example, but after a quick poll of your friends and a quick Google search you might have collected enough information to reexamine and revise your estimates.)
The bad news
Having said all that, I have major problems with this book and what it claims to explain. In fact, I’m angry.
It would be reasonable for Silver to tell us about his baseball models, which he does. It would be reasonable for him to tell us about political polling and how he uses weights on different polls to combine them to get a better overall poll. He does this as well. He also interviews a bunch of people who model in other fields, like meteorology and earthquake prediction, which is fine, albeit superficial.
What is not reasonable, however, is for Silver to claim to understand how the financial crisis was a result of a few inaccurate models, and how medical research need only switch from being frequentist to being Bayesian to become more accurate. Read more…
Tablet table, Google contest for students, watch your incentives, research on web search abandonment.
- Latest Tablets (Luke Wroblewski) — table showing the astonishing variety of tablets released in the last two months.
- Google Code-In Contest for High Schoolers — an international contest introducing 13-17 year old pre-university students to the world of open source software development. The goal of the contest is to give students the opportunity to explore the many types of projects and tasks involved in open source software development. (via Andy Oram)
- Watch Your Incentives — NASDAQ added two new incentives programs, and robotraders responded. On November 1st, there were 369 seconds where the number of quotes in BAC exceeded 17,000; a total of 6.6 million quotes. During those seconds, only 1,879 trades executed. Between market open (9:30am) and 12:45, BAC had 7.8 million quotes and 116,000 trades. Which means 85% of all BAC quotes occurred in those 369 seconds. Which means it is likely that one algo from one firm (all of this quote spam is from Nasdaq) is responsible for 85% of all canceled orders in BAC. (via Robert O’Brien)
- Predicting Web Search Abandonment Rationales (PDF) — Microsoft Research paper on how to predict why people cease to search or to click through on the search results. I’m really impressed by how well they could distinguish “bah, the Internet doesn’t have it, I’m giving up” from “oh, the answer was in a search result snippet, my work here is done.” (via Mark Alen)
Crowdsourcing Flights, Teaching Programming, Redeploying Finance Engineers, and Recognising Cat Faces
- Flightfox — Real people compete to find you the best flights. Crowdsourcing beating algorithms …. (via NY Times)
- Code Monster (Crunchzilla) — a fun site for parents to learn to program with their kids. Loving seeing so much activity around teaching kids to program. (via Greg Linden)
- Telling People to Leave Finance (Cathy O’Neil) — There’s an army of engineers in finance that could be putting their skills to use with actual innovation rather than so-called financial innovation.
A few early and broad questions in our exploration of NYC's startup community.
Since the crisis of 2008 New York City’s massive financial sector — the city’s richest economic engine, once seen to have unlimited potential for growth — has languished. In the meantime, attention has turned to its nascent startup sector, home to Foursquare, Tumblr, 10gen, Etsy and Gilt, where VC investment has surged even as it’s been flat in other big U.S. tech centers (PDF).
I’ve started to poke around the tech community here with a view toward eventually publishing a paper on the rise of New York’s startup scene. In my initial conversations, I’ve come up with a few broad questions I’ll focus on, and I’d welcome thoughts from this blog’s legion of smart readers on any of these.
- How many people in New York’s startup community came from finance, and under what conditions did they make the move? In 2003, Google was a five-year-old, privately-held startup and Bear Stearns was an 80-year-old pillar of the financial sector. Five years later, Google was a pillar of the technical economy and among the world’s biggest companies; Bear Stearns had ceased to exist. Bright quantitatively-minded people who might have pursued finance for its stability and lucre now see that sector as unstable and not necessarily lucrative; its advantage over the technology sector in those respects has disappeared. Joining a 10-person startup is very different from taking a job at Google, but the comparative appeal of the two sectors has dramatically shifted.
- To what degree have anchor institutions played a role in the New York startup scene? The relationship between Stanford University and Silicon Valley is well-documented; I’d like to figure out who’s producing steady streams of bright technologists in New York. Google’s Chelsea office, opened in 2006, now employs close to 3,000 people, and its alumni include Dennis Crowley, founder of Foursquare. That office is now old enough that it can generate a high volume of spin-offs as Googlers look for new challenges. And Columbia and NYU (and soon a Cornell-Technion consortium) have embraced New York’s startup community.