- Robotics Has Too Many Dreamers, Needs More Practical People (IEEE) — Grishin said that while looking for business opportunities, he saw too may entrepreneurs proposing cool new robots and concepts but with no business cases to support them. The robotics industry, he added, needs more startups to fail to allow entrepreneurs to learn from past mistakes and come up with more enduring plans. A reminder that first to found rarely correlates to biggest exit.
- Fixing the Internet for Confidentiality and Security (Mark Shuttleworth) — Every society, even today’s modern Western society, is prone to abusive governance. We should fear our own darknesses more than we fear others. I like the frame of “confidentiality” vs “privacy”.
- Bootstrap Material Design — a material design theme for Bootstrap. Material design (Google’s new design metaphor/language for interactive UIs) is important, to mobile and web what HIG was to MacOS, and it specifically tackles the noisy surprises that are app and web interfaces today.
- Simon Wardley on Bitcoin — Why I think US will adopt bitcoin … it is currently backed by $284m in venture capital, you’re going to get it whether you like it or not.
"finance" entries
Four short links: 1 October 2014
Robot Learning, Internet Confidentiality, Bootstrap Material Design, and Bitcoin Adoption
Network Science dashboards
Networks graphs can be used as primary visual objects with conventional charts used to supply detailed views
With Network Science well on its way to being an established academic discipline, we’re beginning to see tools that leverage it. Applications that draw heavily from this discipline make heavy use of visual representations and come with interfaces aimed at business users. For business analysts used to consuming bar and line charts, network visualizations take some getting used. But with enough practice, and for the right set of problems, they are an effective visualization model.
In many domains, networks graphs can be the primary visual objects with conventional charts used to supply detailed views. I recently got a preview of some dashboards built using Financial Network Analytics (FNA). Read more…
Financial analytics as a service
Analytic services are tailoring their solutions for specific problems and domains
In relatively short order Amazon’s internal computing services has become the world’s most successful cloud computing platform. Conceived in 2003 and launched in 2006, AWS grew quickly and is now the largest web hosting company in the world. With the recent addition of Kinesis (for stream processing), AWS continues to add services and features that make it an attractive platform for many enterprises.
A few other companies have followed a similar playbook: technology investments that benefit a firm’s core business, is leased out to other companies, some of whom may operate in the same industry. An important (but not well-known) example comes from finance. A widely used service provides users with clean, curated data sets and sophisticated algorithms with which to analyze them. It turns out that the world’s largest asset manager makes its investment and risk management systems available to over 150 pension funds, banks, and other institutions. In addition to the $4 trillion managed by BlackRock, the company’s Aladdin Investment Management system is used to manage1 $11 trillion in additional assets from external managers.
On Being a Data Skeptic
"Modelers have a bigger responsibility now than ever before."
People come to data science in all sorts of ways. I happen to be someone who came via finance. Trained as a mathematician, I worked first at a hedge fund and then a financial risk software company, each for about two years, starting in June 2007 and ending in February 2011. If you look at those dates again, you’ll realize I had a front row seat for the financial crisis.
I worked on a few projects in algorithmic trading with Larry Summers at the hedge fund and was invited, along with the other quants at Shaw, to see him discuss the impending doom one evening with Alan Greenspan and Robert Rubin. It honestly kind of surprised and shocked me to see how little they seemed to know, or at least admitted to knowing, about the true situation in the markets. These guys were supposed to be the experts, after all.
Nate Silver confuses cause and effect, ends up defending corruption
A math band-aid will distract us from fixing the problems that so desperately need fixing.
This piece originally appeared on Mathbabe. We’re also including Jordan Ellenberg’s counter-point to Cathy’s original post as well as Cathy’s response to Jordan. All of these pieces are republished with permission.
I just finished reading Nate Silver’s newish book, The Signal and the Noise: Why so many predictions fail – but some don’t.
The good news
First off, let me say this: I’m very happy that people are reading a book on modeling in such huge numbers – it’s currently eighth on the New York Times best seller list and it’s been on the list for nine weeks. This means people are starting to really care about modeling, both how it can help us remove biases to clarify reality and how it can institutionalize those same biases and go bad.
As a modeler myself, I am extremely concerned about how models affect the public, so the book’s success is wonderful news. The first step to get people to think critically about something is to get them to think about it at all.
Moreover, the book serves as a soft introduction to some of the issues surrounding modeling. Silver has a knack for explaining things in plain English. While he only goes so far, this is reasonable considering his audience. And he doesn’t dumb the math down.
In particular, Silver does a nice job of explaining Bayes’ Theorem. (If you don’t know what Bayes’ Theorem is, just focus on how Silver uses it in his version of Bayesian modeling: namely, as a way of adjusting your estimate of the probability of an event as you collect more information. You might think infidelity is rare, for example, but after a quick poll of your friends and a quick Google search you might have collected enough information to reexamine and revise your estimates.)
The bad news
Having said all that, I have major problems with this book and what it claims to explain. In fact, I’m angry.
It would be reasonable for Silver to tell us about his baseball models, which he does. It would be reasonable for him to tell us about political polling and how he uses weights on different polls to combine them to get a better overall poll. He does this as well. He also interviews a bunch of people who model in other fields, like meteorology and earthquake prediction, which is fine, albeit superficial.
What is not reasonable, however, is for Silver to claim to understand how the financial crisis was a result of a few inaccurate models, and how medical research need only switch from being frequentist to being Bayesian to become more accurate. Read more…
Four short links: 2 November 2012
Tablet table, Google contest for students, watch your incentives, research on web search abandonment.
- Latest Tablets (Luke Wroblewski) — table showing the astonishing variety of tablets released in the last two months.
- Google Code-In Contest for High Schoolers — an international contest introducing 13-17 year old pre-university students to the world of open source software development. The goal of the contest is to give students the opportunity to explore the many types of projects and tasks involved in open source software development. (via Andy Oram)
- Watch Your Incentives — NASDAQ added two new incentives programs, and robotraders responded. On November 1st, there were 369 seconds where the number of quotes in BAC exceeded 17,000; a total of 6.6 million quotes. During those seconds, only 1,879 trades executed. Between market open (9:30am) and 12:45, BAC had 7.8 million quotes and 116,000 trades. Which means 85% of all BAC quotes occurred in those 369 seconds. Which means it is likely that one algo from one firm (all of this quote spam is from Nasdaq) is responsible for 85% of all canceled orders in BAC. (via Robert O’Brien)
- Predicting Web Search Abandonment Rationales (PDF) — Microsoft Research paper on how to predict why people cease to search or to click through on the search results. I’m really impressed by how well they could distinguish “bah, the Internet doesn’t have it, I’m giving up” from “oh, the answer was in a search result snippet, my work here is done.” (via Mark Alen)
Four short links: 1 October 2012
Crowdsourcing Flights, Teaching Programming, Redeploying Finance Engineers, and Recognising Cat Faces
- Flightfox — Real people compete to find you the best flights. Crowdsourcing beating algorithms …. (via NY Times)
- Code Monster (Crunchzilla) — a fun site for parents to learn to program with their kids. Loving seeing so much activity around teaching kids to program. (via Greg Linden)
- Telling People to Leave Finance (Cathy O’Neil) — There’s an army of engineers in finance that could be putting their skills to use with actual innovation rather than so-called financial innovation.
- Kittydar (GitHub) — cat face recognition in Javascript.
What caused New York’s startup boom?
A few early and broad questions in our exploration of NYC's startup community.
Since the crisis of 2008 New York City’s massive financial sector — the city’s richest economic engine, once seen to have unlimited potential for growth — has languished. In the meantime, attention has turned to its nascent startup sector, home to Foursquare, Tumblr, 10gen, Etsy and Gilt, where VC investment has surged even as it’s been flat in other big U.S. tech centers (PDF).
I’ve started to poke around the tech community here with a view toward eventually publishing a paper on the rise of New York’s startup scene. In my initial conversations, I’ve come up with a few broad questions I’ll focus on, and I’d welcome thoughts from this blog’s legion of smart readers on any of these.
- How many people in New York’s startup community came from finance, and under what conditions did they make the move? In 2003, Google was a five-year-old, privately-held startup and Bear Stearns was an 80-year-old pillar of the financial sector. Five years later, Google was a pillar of the technical economy and among the world’s biggest companies; Bear Stearns had ceased to exist. Bright quantitatively-minded people who might have pursued finance for its stability and lucre now see that sector as unstable and not necessarily lucrative; its advantage over the technology sector in those respects has disappeared. Joining a 10-person startup is very different from taking a job at Google, but the comparative appeal of the two sectors has dramatically shifted.
- To what degree have anchor institutions played a role in the New York startup scene? The relationship between Stanford University and Silicon Valley is well-documented; I’d like to figure out who’s producing steady streams of bright technologists in New York. Google’s Chelsea office, opened in 2006, now employs close to 3,000 people, and its alumni include Dennis Crowley, founder of Foursquare. That office is now old enough that it can generate a high volume of spin-offs as Googlers look for new challenges. And Columbia and NYU (and soon a Cornell-Technion consortium) have embraced New York’s startup community.
Wall Street’s robots are not out to get you
Putting high-frequency trading into perspective.
Technology is critical to today’s financial markets. It’s also surprisingly controversial. In most industries, increasing technological involvement is progress, not a problem. And yet, people who believe that computers should drive cars suddenly become Luddites when they talk about computers in trading.
There’s widespread public sentiment that technology in finance just screws the “little guy.” Some of that sentiment is due to concern about a few extremely high-profile errors. A lot of it is rooted in generalized mistrust of the entire financial industry. Part of the problem is that media coverage on the issue is depressingly simplistic. Hyperbolic articles about the “rogue robots of Wall Street” insinuate that high-frequency trading (HFT) is evil without saying much else. Very few of those articles explain that HFT is a catchall term that describes a host of different strategies, some of which are extremely beneficial to the public market.
I spent about six years as a trader, using automated systems to make markets and execute arbitrage strategies. From 2004-2011, as our algorithms and technology became more sophisticated, it was increasingly rare for a trader to have to enter a manual order. Even in 2004, “manual” meant instructing an assistant to type the order into a terminal; it was still routed to the exchange by a computer. Automating orders reduced the frequency of human “fat finger” errors. It meant that we could adjust our bids and offers in a stock immediately if the broader market moved, which enabled us to post tighter markets. It allowed us to manage risk more efficiently. More subtly, algorithms also reduced the impact of human biases — especially useful when liquidating a position that had turned out badly. Technology made trading firms like us more profitable, but it also benefited the people on the other sides of those trades. They got tighter spreads and deeper liquidity.
Some sideways thinking about cyberwarfare
Cyberwarfare needs to be framed far more broadly.
When we hear the term “cyberwarfare” we think of government-backed hackers stealing data, or releasing viruses or other software exploits to disrupt another country’s capabilities, communications, or operations. We imagine terrorists or foreign hackers planning to destroy America’s power grid, financial systems, or communications networks, or stealing our secrets.
I’ve been thinking, though, that it may be useful to frame the notion of cyberwarfare far more broadly. What if we thought of JP Morgan’s recent trading losses not simply as a “bad bet” but as the outcome of a cyberwar between JP Morgan and hedge funds? More importantly, what if we thought of the Euro’s current troubles in part as the result of a cyberwar between the financial industry and the EU?
When two nations with differing goals attack each other, we call it warfare. But when financial firms attack each other, or the financial industry attacks the economy of nations, we tell ourselves that it’s “the efficient market” at work. In fact the Eurozone crisis is a tooth-and-claw battle between central bankers and firms seeking profit for themselves despite damage to the livelihoods of millions.
When I see headlines like “Merkel says Euro Rescue Funds Needed Against Speculators” or “Speculators Attacking the Euro” or “Banksters Take Us to the Brink” it’s pretty clear to me that we need to stop thinking of the self-interested choices made by financial firms as “just how it is,” and to think of them instead as hostile activities. And these activities are largely carried out by software trading bots, making them, essentially, a cyberwar between profiteers and national economies (i.e. the rest of us).
Read more…