"finance" entries

Data science makes an impact on Wall Street

The O'Reilly Data Show Podcast: Gary Kazantsev on how big data and data science are making a difference in finance.

Charging_Bull_Sam_valadi_FlickrHaving started my career in industry, working on problems in finance, I’ve always appreciated how challenging it is to build consistently profitable systems in this extremely competitive domain. When I served as quant at a hedge fund in the late 1990s and early 2000s, I worked primarily with price data (time-series). I quickly found that it was difficult to find and sustain profitable trading strategies that leveraged data sources that everyone else in the industry examined exhaustively. In the early-to-mid 2000s the hedge fund industry began incorporating many more data sources, and today you’re likely to find many finance industry professionals at big data and data science events like Strata + Hadoop World.

During the latest episode of the O’Reilly Data Show Podcast, I had a great conversation with one of the leading data scientists in finance: Gary Kazantsev runs the R&D Machine Learning group at Bloomberg LP. As a former quant, I wanted to know the types of problems Kazantsev and his group work on, and the tools and techniques they’ve found useful. We also talked about data science, data engineering, and recruiting data professionals for Wall Street. Read more…

Comment
Four short links: 12 February 2015

Four short links: 12 February 2015

Finance Instrument, Retro Predictions, Trust Engineering, and Haptics

  1. Update on indie.vcWe’ve worked with the team at Cooley to create an investment instrument that has elements of both debt and equity. Debt in that we will not be purchasing equity initially, but, unlike debt, there is no maturity date, no collateralization of assets and no recourse if it’s never paid back. The equity element will only become a factor if the participating company chooses to raise a round of financing or sell out to an acquiring company. We don’t have a clever acronym or name for this instrument yet, but I’m sure we’ll come up with something great.
  2. How Nathan Barley Came True (Guardian) — if you haven’t already seen Nathan Barley, you should. It’s by the guy who did Black Mirror, and it’s both awful and authentic and predictive and retro and … painfully accurate about the horrors of our Internet/New Media industry. (via BoingBoing)
  3. Trust Engineers (Radio Lab) — Facebook has a created a laboratory of human behavior the likes of which we’ve never seen. We peek into the work of Arturo Bejar and a team of researchers who are tweaking our online experience, bit by bit, to try to make the world a better place. Radio show of goodness. (via Flowing Data)
  4. DARPA’S Haptix ProjectThe goal of the HAPTIX program is to provide amputees with prosthetic limb systems that feel and function like natural limbs, and to develop next-generation sensorimotor interfaces to drive and receive rich sensory content from these limbs. Today it’s prosthetic limbs for amputees, but within five years it’ll be augmented ad-driven realities for virtual currency ambient social recommendations.
Comment
Four short links: 27 January 2015

Four short links: 27 January 2015

Autonomous Corporations, Abstract Thought, Down Rounds, and Distributed Messaging

  1. Decentralised Autonomous Corporations — Charlie Stross’s near-future fiction of Accelerando comes closer to reality: Malice – revenge for waking him up – sharpens Manfred’s voice. “The president of agalmic.holdings.root.184.97.AB5 is agalmic.holdings.root.184.97.201. The secretary is agalmic.holdings.root.184.D5, and the chair is agalmic.holdings.root.184.E8.FF. All the shares are owned by those companies in equal measure, and I can tell you that their regulations are written in Python. Have a nice day, now!” He thumps the bedside phone control and sits up, yawning, then pushes the do-not-disturb button before it can interrupt again. After a moment he stands up and stretches, then heads to the bathroom to brush his teeth, comb his hair, and figure out where the lawsuit originated and how a human being managed to get far enough through his web of robot companies to bug him.
  2. Coding is Not the New Literacy (Chris Grainger) — We build mental models of everything – from how to tie our shoes to the way macro-economic systems work. With these, we make decisions, predictions, and understand our experiences. If we want computers to be able to compute for us, then we have to accurately extract these models from our heads and record them. Writing Python isn’t the fundamental skill we need to teach people. Modeling systems is. Amen!
  3. Let’s Stop Laughing at Groupon (Fortune) — it is much easier to survive a valuation decline as a public company than as a private one.
  4. nsq — Bitly’s open sourced realtime distributed messaging platform.
Comment
Four short links: 1 October 2014

Four short links: 1 October 2014

Robot Learning, Internet Confidentiality, Bootstrap Material Design, and Bitcoin Adoption

  1. Robotics Has Too Many Dreamers, Needs More Practical People (IEEE) — Grishin said that while looking for business opportunities, he saw too may entrepreneurs proposing cool new robots and concepts but with no business cases to support them. The robotics industry, he added, needs more startups to fail to allow entrepreneurs to learn from past mistakes and come up with more enduring plans. A reminder that first to found rarely correlates to biggest exit.
  2. Fixing the Internet for Confidentiality and Security (Mark Shuttleworth) — Every society, even today’s modern Western society, is prone to abusive governance. We should fear our own darknesses more than we fear others. I like the frame of “confidentiality” vs “privacy”.
  3. Bootstrap Material Design — a material design theme for Bootstrap. Material design (Google’s new design metaphor/language for interactive UIs) is important, to mobile and web what HIG was to MacOS, and it specifically tackles the noisy surprises that are app and web interfaces today.
  4. Simon Wardley on BitcoinWhy I think US will adopt bitcoin … it is currently backed by $284m in venture capital, you’re going to get it whether you like it or not.
Comment

Network Science dashboards

Networks graphs can be used as primary visual objects with conventional charts used to supply detailed views

With Network Science well on its way to being an established academic discipline, we’re beginning to see tools that leverage it. Applications that draw heavily from this discipline make heavy use of visual representations and come with interfaces aimed at business users. For business analysts used to consuming bar and line charts, network visualizations take some getting used. But with enough practice, and for the right set of problems, they are an effective visualization model.

In many domains, networks graphs can be the primary visual objects with conventional charts used to supply detailed views. I recently got a preview of some dashboards built using Financial Network Analytics (FNA). Read more…

Comment

Financial analytics as a service

Analytic services are tailoring their solutions for specific problems and domains

In relatively short order Amazon’s internal computing services has become the world’s most successful cloud computing platform. Conceived in 2003 and launched in 2006, AWS grew quickly and is now the largest web hosting company in the world. With the recent addition of Kinesis (for stream processing), AWS continues to add services and features that make it an attractive platform for many enterprises.

A few other companies have followed a similar playbook: technology investments that benefit a firm’s core business, is leased out to other companies, some of whom may operate in the same industry. An important (but not well-known) example comes from finance. A widely used service provides users with clean, curated data sets and sophisticated algorithms with which to analyze them. It turns out that the world’s largest asset manager makes its investment and risk management systems available to over 150 pension funds, banks, and other institutions. In addition to the $4 trillion managed by BlackRock, the company’s Aladdin Investment Management system is used to manage1 $11 trillion in additional assets from external managers.

Read more…

Comment: 1

On Being a Data Skeptic

"Modelers have a bigger responsibility now than ever before."

On Being a Data Skeptic CoverPeople come to data science in all sorts of ways. I happen to be someone who came via finance. Trained as a mathematician, I worked first at a hedge fund and then a financial risk software company, each for about two years, starting in June 2007 and ending in February 2011. If you look at those dates again, you’ll realize I had a front row seat for the financial crisis.

I worked on a few projects in algorithmic trading with Larry Summers at the hedge fund and was invited, along with the other quants at Shaw, to see him discuss the impending doom one evening with Alan Greenspan and Robert Rubin. It honestly kind of surprised and shocked me to see how little they seemed to know, or at least admitted to knowing, about the true situation in the markets. These guys were supposed to be the experts, after all.

Read more…

Comment: 1

Nate Silver confuses cause and effect, ends up defending corruption

A math band-aid will distract us from fixing the problems that so desperately need fixing.

This piece originally appeared on Mathbabe. We’re also including Jordan Ellenberg’s counter-point to Cathy’s original post as well as Cathy’s response to Jordan. All of these pieces are republished with permission.


I just finished reading Nate Silver’s newish book, The Signal and the Noise: Why so many predictions fail – but some don’t.

The good news

First off, let me say this: I’m very happy that people are reading a book on modeling in such huge numbers – it’s currently eighth on the New York Times best seller list and it’s been on the list for nine weeks. This means people are starting to really care about modeling, both how it can help us remove biases to clarify reality and how it can institutionalize those same biases and go bad.

As a modeler myself, I am extremely concerned about how models affect the public, so the book’s success is wonderful news. The first step to get people to think critically about something is to get them to think about it at all.

Moreover, the book serves as a soft introduction to some of the issues surrounding modeling. Silver has a knack for explaining things in plain English. While he only goes so far, this is reasonable considering his audience. And he doesn’t dumb the math down.

In particular, Silver does a nice job of explaining Bayes’ Theorem. (If you don’t know what Bayes’ Theorem is, just focus on how Silver uses it in his version of Bayesian modeling: namely, as a way of adjusting your estimate of the probability of an event as you collect more information. You might think infidelity is rare, for example, but after a quick poll of your friends and a quick Google search you might have collected enough information to reexamine and revise your estimates.)

The bad news

Having said all that, I have major problems with this book and what it claims to explain. In fact, I’m angry.

It would be reasonable for Silver to tell us about his baseball models, which he does. It would be reasonable for him to tell us about political polling and how he uses weights on different polls to combine them to get a better overall poll. He does this as well. He also interviews a bunch of people who model in other fields, like meteorology and earthquake prediction, which is fine, albeit superficial.

What is not reasonable, however, is for Silver to claim to understand how the financial crisis was a result of a few inaccurate models, and how medical research need only switch from being frequentist to being Bayesian to become more accurate. Read more…

Comments: 3
Four short links: 2 November 2012

Four short links: 2 November 2012

Tablet table, Google contest for students, watch your incentives, research on web search abandonment.

  1. Latest Tablets (Luke Wroblewski) — table showing the astonishing variety of tablets released in the last two months.
  2. Google Code-In Contest for High Schoolersan international contest introducing 13-17 year old pre-university students to the world of open source software development. The goal of the contest is to give students the opportunity to explore the many types of projects and tasks involved in open source software development. (via Andy Oram)
  3. Watch Your Incentives — NASDAQ added two new incentives programs, and robotraders responded. On November 1st, there were 369 seconds where the number of quotes in BAC exceeded 17,000; a total of 6.6 million quotes. During those seconds, only 1,879 trades executed. Between market open (9:30am) and 12:45, BAC had 7.8 million quotes and 116,000 trades. Which means 85% of all BAC quotes occurred in those 369 seconds. Which means it is likely that one algo from one firm (all of this quote spam is from Nasdaq) is responsible for 85% of all canceled orders in BAC. (via Robert O’Brien)
  4. Predicting Web Search Abandonment Rationales (PDF) — Microsoft Research paper on how to predict why people cease to search or to click through on the search results. I’m really impressed by how well they could distinguish “bah, the Internet doesn’t have it, I’m giving up” from “oh, the answer was in a search result snippet, my work here is done.” (via Mark Alen)
Comment
Four short links: 1 October 2012

Four short links: 1 October 2012

Crowdsourcing Flights, Teaching Programming, Redeploying Finance Engineers, and Recognising Cat Faces

  1. FlightfoxReal people compete to find you the best flights. Crowdsourcing beating algorithms …. (via NY Times)
  2. Code Monster (Crunchzilla) — a fun site for parents to learn to program with their kids. Loving seeing so much activity around teaching kids to program. (via Greg Linden)
  3. Telling People to Leave Finance (Cathy O’Neil) — There’s an army of engineers in finance that could be putting their skills to use with actual innovation rather than so-called financial innovation.
  4. Kittydar (GitHub) — cat face recognition in Javascript.
Comment: 1