Wall Street’s robots are not out to get you

Putting high-frequency trading into perspective.

ABOVE by Lyfetime, on FlickrTechnology is critical to today’s financial markets. It’s also surprisingly controversial. In most industries, increasing technological involvement is progress, not a problem. And yet, people who believe that computers should drive cars suddenly become Luddites when they talk about computers in trading.

There’s widespread public sentiment that technology in finance just screws the “little guy.” Some of that sentiment is due to concern about a few extremely high-profile errors. A lot of it is rooted in generalized mistrust of the entire financial industry. Part of the problem is that media coverage on the issue is depressingly simplistic. Hyperbolic articles about the “rogue robots of Wall Street” insinuate that high-frequency trading (HFT) is evil without saying much else. Very few of those articles explain that HFT is a catchall term that describes a host of different strategies, some of which are extremely beneficial to the public market.

I spent about six years as a trader, using automated systems to make markets and execute arbitrage strategies. From 2004-2011, as our algorithms and technology became more sophisticated, it was increasingly rare for a trader to have to enter a manual order. Even in 2004, “manual” meant instructing an assistant to type the order into a terminal; it was still routed to the exchange by a computer. Automating orders reduced the frequency of human “fat finger” errors. It meant that we could adjust our bids and offers in a stock immediately if the broader market moved, which enabled us to post tighter markets. It allowed us to manage risk more efficiently. More subtly, algorithms also reduced the impact of human biases — especially useful when liquidating a position that had turned out badly. Technology made trading firms like us more profitable, but it also benefited the people on the other sides of those trades. They got tighter spreads and deeper liquidity.

Many HFT strategies have been around for decades. A common one is exchange arbitrage, which Time magazine recently described in an article entitled “High Frequency Trading: Wall Street’s Doomsday Machine?”:

A high-frequency trader might try to take advantage of minuscule differences in prices between securities offered on different exchanges: ABC stock could be offered for one price in New York and for a slightly higher price in London. With a high-powered computer and an ‘algorithm,’ a trader could buy the cheap stock and sell the expensive one almost simultaneously, making an almost risk-free profit for himself.

It’s a little bit more difficult than that paragraph makes it sound, but the premise is true — computers are great for trades like that. As technology improved, exchange arb went from being largely manual to being run almost entirely via computer, and the market in the same stock across exchanges became substantially more efficient. (And as a result of competition, the strategy is now substantially less profitable for the firms that run it.)

Market making — posting both a bid and an offer in a security and profiting from the bid-ask spread — is presumably what Knight Capital was doing when it experienced “technical difficulties.” The strategy dates from the time when exchanges were organized around physical trading pits. Those were the bad old days, when there was little transparency and automation, and specialists and brokers could make money ripping off clients who didn’t have access to technology. Market makers act as liquidity providers, and they are an important part of a well-functioning market. Automated trading enables them to manage their orders efficiently and quickly, and helps to reduce risk.

So how do those high-profile screw-ups happen? They begin with human error (or, at least, poor judgment). Computerized trading systems can amplify these errors; it would be difficult for a person sending manual orders to simultaneously botch their markets in 148 different companies, as Knight did. But it’s nonsense to make the leap from one brokerage experiencing severe technical difficulties to claiming that automated market-making creates some sort of systemic risk. The way the market handled the Knight fiasco is how markets are supposed to function — stupidly priced orders came in, the market absorbed them, the U.S. Securities and Exchange Commission (SEC) and the exchanges adhered to their rules regarding which trades could be busted (ultimately letting most of the trades stand and resulting in a $440 million loss for Knight).

There are some aspects of HFT that are cause for concern. Certain strategies have exacerbated unfortunate feedback loops. The Flash Crash illustrated that an increase in volume doesn’t necessarily mean an increase in real liquidity. Nanex recently put together a graph (or a “horrifying GIF“) showing the sharply increasing number of quotes transmitted via automated systems across various exchanges. What it shows isn’t actual trades, but it does call attention to a problem called “quote spam.” Algorithms that employ this strategy generate a large number of buy and sell orders that are placed in the market and then are canceled almost instantly. They aren’t real liquidity; the machine placing them has no intention of getting a fill — it’s flooding the market with orders that competitor systems have to process. This activity leads to an increase in short-term volatility and higher trading costs.

The New York Times just ran an interesting article on HFT that included data on the average cost of trading one share of stock. From 2000 to 2010, it dropped from $.076 to $.035. Then it appears to have leveled off, and even increased slightly, to $.038 in 2012. If (as that data suggests) we’ve arrived at the point where the “market efficiency” benefit of HFT is outweighed by the risk of increased volatility or occasional instability, then regulators need to step in. The challenge is determining how to disincentivize destabilizing behavior without negatively impacting genuine liquidity providers. One possibility is to impose a financial transaction tax, possibly based on how long the order remains in the market or on the number of orders sent per second.

Rethinking regulation and market safeguards in light of new technology is absolutely appropriate. But the state of discourse in the mainstream press — mostly comprised of scare articles about “Wall Street’s terrifying robot invasion” — is unfortunate. Maligning computerized strategies because they are computerized is the wrong way to think about the future of our financial markets.

Photo: ABOVE by Lyfetime, on Flickr


tags: , ,
  • http://web.ncf.ca/shawnhcorey/ Shawn H Corey

    Well, I’m a computer programmer and I think allowing computers to do the trading is the dumbest idea yet. Here’s why:


    And this problem didn’t even involved computers; it was all done with mechanical relays.

    Just because something is locally optimized doesn’t mean the system isn’t headed for disaster.

  • http://twitter.com/littleidea Andrew Clay Shafer

    The problem isn’t using technology. The problem is inherent in the potential to ‘execute arbitrage strategies’.

    The claim of tighter spreads and deeper liquidity appears to be provably untrue, but I’d love to see data to support this assertion.

    I don’t have a problem with self driving cars, but I also don’t want to see them driving around beyond the limits of speeds humans can react to, particularly when they are going to be interacting with those humans and their reactions.

    Improbable events lead to catastrophes.

    Models for pricing options are predicated on there being no arbitrage opportunities in the market. When arbitrage strategies provably exist, they should be removed.

    A per transaction tax needs to offset the potential to seek HFT arbitrage opportunities. This seems like the straight forward solution, of course, implementation is another issue.

    I don’t have an issue with capturing value, but by definition, ‘arbitrage’ creates none.

    • Alex Tolley

      I don’t see an intrinsic problem with arbitrage. Arbitrage includes strategies like hedging, risk management, etc. As long as these activities prevent gross mispricing of assets, that is fine and arguably keeps most players honest.

      The question is, how equal do prices really need to be, and what is the impact on the behavior of actors when this is the primary goal? [I am deliberately excluding the scalping activities here, and which should a separate topic].

  • http://twitter.com/Lexwerks Jason Miller

    Still not answered: “High-frequency trading raises an existential question for capitalism…: Why do we [even] have stock markets?” (http://www.wired.com/business/2012/08/ff_wallstreet_trading/all/) To put it another way, if shares of companies are thrown about like so much electronic confetti, what does it mean to invest in a company or own shares of a corporation? I find it interesting that “own” and “invest” don’t show up in this defense of HFT. But, picking up and extending from Andrew’s comment, it’s like we’ve got a self-driving car that was invented for no greater purpose than driving around on roads — which is not to be confused with moving humans to where the humans want to be. In this same way we’re trying to get from the present to the future by putting a lot of trust in the stock market that has codified values of “arbitrage and liquidity” rather than “invest and own.”

    Which words describe what you’re trying to do with those 401K or IRA or other retirement contributions?

    • Boris Iyutin

      Stock price reflects company book price plus company current perceived opportunity. The latter is very volatile. If you can not stomach volatility, invest in bonds.

      HFT companies are not there to invest, they are making sure that prices are the same on all the markets and reflect all available information.

      • Alex Tolley

        HFT companies are not there to invest, they are making sure that prices are the same on all the markets and reflect all available information.

        Prices must be equal at microsecond intervals? Nonsense.

        • iyutin

          Information propagates with the speed of light, so are the prices. Just because human brain can not handle it at this speed, does not makes it nonsense.

  • http://twitter.com/hugh_knowles Hugh Knowles

    1. HFT is just demonstrating the systemic problems in the financial system but at the millisecond scale rather than months or years.
    2. Surely HFT is another example of humans mistaking speed for progress. A better question would be do the markets serve the purpose of creating real value for society and does HFT help us do that? Currently i would venture the answer to both of those questions is ‘no’. HFT would not be nearly so prevalent if we were trying to create long term value rather than shallow short term gains.

  • Randy Parker

    Maybe HFT provides some slight improvement in market efficiency, though I doubt it is significant. On the other hand, it opens the door to whole new landscapes of difficult to detect and prosecute fraud. From high speed front-running, to encrypted patterns of quotes signaling intent to colluding algorithms, to schemes beyond a layman’s imagination, my bet is that the reliable profits produced by HFT often result from crimes that the SEC is unable to detect. If a scheme is theoretically possible, we should assume it has been implemented.

    How many finance firms would walk away from a profitable network of algorithms that they believe are undetectable?

  • Stephen

    I’m an outsider to HFT but two points strike me.
    1. As a biologist I’m acutely sensitive to the idea of chaotic systems failures – in cell signalling, in ecosystems even in the brain (e.g. epilepsy). I understand that the HFT systems in question are supposed to have failsafes- but on a number of occasions these don’t seem to have worked. It seems to me that the more dominant the HFT trades become, the larger the volume, higher speed with lower margins then the less resilient the system becomes. It also strikes me that this story sounds like a repeat of many recent financial disasters – sophisticated mathematical trading strategies that were superbly adapted to the market model and worked brilliantly right up to the moment the market dynamics totally flipped and they stopped working (LTCM). So I’d like to know why the failsafes failed the last few times – and why they were special cases?
    2. There is a deeper unease here I think. The algorithms are perhaps adding to a feeling that the profits are ‘money for nothing’. No real value is being created or service is being provided but a game is being played that creates money for the lucky initiates. Adam Smith at least had a theory about how trade created wealth for both parties – even nations- that was probably in large part true. The suspicion here is that the wealth created is just gaming a system with the players grabbing more and more of economic wealth with no real output or contribution of their own- just shuffling and chiseling with each shuffle. After all it’s a totally automatic system! More and more I have some sympathy with this view.

    • Doug Hill25

      Stephen: “The algorithms are perhaps adding to a feeling that the profits are ‘money for nothing’. No real value is being created or service is being provided but a game is being played that creates money for the lucky initiates.”

      My thoughts exactly. I kept wanting to ask the author, excuse my naïveté, but is there anything exactly being created here (other than profits)?

      • iyutin

        >My thoughts exactly. I kept wanting to ask the author, excuse my naïveté, but is there anything exactly being created here (other than profits)?

        Yes, reduced transaction cost and higher market accessibility for all the participants. Among other things, HFT is a big investor in infrastructure and new tech. It speeds up innovation cycle.

  • JudgeFedd

    The argument about the algorithms of HFT systems misses the point, the real problem with HFT is that they often exploit the plumbing of the exchanges to gain an unfair advantage over non HFT traders (i.e. the individual investor). Companies utilizing HFT systems co-locate directly into the exchange in order to gain an advantage in speed. They can see the entire depth of the order book and can front end any order traditionally placed into the exchange. Imagine what you would be able to do if you can front end the market. They have the ability to add orders to the market that you will not be able to see and many times orders are place in the market as bait only to be canceled right before they are executed. This chips away at normal market activity as a parasite feeding off it’s host. The markets and exchanges have become a mockery of what they once were. Bottom line is HFT are yet another way for Wall Street to use its power to extract wealth from people without creating any value.

    • Walker McKittrick

      “Exploit the plumbing of the exchanges to gain an unfair advantage” is so very spot on.

      Today’s HFT’s only exist because their computers are physically close (and electronically closer) to the exchanges’ computers so that their trades ‘arrive’ at the exchange faster than anybody else’s.

      The average trader cannot place his computer as close to the exchange’s computer as the HFT trader can, because it’s very expensive to do so. (And it’s expensive because there’s not physically enough space in the data center to host the computers of millions of traders.)

      Move those HFT computers outside of the exchange’s data center (or impose a fixed delay in time on every trade, which logically amounts to the same thing) and the HFT advantage goes away almost entirely.

      Is this not a classic case of exploiting the plumbing?

      I find it depressing that intelligent people like Renee do not make intellectually honest arguments in discussions like this. She could very well forgo repeating the talking points from the industry (which she had an obligation to repeat verbatim when she was a trader) as she is no longer part of that cult.

      • iyutin

        > or impose a fixed delay in time on every trade, which logically amounts to the same thing

        There are a lot of slow exchanges where fixed delay naturally exists because of old tech on the exchange side. It does not hurt HFT at all. Colos – are the way for exchanges to make money the way they used to do it long ago by charging for a desk on the trading floor.

        • JudgeFedd

          It has been said that many HFT system don’t use conventional order types (market, limit, etc.). They have “lower level” order types that allow them to submit invisible orders into the market (so called dark pools, etc.). It’s a rigged game where the HFT system are playing with cards you couldn’t possibly have.

          You do make a good point about the exchanges making money off the co-location costs. This is one motive to allow this behavior to continue.

          In the end HFT systems do nothing to help price discovery or liquidity.

          • http://noupside.tumblr.com Renee

            Dark pools are not “the market” in the sense that you seem to think. They exist to cross massive block orders…the kind of trades that used to be done over the phone, professional to professional. Individual investors have no business there.

            Are there some HFT shops that use flash orders and the like? Yes. Is it a majority of them? No. My main point was that one of the problems with this discussion is that there’s no nuance, and a majority of casual commenters are misinformed.

            As you so helpfully pointed out, I no longer have an “obligation” to parrot industry talking points…so maybe the fact that I’ve expressed this opinion means that I believe it.

          • iyutin

            to Judge Fedd: You might want to read Wikipedia on hidden orders and dark pools before starting the argument. You are clearly confused. No offense.

          • JudgeFedd

            The point I was trying to make is that HFT systems submit invisible trades to the market. Dark Pools are similar as it is also invisible to most normal market participants. Both are ways for these large financial institutions to game the system to their advantage. Yes Dark Pools was slightly off topic but still another perversion of the market.

            Renee, BTW I wasn’t the one commenting on your motivation.

  • Pingback: Four short links: 21 August 2012 - O'Reilly Radar()

  • Alex Tolley

    Imagine what you would be able to do if you can front end the market.

    Exactly. And “front running” orders is [was?] illegal.

    The author is so young that she doesn’t appear to understand the purpose of stock markets, and assumes that the activities she [profitably?] engaged in, is desirable and a primary function of the market, rather than an emergent behavior.

    As Upton Sinclair said: “It is difficult to get a man to understand something, when his salary depends on his not understanding it.”

    • http://www.ravimik.com/ Ravi Mikkelsen

      I wouldn’t exactly say that it’s an emergent behavior, but rather a continuing trend of increasing communications technology. Investors have always sought to increase the speed at which they received information about the stocks and were then able to act upon it, via steam ships, telegraph, telephone & now the computer.

      If the primary use case of the market is changing (whether because of technology, regulation or our own desires) wouldn’t that then change the primary function of the market?

      • Alex Tolley

        If the primary use case of the market is changing … wouldn’t that then change the primary function of the market?

        I don’t think so. The primary function of the stock market is to allow public raising of capital for companies and to ensure enough liquidity that shares can be bought and sold in volume. The purpose of liquidity is to reduce ownership risk – i.e. you can readily liquidate holdings in changing circumstances.

        Historically, market makers on the floor of the exchange ensured liquidity. During the 1987 crash, a notable feature was that NYSE market makers, although overwhelmed, did continue making markets. In contrast, the computer based dealer market, NASDAQ, dealers simply stepped away from their computers and liquidity completely disappeared. The rather suspect penny stock markets that had been active during the summer of 1987, provided no bids for most stocks, IOW, there was no liquidity.

        Computer based arbitrage was practiced during this period, e.g. stock index arbitrage through program trading, although execution required coordinated buying or selling of stocks on the NYSE.

        At about the same time, the trading in currencies was already larger than global trading, and was already becoming a game between traders.

        Fast forward to today. Trading volume vastly dwarfs that of the late 1980’s, hugely increasing turnover. Should this be necessary for individuals and mutual funds to manage stock portfolios? Obviously not.

        What has happened is that stock (and bond) markets have become games run between players. HFT is just another weapon in that game, and has nothing to do with the primary function of the market. At its worst, it is just scalping, an activity that is understood to be a public bad in other markets, like ticket sales.

        HFT is just the bete noir of the 2000’s, but the growing scale of “trading” activity is the problem. Trading needs to be kept to a secondary activity, because its current dominance has effects on the primary function of the market and the behavior of its players.

        • iyutin

          > In contrast, the computer based dealer market, NASDAQ, dealers simply stepped away from their computers and liquidity completely disappeared.

          Yes, because they did not have obligation to stay on the market and loose money.

          To avoid this some exchanges have a requirement for market makers to maintain minimum spread or face steep fines. Usually obligations like these come with some favors like guaranteed market share or lower transaction fee. During flash crashes the risk of staying on the market becomes so high that market makers prefer to pay fines.

          > Trading needs to be kept to a secondary activity, because its current dominance has effects on the primary function of the market and the behavior of its players.

          It IS a secondary activity. The biggest movers of the market are usually macro economic news,

    • http://noupside.tumblr.com Renee

      Front running is and always has been illegal and unethical, whether one does it with fast computers or via unscrupulous human floor brokers capitalizing on information asymmetry and their clients’ lack of access to technology. Interestingly, the Wikipedia article on front-running focuses almost exclusively on the latter.

      BTW I wish you hadn’t brought up my age; it’s quite condescending.

  • Boris Borcic

    It deserves pointing out that HFT implies the time-slicing of ownership in slices too narrow for any human mind to know oneself owner (or representing an owner). This denies assumptions on the nature and meaning of ownership that thousands of years have ingrained in convention.

  • Alexa Tolley

    While much of HFT may be arbitrage (who knows?) it is also clear that it has automated gaming.

    In the human scale world, spreading incorrect rumors to manipulate stock prices is illegal. Yet providing fake signals is at least as old as Rothschild’s famous faked selling after the battle of Waterloo, and exemplified in the comedy movie, “Trading Places”.

    This gaming in the markets has steadily developed. Traders have long since been able to see the order flow, but as long as they were not also executing client orders, that “inside information” was considered OK. But large trades can also influence markets, and smart firms did some game playing to improve price execution.

    As true arbitrage opportunities dry up, the trading resources will increasingly be used to play games, much as happened in the currency markets; because how do you arbitrage currencies?.

    Thus HFT just becomes an exercise in game theory, having little to do with arbitraging prices and certainly nothing to do with the primary function of the market.

    • iyutin

      > As true arbitrage opportunities dry up, the trading resources will increasingly be used to play games

      There is a long list of the countries/products which are not yet on electronic market. There will be space for improvement for some time.

      Then there is always a balance between how much you invest in infrastructure vs how much you can make arbitraging. When these two numbers getting closer to each other, participants start moving to other fields.