Tag Archives: high frequency trading

What is it exactly that quants do?

This is the fourth post in a series of curated interviews from Reddit’s IAmA subreddit, a place where individuals of various professions and backgrounds can request the community to “ask me anything”. There has been a lot of new readers to the blog recently, so I strongly recommend new readers to first read my first post on investment bankers.

New readers should read part 1 and part 2 of this series also.

I have selected some curated questions and responses from that, in my opinion, comprise some of the best quant/algorithmic trading/high frequency trading interviews on Reddit.

I am a day trader who is up 35% over the past 12mos. AMA

• What sort of algorithms does the bot use? What is its strategy?

The bot uses reversion to the mean strategies and is fully automated (its running as we speak) and completely non-discretionary. It requires no intervention – I could die right now and it wouldn’t even notice. It would keep trading and shut itself down at the close. I do extensive back testing using GA optimization and validate using Monte Carlo and walk forward techniques. The bot runs 100% off live data, but is optimized using historical data. I get my historical end of day and intraday data from IQFeed and real time data via my broker’s (Interactive Brokers) API.

My system is reversion to the mean, it doesn’t trade trends. I also don’t use moving averages or any other technical analysis indicators. Its all statistically based. I’d agree backtesting is worthless if you aren’t careful how you do it. Its certainly very easy to uncover fool’s gold. Note that I don’t use the off the shelf dogshit most retail traders attempt to optimize parameters with, plus I test everything 100% out of sample. EVERYONE who trades does some level of backtesting, even discretionary traders. Unless you’re wiping your brain clean each day, your trade decisions are all made based on the positive or negative outcomes of your prior trades. We’re all curve fitting, some of us are just more optimal than others about how we do it.

• Any books you could suggest? Websites you frequent or did frequent to get your feet wet?

I’ve found the best trading books are those from the academic Computational Finance community. Stay away from the Wade Cook garbage at Barnes and Noble. Here’s some favorites:

• Biologically Inspired Algorithms for Financial Modelling (Natural Computing Series)
• Computational Intelligence in Economics and Finance (Advanced Information Processing) Volume 1 and II
• Evidence-Based Technical Analysis: Applying the Scientific Method and Statistical Inference to Trading Signals
• How to Solve It: Modern Heuristics
• Way of the Turtle: The Secret Methods that Turned Ordinary People into Legendary Traders
• The Black Swan: The Impact of the Highly Improbable

In terms of websites, I hang around the Elite Trader forums quite a bit. They’re mean as hell there, so put on your thickest flame jacket before you post, but there’s some nuggets of useful info buried amongst the endless sophomoric dick slamming. Look at the posts by lescor and acrary. lescor (my personal hero) is a former fireman turned prop trader who pulls $50K+/month out of the markets and his equity curve is nearly a straight line.

• Is there a specific broker that you can suggest with lower fees?

I’d suggest Interactive Brokers (they are who I use). They charge 0.005/share but you can get that down slightly with rebates if you add liquidity. There are other brokers out there, but IB is really the best option for retail traders. You can trade stocks, options, futures, and/or FX through IB. And they have the largest selection of markets to trade, including international markets like the DAX, KOSPI, FTSE, etc. Its hard to find a market they don’t offer, really.

Quant/high frequency trader “expert” AMA

• How much does the latency of the data feed play into high frequency trading?

Latency is extremely important if the signals you trade are short term (up to hours). most exchanges have “colocation” facilities in which firms can rent space to run computers with their trading software. Every millisecond is important; firms spend millions of dollars to get an edge over the competitors. A lot of automated trading systems are not high frequency, so being the fastest isn’t that important. Most high frequency trading strategies are based on market microstructure (e.g. There are more shares on the best bid than the best offer, so the price is more likely to go up). It’s a strategic game that’s separated from supply and demand considerations. It is definitely the case that different dollar-amount stocks behave differently in microstructure and in short term movements.

• What percentage of the total daily volume on SP500 index futures do you need to trade in order for HF trading to work ? Do you trade any illiquid equities, stocks with a small float ? What aspect of statistics do you use in your daily job ? Do you back-test your strategies on historical data ? If yes, how far back do you test ?

• You can trade a very small amount (1 share per day) and still qualify as high frequency if the signals you are trading are based on high-frequency market data.
• Yes, we trade illiquid stocks. Any strategy tailored for liquid names obviously has to be rethought for illiquid names, but it is possible and profitable to trade them.
• I do a lot of regressions and similar testing to fit models, etc.
• Yes, we backtest strategies. If it is a simple code change, we might just test a few months; for serious risk management strategy change, it could be a couple years.

• What does high frequency mean? What would be an average or target duration for the length a trade stays open?

High frequency strategies are generally characterized by:

* High cancellation ratio – most limit orders are cancelled before they are filled.
* Low holding period – on the order of hours or less.

In general, high frequency traders don’t try to buy and immediately flip for a higher price (there are statarb strategies that do this). Instead, you usually have ‘signals’ that give you an indication of where things are going based on factors like market microstructure. Your signals indicate that, if you put out a buy limit order and you get filled, then you are in expectation making money. That doesn’t mean you should sell right away when you get filled; you again should generally sell only when you think you are making money by selling.

Is it as simple as “If X happens, does Y?” or is it much more complicated? Back when automated trading was a semi-serious hobby for me, people would say they trade “on book”, not on price. If that sounds familiar to you, could you give an example?

Examples (suppose A and B are correlated):

Trading on price: if stock A moves up and stock B moves down, predict they will converge in price (sell A and buy B).

Trading on book: if there are more bids than offers for stock A and more bids than offers for stock B, predict they are both going up (buy/post bids for both, or at least cancel offers at the front of the queue.)

Some strategies have very simple ideas and others are very mathematically complex.

IAmA partner at a mid-sized proprietary trading firm

•  Could you also give some insight into the technology stack you use (programming languages, OS, open source tools, etc)?

 I’m going to assume that you, like most hobbyists, are trying to dump your data directly from the feed into the database. Your generic databases aren’t designed to be used like this. Take your feed data, process it however you need to, and then just append it to a binary log file, flushing every X amount of bytes. At the end of day, clean or process the data, and then insert it into your RDBMS.
If you need to query lots of data (billion and up), your RDBMS wont be optimal. You can try selectively loading sets of data you want to work with from your binary log files into the RDBMS, but this will limit your working set. You can try to use one of the NoSQL databases if your access patterns are a good match.
If you want something that will perform very well, simple emulate the way kdb+ or Vertica store data. Sort your binary log files by keys, index the file, and memory map for fast searching. On top of that you can splay the file, add block level compression, etc.

We work with several technologies.

• Linux/Windows
• C/C++ and Java with a sprinkling of some lesser known language like Clojure/J
• Lots of open source and we do give back
• Some proprietary third party software/hardware. A lot of our infrastructure was custom built earlier last decade and has been/is being phased out.

• Could you talk a little more about general infrastructure and what parts are key to operating HFT strategies? How do you decide what programming language to choose or rather what is your development process?

I think individuals can compete if they know their shit, put in the time, focus on a niche, and have capital. In my opinion, most people are living in a pipe dream if they think they can go from novice to ATM operator printing money. That is not how it works. There are things larger firms will not operate in because it simply isn’t worth it. You have to consider that there are fixed costs and we need X amount of profit to make deploying something worthwhile. These niche markets are probably less common than one may think. Carve out a strategy where you have a competitive advantage, don’t try to beat me at my own game.
You need low latency infrastructure and fast execution, what specifically do you want to know? If we need to develop and implement something, we define constraints and requirements and pick a language from there. It’s very much a best tool for the job process and generally high performance components are written in C++, enterprise components in Java, prototyping/fast tooling is done in some scripting language, and text processing in awk.

What is it exactly that quants do?

This is the third post in a series of curated interviews from Reddit’s IAmA subreddit, a place where individuals of various professions and backgrounds can request the community to “ask me anything”. There has been a lot of new readers to the blog recently, so I strongly recommend new readers to first read my first post on investment bankers and my second post on algorithmic trading and high frequency trading.

I have selected some curated questions and responses from that, in my opinion, comprise some of the best quant/algorithmic trading/high frequency trading interviews on Reddit.

IAMA 100% automated independent retail trader. I trade around 800k to 1.5 million shares a day and make 2cents/trade on average. AMAA

• What books do you recommend?

Early on, I was in nasty drawdown period and I was having trouble figuring out what was off. I made the same mistake virtually all traders make, I caved to the vast collection of trading psychology books. When the guy mentoring me found out what I was reading, he gave me the following gem: Only pikers worry about psychology, either you have an edge and you exploit it, or you don’t have one and you lose and chase every other excuse.

Trading and Exchanges
Options, Futures & Other Derivatives
Option Volatility & Pricing
Volatility Trading
Dynamic Hedging

99% of finance books are garbage, but those are the ones I thought helped me in some way or another. There’s also plenty of interesting research papers if you’ve got access to some databases.

• What was your process in developing your model?

I don’t have just 1 strategy/model/whatever you want to call it. I have a portfolio of 6 strategies, the oldest being the one I initially started out with. I have had to retire several strategies. When a strategy falls below benchmark performance I evaluate and then either continue or retire it. My strategies range from high accuracy with 2:1 risk:reward ratios to low accuracy and low risk:reward ratios. When I develop a model, I attempt to exploit some fundamental behavior whether that be news, volume, trader driven. From there, I systematically formalize rules and test.

• What is your process in strategy development? In other words, how were the patterns discovered?

• I rarely look at bar/candlestick charts. I’ll peek at one after hours or when I pull up the S&Ps or something like that. I personally don’t put much weight into price patterns/indicators, so I tend to ignore those. That being said, I do visualize data to spot behavior, such as mean reversion. I have a whole suite of scripts in Matlab and more recently R to help me analyze timeseries. When I develop a strategy, I lay down what I’m trying to exploit. For example, if I notice that an instrument is poised to be range bound for some period of time, I’ll go with a mean reverting model and adjust as needed.
• I do use volume, but it’s become a diminishing factor.
• I use any information I can get my hands on. From tick data to fundamentals, news, relative strength to other equities, etc.

IAmA High-Frequency Trader. AMA.

•  I seem to get that it’s a very computer-oriented thing, so what exactly are you doing? Telling the computer what to do? What was your field of study in college?

 Mainly, we create statistical models of a variety of products in order to have some type of predicted price. I was in Computer Science, making the discrete math and probability right up my alley.

•  What kind of resources would one need to pull together a high frequency trading program?

My best estimate: about half a dozen people, six to twelve months, and about $5-10 million to get a workable (but simple and narrow) high frequency trading firm. The capital needed for trading isn’t very high, but the technological outlay is pretty big.

•  Legal issues aside, how easy is it (if it’s possible at all) for an employee to take the system/software and run a copy at home for his own gain?

 Impossible. The amount of money that must be spent on infrastructure to even think about putting together a simple HF trade is probably close to the $10 million mark. Plus, you have no reason to. Work on your trade at work, and, while you won’t get 100% of the profits, you’ll get a good enough chunk to make up for all the infrastructure you get.

Algo Trader at a NYC HFT firm

• Where exactly do you come in? I assume you are not executing or approving individual trades (as they would be moving way to fast), nor do you seem to be writing the code. (I might be wrong here.) Are you setting strategy – i.e. something like “today we are going to try to exploit the spread between natty gas and crude”?

Good question, because you’re right – I am neither writing code, nor manually trading all day. What it comes down to is really watching everything, and reacting. Not everything can be automated. This may sounds simple, but it takes a holistic understanding of both your products, your aims, your markets, your exchanges, to do it well. There are times when for 2 hours i will be doing nothing than doing mental math, checking that every trade the system does is good. This is in quiet times.

Then there are times, when for some reason an algo starts building up a huge position. And the limits we have on it need to be increased, because we see opportunity. I must manually change this, then think, wait, how will this affect our exposure in X industry. What is our interest rate and currency exposure. Which exposure do I want? Which exposure do fellow desks have? How big do we want to go? Who is the counterparty, if there is a way to know? How can I adjust the algo settings? Maybe be more, or less aggressive? Think about similar algos, how they trade similar ways, and we need to tweak them relatively?

These are a few of the concerns that I automatically consider as I see all my trades aggregating, and I start to make decisions based on all the news and information I have coming into me, be it from Bloomberg, from brokers, even CNBC if I’m too busy to be reading. I cant leave this up to the computer because there is always a human element. And like we all know, leaving it in control of machines can lead to unexpected black swan disasters due to glitches or random inferior machine logic.

• Say you have a trading idea. How do you test it?

We have databases full of old tick data that we backtest on, but most of them you are pretty certain will work and will just put into use. This is simply by experience and just knowing the markets – as a simple example, if we can have our computers read the ECB’s interest rate announcement first, and be the fastest to trade, we will clearly make money. Most are far more complex, but that’s the idea – most should work. If it loses for the first few days or hours, we retool it and can eventually dump it. But that’s rare – even if it makes only a bit, we keep it. Leverage is used for general company wide leverage, not specific per trade. Books – Hull is one of the basics. Trading ideas, you would probably learn more from reading all of Wikipedia’s finance tabs, Investopedia, and the various industry biographies – like FIASCO, Liar’s poker, Genius Failed, Den of Theives, etc. The more algo based strategies are almost all derivatives of basic trading ideas, just faster. Some elaborate stat stuff, but a lot of it is quite simple. And the elaborate stuff no one is going to share, sorry.

• Can you explain/describe what your average day is like?

Get in around 7, having read some sort of financial publication on the way to work. Spend first half hour checking positions, reviewing current FX rates, how the Japan markets closed and how the Euro ones have opened, make sure nothing happened in the futures markets to move anything too much. Read general news. Next hour prepping systems, various manual checks and such, checking strategies, double and triple checks, never can be too sure. Then prep for market open.

During open hours, need at least 2 guys of 3 at my desk at any time, one to be eyeing and checking every trade that comes through, another to be doing accounting/admin/checking positions/pnl/random stuff with backoffice and risk. Of course, monitoring, retooling, reacting and pre-acting on various information that comes along.

Then by the close make sure all positions are as we want, otherwise we would have to act on them on the close – various ways to do this, or work with the japanese markets and australians as they start up – or futures – or fx – theres always a market to work on/in.

Then finish up mundane tasks of finishing the day and closing the books by 5 or so. Until 8 or whenever we feel like going home, working on new strats. Sometimes do this intraday depending on vola.

• I want to do some side investing with my money. Any books that you recommend and particular strategies I should pay attention to?

Totally depends on your risk profile, capital levels, and access to markets.
Go to the local library and read every book on the markets and trading they have. I’d say that would be around 100 books, if you know everything in them you’ll be set. The biggest mistake people make is not knowing enough before they start. Really only trade a few thousand to get a feel for trading, while you are reading and learning, and until you can objectively look at yourself and say, I am more knowledgeable than 80% of market participants. That may take a year of reading a few books a week until you seriously start trading, but it’s the only way to be truly successful.

I will be posting more interviews soon, so please follow Curated Alpha via Email, RSS, or Twitter.

What is it exactly that quants do?

This is the second post in a series of curated interviews from Reddit’s IAmA subreddit, a place where individuals of various professions and backgrounds can request the community to “ask me anything”. The first post I wrote about investment bankers was well received, and I received several requests for IAmAs on other positions in the financial services industry, so I have decided to write a series of posts on quants that work with algorithmic trading and high frequency trading.

I encourage new readers to read the first post to understand my motivation for curating these interviews, but it can basically be boiled down to this: Practitioners in the financial industry generally aren’t motivated to share information with others and the ones that do are sometimes at risk of being fired by their employers. This results in significantly less robust online financial discussion when compared to other online communities.

There is also an issue of signal to noise — 95% of financial-related websites are crap because their primary purpose is to try to sell you something (usually newsletter prescriptions). Reddit’s IAmA subreddit is a surprisingly good source for quality discussion around interesting topics precisely because there are no ulterior motives (aside from accumulating “karma”) and because its done anonymously, allowing practitioners to say and explain things without the limitations that hold back most practitioners in the financial industry.

I have selected some curated questions and responses from that,  in my opinion,  comprise some of the best quant/algorithmic trading/high frequency trading interviews on Reddit. There is some good content both for novices that are being introduced to algorithmic trading and HFT for the first time, as well as developers that may wish to attempt developing some basic automated trading systems during their spare time.

IAMA (was) HFT algo programmer for a major investment bank for 10 years and currently a pro trader. AMA

• What are the basic tricks of HFT?

HFTs do variety of things, but since their profit margin is very small, they need to be very high probability trades. Arbitrage is a big area for HFTs. When you hear about co-location and receiving quotes faster than the rest of the market, this is arbitrage. Essentially, they look for times when say AAPL is 224.25 in one market and 224.29 in another market and simultaneously buy the lower and sell the higher. When I say one market vs another this could be different exchanges, instruments (such as options), derivatives such as AAPL portion in S&P500. The second big area of HFTs is market making. If a stock is quoting bid/ask = $40/$40.20, this means a buy will be executed at $40.20 and a sell will be executed at $40. The HFT may add bid/ask = $40.10/$40.11 to the market. Note the price has improved for both buyer and seller. This increases the chances of trades taking place and the HFT profits 0.01.

• Is it true that it is virtually impossible to profit or even make a living by retail trading?

No, its very possible to profit, make a living or be a millionaire by retail trading but its very very hard, because everyone has picked bad habits during their early investing/trading days. Many of these are reinforced by TV and web media.
For example:

• buy and hold and everything will eventually be profitable.
• buy a winner all the way to the top
• buy (scale in) a loser till it becomes profitable.
• the bull/bear market will never end, keep buying

Paul Rotter makes about $60 million a year trading. Al Brooks makes a living trading the size of small institutions. Most of these are day traders. About 80% of traders will lose everything they have. 11% will be consistently be profitable. The rest will break even. You have to trade for a while to move to the top bracket, but it can be done. I trade for a living and know quite a few people who do.

IAmA Someone who worked for Investment Banks developing automated trading systems AMA

• If you’re capable of making a program that is obviously extremely profitable – why don’t you use it for personal stock trading and rake in money day trading or something?

It’s a team effort, the same reason a game developer at Sony doesn’t go and make their own games.

If you are using E-trade, you are just day trading. Fees per trade will be about $8. Using strategies like the ones described above involve many smallish trades. A common strategy is finding a relationship between two stocks – lets say DELL historically has been worth 1.3 times more per share than MSFT. Your strategy will then to be to buy or sell MSFT (or DELL) when one diverges from that multiplier. You could do this on E-trade, but the price may have moved by the time you enter in your transaction, and its pretty exhausting to sit there and calculate the multiplier constantly, and then trade based on it. So you need automated tools. There are some guys out there like Brokertech that will help you get set up for far smaller fees, but even then, to implement any strategy, I would say you want at least several hundred thousand to play with. To really do this properly though, you need access to historical data, real time market data, often level 2 market data, direct connections to exchanges, development time, etc, and all of these things are expensive. I do know some guys that have done “bedroom trading,” none have really run too far with it. Sometimes they are one-trick ponies and they find a reasonably profitable strategy that doesn’t work after time, or they just don’t have the scale (IE money) to diversify their strategies and consistently make enough money to justify the risk.

• You say that IB has contributed to global growth. Would you say the same thing of high speed automated trading? From what I understand of it, it seems like stealing. Am I missing something?

Why does it seem like stealing ?

Let me give an example with FX. With a currency pair say EURUSD you have a spread, the difference between the buy and the sell price (a useful tip if you’re going on holiday: you can easily tell how good a price you’re getting at a currency exchange is by looking at how big the spread is). The tighter the spread, the better the prices.

Supply and demands for currency is for practical purpose changing continuously (at this very second there are thousands of people who want to buy/sell USD). But imagine currency exchanges only updated every 500ms.
Now if you’re a market maker who wants to offer a price for trading you know that supply and demand has changed since that last 500ms update, and based upon historical data you might estimate that the price is going to have moved by 10pips so you take the price you’re willing to trade at and add 10pips to the spread. Because if you don’t you might find that when the next exchange price update comes out that your price is off the market by 10pips and you’ll get slammed by people doing arbitrage trading (because your price is so off market people can buy cheaply elsewhere and sell to you at profit with no risk to themselves).

Now imagine the exchange updated every 100ms, on average the price might only move 1pip every 100ms, so rather than adding 10pips to your spread the market-maker only needs to add 1pip to their spread, so now prices for currency trading are better for everyone!

By increasing the frequency of trading you reduce the risk taken on by the market-maker and thus allow them to offer better prices because they no-longer have to account for that risk. This isn’t just theoretical, as the update frequency of FX exchanges has increased over the last decade spreads have been been dropping steadily.

IAmA Algorithmic trading developer at a large Wall St bank. AMAA

• How do you make money and who are the losers?

The bid/ask spread. I can probably explain a bit more later, but when you hear IBM is trading at $82.50, you can neither buy nor sell it at 82.50 You can buy it for maybe 82.55 and sell it for 82.45, and a guy called the market maker keeps the spread. For this he has certain responsibilities, like making sure he will always buy your IBM shares, even when everyone is selling. Spreads used to be 10 or 20 cents wide, now they are about a penny wide. The massive volume has brought these spreads down, and out of market makers pockets. Other guys look for inefficiencies in pricing. Without getting too detailed, there are stocks out there that represent a group of 30 or 500 stocks. If you see that particular stock is not trading the price that the basket of stocks is supposed to be trading at, you buy one or sell the other, and have a theoretical profit, because they should always be the same price. Its a good and difficult question. there are so many participants in the market, who trade for very different reasons, its tough to say anyone wins or loses, but I think this is the most direct answer. For the record, I think Joe Schmoe, aka a retail trader benefits. Trading costs used to be very high.

• What’s the salary like?

It depends. For a programmer like myself, I would say 150-500k. A lot of it is very bonus based, where base salary is around 100k. I would say 90% of these firms are based in NY and Chicago, and in NY that doesn’t make you rich.
If you do something awesome for the firm though that makes them boatloads of money, there is nothing stopping you from getting wheelbarrows full of money at year end. For a quant, there is no limit really. Most quants make in the hundreds of thousands to tens of millions.

Subscribe to Curated Alpha via Email, RSS, or Twitter .