Symbol, Symbol, On the Wall…

Ian Hoenisch of ITG lays out the perils of current test symbol regimes and describes the work to reduce the inherent risk in testing.
How has testing been done previously (i.e. ZVZZT) and what have been the drawbacks or risks?
The initial production testing was typically done by real ticker symbols that are out of the money, like ‘buy IBM for a Dollar’, which has some limitations and fat finger figure problems. If you accidentally sell for a dollar, your order is going to get filled, and this can create substantive risk. Some exchanges then implemented test symbols, like NASDAQ’s ZVZZT, but not all exchanges have agreed to use test symbols. Production testing is difficult at the best of times and as it is not just about getting a fill on a test symbol; all the other issues, such as getting market data to trigger risk limits, determining if you are 10% off last fill price to trigger warnings for bad fills, flagging trade-throughs, creating testing for locked or crossed markets, etc – all of these are difficult without a truly live environment.
What has emerged now is a parallel environment using real stock symbols – real market data (although it is probably delayed in most cases) you are not trading in a live environment. This factor has emerged not just on the exchange side but also in back office systems like Omgeo. The other element of test symbology that we are trying to work out is a true end-to-end solution and quotes, routing, executions, allocations and settlement. In the past, you could not use the Internet, so you had to get a whole new set of circuits to NASDAQ’s test system.
Technically speaking, how does the test symbol fit into the normal FIX message?
The FIX part was easy because it looks and behaves like a regular symbol. Getting everyone to understand that this is a test symbol is hard. For example, when we test in production with ZVZZT, there are many systems downstream from order routing that need to ignore test orders; compliance, dollar amount checks, OATS reporting, other SEC and FINRA reports that all need to exclude risk symbology. Buyside traders who want to test their systems often cannot create an order without initiation from the portfolio manager, thereby restricting them from creating a ZVZZT test order. Technically speaking, adapting the FIX message for test symbology is simple, but the surrounding elements will take more work to align.
What changes need to be made to broker systems to incorporate test symbols? At the exchange?
The buy-side will probably rely on their OMS vendors or alternatively they can create a dummy account with dummy cash that they do not have to report to accounting. While the buy-side can handle that more easily, on the sell-side, we are required to modify a number of systems to handle test symbology and exchanges have to adjust their systems to not report OATS and executions if they have a test symbol. Exchanges also pay for quotes on filled orders, as a part of a revenue-sharing deal for market data dissemination. This would be impossible with ZVZZT orders, else everyone would build their own matching system and send in endless test orders.
Everybody has to make the business logic changes so NASDAQ initially had 30 different test symbols beyond ZVZZT, and NYSE had TESTa and TESTe, so any time that the symbols changed, there was a considerable amount of code that had to be updated.
What are the most significant improvements made to the new test symbols?
Many firms have coded exception handling for testing or they set up parallel environments, so not all firms will perceive the need for change. The first incentive for adoption will be reducing the complexity of maintaining parallel systems and multiple code exceptions, and the second incentive is the reduction of risk resulting from testing in production because the alternatives are too difficult. Testing via live orders, for example, with new parameters on an algo order (to see whether it works) is dangerous.
Will this be a one-time system change or an ongoing process of changing new test symbols?
The smaller the number of test symbols you have the better. Typically, traders have created separate symbols for busts, busts and corrections, trade-throughs and partials, so you end up with scenario testing. The only way to do this was in an ATS that sought a match between a buyer and a seller at a crossed price. Testing with an algo, however, requires different symbols to exhibit different behaviors. There is a set of symbols that are needed and once you start working with a set of symbols, you have the equivalent of the MIC codes for standardizing a market center’s name. I envision that we will have a list of 20 or 30 test symbols with certain behaviors behind them and a method for adding new symbols. People will have to update their systems to include additional approved test symbols.
How would you characterize adoption of the new test symbols?
Although many people I speak to would identify test symbology as a difficult issue, it is not high on their priority lists as it is not a revenue generator. It is hard to calculate the cost savings involved or to assign a value to it, in order to encourage its use.
The risk management systems that exist now are: value at risk, global risk management, hedging and buying power among others. The problem for those responsible for testing systems is how do you quantify the effect of accidentally buying IBM when the trade was supposed to be sent to a QA system and they buy it for real? There is an incentive for firms to simply trade out of it and accept a loss?
Part of the problem is that nobody is calculating how many errors occur and the problem is that in the past there has been nowhere to talk about test symbology. If we are able to gather enough data through this working group, we can say to the industry, “This is a $10 million problem, but with focus we can drive this down to zero.” If we have the data, we could make the case that this is an opportunity not to be missed.
What’s next?
Our next goal is to work with local market participants to implement this in as many exchanges, regions and asset classes as we can and also as large a life cycle of the trade as possible. Asset classes are easy to understand, but the life cycle must include quotes, routing, executions, allocations, settlement, fails and reporting: you have to be able to test this all the way through.
It is important for people to remember that this is not a technology problem, it is a business problem. The software at the exchanges can make the same exceptions and deal with the test symbol lists and that is a ‘one time’ problem to solve. The business problem is the one that is harder to solve – actually showing the value of doing this in reduced risks, in reduced trade errors, in customer satisfaction, in reduced test cycles – using the same system instead of wiring up a different system. These issues are harder to quantify to make the business case. Although we all somehow feel the value, we do not know the value.