Testing The Engine

Nirav Shah, Global Head Capital Markets Group, aurionPro looks at the evolution of FIX testing procedures, and where they go from here.
Evolution of Testing
FIX testing and FIX engines have evolved together at a rapid pace over the last 10 years or so. Early FIX testing used to be done primarily using in-house tools on top of the FIX engine, having the ability to send and receive FIX messages. The main focus of such tools was more on FIX semantics rather than business use cases. Even validation used to be manual, and tools were customised to requirements, and would have a flavor of the tester itself. This, according to me, was Generation I of FIX testing.
Then it evolved and Generation II abstracted the FIX semantics and the focus moved onto testing the business use cases. I think as usage of FIX increased, rightly so, the testing focus also shifted towards ensuring business use cases were tested properly. The market started demanding scalability and flexibility.
Generation III mirrored the FIX usage explosion across multiple asset classes, and hence the focus moved to performance across multi-markets and multi-asset baskets. Suddenly trading latencies had moved from milliseconds to microseconds and focus was also on rigorously testing the performance of the FIX infrastructure.
The current generation is now focusing more towards automation, regression testing, and minimal effort in adding new products across multiple asset classes and moving towards the middle and back office. This is a challenging and interesting period where FIX testing and FIX products need to demonstrate the scalability to move to allocations, confirmations, etc. The process has so far tailed FIX evolution, but it also needs to catch up with the evolution of software testing processes. The current demand of the market is to be better, faster and flexible; able to meet the ever-growing landscape of new execution venues, new products, and new asset classes.
Current Testing Focus
The greatest concern for both sellside and buy-side has always been to ensure fast time to market with reliable quality. For a sell-side, having multiple clients sending orders, the focus would be to manage customer specific RO E. The automation of testing has to play a huge role going ahead as FIX evolves more into allocations and confirmations, and as the markets finally start moving from FIX 4.2 to FIX 4.4 or FIX 5.0. A major reason why firms have opted to delay upgrades is because regression testing is not automated.
As markets have grown faster and faster, the main challenge for a sell-side is ensuring good quality of FIX systems, so that there is no down time. We need to understand that trading is now truly global. A down-time of 10 minutes in 24 hours is just about acceptable, and therein lies the biggest challenge the industry is facing.
For a buy-side that connects to multiple brokers in multiple markets (and most of the time multiple brokers for the same execution venue), the system needs to be customisable pretty quickly, and to ensure that the same performance is obtained in terms of latency. Reference data is a major headache for a buy-side who is connecting to multiple brokers in multiple markets.
Why Third Party Test Now?
This is purely a strategic decision. Seeking third party testing tools and even outsourcing them gives the firm the required bandwidth to focus on the core business of trading, and more importantly saves costs and ensures high quality.
There are numerous examples of buy-side and sell-side firms struggling to keep control of costs on their FIX infrastructure due to a lack of FIX expertise and not using the right tools. It’s no longer a classical dilemma between “build or buy”; FIX testing is now moving to a domain of out-sourced testing and hosting.
Testing FIX infrastructure has rapidly evolved into a product space with new ideas, and a primary aim of increasing operational efficiency. We have seen a number of our clients, many of whom have been FIX compliant for years, still lacking a testing platform or framework. Many times, this has resulted in them being slow off the mark in acquiring new customers or launching new products, while smaller firms have been more dynamic and have invested in the right technology and tools.
The Legacy of FIX Variants
Having different versions of FIX, or even a non-standard implementation, is a fact of life which has to be dealt with. This is due to the lack of proper testing tools, and where the firms have opted to delay an upgrade and adopt nonstandard implementations.
A sell-side has to adapt its FIX ROE and systems to the needs of customers; and in times where margins are shrinking and money is made only when the customer starts trading, being able to on-board the customer as quickly as possible has led to different versions and non standard implementations. I think these are more business driven decisions, aimed at minimising the impact on FIX infrastructure. A robust and a truly integrated FIX testing tool would help to resolve many such cases, as the technology has to move at the same pace if not ahead of business needs.
The net effect is that FIX systems are no longer scalable to match performance levels, and a lot of cost and effort is now going into maintaining the status quo, thereby slowing down the upgrades or the ability to add required features. It’s no secret that a lot of firms are still at FIX 4.1 or FIX 4.2 and not many have gone to 4.4 or 5.0 precisely because of these reasons. A lot of the above has to do with a lack of a proper testing platform – really the fear of not being able to test the system completely is pushing a lot of firms towards non-standard implementations.
If the end-client for a sell-side firm is the end customer who needs tobe on-board quickly, then not much of this process can be done by the end-client. When a new customer is being brought on-board, it starts off a new relationship. The humaninteraction required in guiding the end customer for certification testing can and should never be removed. This inspires confidence in the endcustomer, that the sell-side firm really cares about his/her business. It also allows the sell-side firm to get a deep insight into its customer’s profile, which can in turn help the sell-side firm to offer more relevant products and expand the relationship. However, if there is a new feature being added or the buy-side OMS has changed or been upgraded, then the automation can be achieved in testing by the end-client, and it really needs to be there to achieve efficiency. A balance needs to be struck between automation and human interaction to achieve optimal efficiency and maximum business expansion.
Now, suppose a new algorithm or trading strategy is being launched by a firm, then the end-client is going to be the execution trader on the desk, and they need to be involved in the testing process. The current tools are very tightly coupled with FIX tags and the person testing it needs to know the FIX tags very well. Normally, the FIX expert would sit with the trader and test the system, thereby leading to a duplication of effort. There is a definite need for, and there is an evolution happening in the FIX Testing arena, to address this. The bottom line is that the FIX testing tools are evolving into FIX testing platforms that have the ability to address multiple users, including traders, FIX testers, and developers to derive the most value from them.