John Bates of Progress explains how complex event processing works and how it can simplify the use of algorithms for finding and capturing trading opportunities.
A brief summary of Complex Event Processing
Complex Event Processing (CEP) is about treating actions that happen all the time as specific events, which describe the action, and then being able to analyze those events as they are streaming through a system, while looking through them for patterns that create opportunities or threats. In the trading world, this means things like trading opportunities, such as monitoring a set of instruments across multiple trading venues and looking for particular patterns. Those patterns might be high frequency trading (HFT), statistical arbitrage, correlation relationship between two items, or even execution algorithms that are slicing orders based on some predefined metric.
The threats often focus around pre-trade risk. For example, will placing the trade exceed predefined risk levels, or run into potentially abusive trades, like a wash trade. CEP is about being able to monitor business in real-time to analyze what is happening now and, based on that, to try to predict what is about to happen and act on it immediately.
The value of Complex Event Processing
The world of trading is so fast moving. Research done by the AITE Group suggests that the average lifespan of a trading algorithm can be as short as three months. This is because new trading patterns are constantly coming to light and ones that might have been very successful might no longer be available as the markets become more efficient. In the old days, trading algorithms were like a cottage industry, in much the same way as the making of muskets used to be. Highly paid and highly skilled craftsmen would handcraft the algorithm. It was the domain of the very rich and not very many could be involved in the game.
With the advent of CEP technologies in the last ten years, now anyone can find patterns in fast-flowing data feeds, but more importantly, CEP provides the tools for business people to describe new algorithms quickly. This means that traders can keep up with a trading world that is moving ever faster, and which the handmade craftsmen struggle to keep up with. Suddenly, it has become easier for smaller firms to create algorithms to compete with the larger ones. There has been a revolution in software for the trading space, in that firms of all sizes now have access to the technology that was previously available only to Tier 1 banks.
Peeking under the bonnet
In a CEP platform, there is an engine which has the tools that allow you to model and visualize new strategies as they are running, as well as see any opportunities or threats. On top of this is an adaptive layer, with connectors to convey different formats of events in and out of the processing engine, taking in market data and sending out trades. CEP platforms can work off a simple consolidated feed, but organizations find that it is better to connect to trading venues directly because it reduces the latency and things can be seen as they happen.
What we tend to find is that most customers use FIX to connect to trading venues, so there are hundreds of variants of adapters which talk to different protocols, often over FIX. In the FX world, a trader might connect directly to a bank using their FIX interface; a trader might connect directly to a trading venue, which is often over FIX; of course, they mightalso connect to their own internal systems, databases, middle-ware buses, etc. A trader can plug into whatever source of information they have and even fuse those together. For example, connecting to a news feed and a stock feed, and looking for patterns where the news affects the pattern of the stock prices.
Portable strategies
One of the powers of CEP is to extract away from the specifics of the data sources. For example, a trader might build a strategy that tracks when any news article comes up about any particular stock in their portfolio, followed by a rise or a fall of greater than 5% in the value of that stock within five minutes. This involves temporal, logical, multiple streams, pattern matching and correlation. The trader could then plug into any source of news and any source from a trading venue and that same algorithm would work because the abstraction is dealing with market data, news and order placement and is not hard-wired into one trading venue’s interface.
This helps firms apply algorithms in different trading venues, contexts and geographies, which is why CEP has developed a market so quickly around the world. Traders can take the generic framework and add their own intellectual property into the algorithm, but it is using all those capabilities built up over ten years of development.
Typical use cases
There has been an evolution of the types of user, use case and buyer. CEP started in the world of the broker and the use case then was ‘we want to be able to build new algorithms for the buy-side more quickly and we just cannot keep up’. Within those organizations, different groups including equities, futures options,FX, prime brokerage and proprietary trading adopted it and over time it spread across asset classes. It then developed in the buy-side, particularly in hedge funds. Although hedge funds adopted CEP for their proprietary trading purposes, it has not been adopted by traditional asset managers, who only use it through their brokers.
The use case has also evolved in the last couple of years from trading to real-time risk management and market surveillance. New groups within banks are taking up CEP technology, such as risk, surveillance and compliance departments. The regulators are also adopting this technology as well as the trading venues themselves. It is important for the regulators to move from a world where they have either no visibility or visibility in the rear view mirror. It is important that regulators move to a more proactive, predictive world where they are able to monitor what is going on.
Predatory reputation
All forms of trading are intended to make money, but are these predatory? Certainly, intra-day trading, what might be called high frequency, is aggressive by nature because it moves in and out of patterns quickly, but that can be good for the market. This is the way things are evolving; we cannot turn it off without going back to the Stone Age. On the other hand, if traders are using illegal practices such as quote-stuffing, in which they try to ‘gum up’ the markets on purpose to distract others while they make a profit, is clearly wrong.
But this is where real-time surveillance can help to catch those people.It would be like equipping the regulators for high speed pursuit so they can keep up with the traders’ Ferraris, instead of having them currently riding bicycles. Better monitoring is required, but it is inappropriate to view everybody as predatory. There are a few ‘bad apples’ out there who need to be clamped down on and this is where CEP technology can help. The Commodity Futures Trading Commission is discussing the possibility of inspecting algorithms but it would take huge manpower. Why not have a free market, monitor it closely, discover the problems as they are happening and then crack down on them?
Secrets Revealed: Trading Tools Uncover Hidden Opportunities
