David Leinweber, author of Nerds on Wall Street: Math, Machines and Wired Markets, talks about the impact of big data on technology, trading, regulation and risk management.
David Leinweber heads the Lawrence Berkeley National Laboratory Computational Research Division’s Center for Innovative Financial Technology, which was created to help build a bridge between the computational science and financial markets communities.
His professional interests focus on modern information technologies in trading and investing. As the founder of two financial technology companies, and as a quantitative investment manager, he is a pioneer in the transformation of markets. At the RAND Corporation he directed research on real-time applications of artificial intelligence that led to the founding of Integrated Analytics Corporation (IAC). IAC was acquired by the Investment Technology Group; its product became QuantEx, an electronic execution system used for millions of institutional equity transactions daily.
He is a graduate of MIT in physics and computer science. He also has a Ph.D. in Applied Mathematics from Harvard.
FIXGlobal: How has big data changed the technology and hardware requirements for latency-sensitive trading strategies?
David Leinweber:Â Every system has a capacity. This is true for both networks and the many execution systems that make up our modern market systems. Coordination based on new rules places additional demands on that capacity.
We are talking about structured big data, getting too big and too fast for systems to handle. For unstructured big data, some forms – such as simple ‘one-note news’ – has also created a race in latency.
FG: Does big data affect short term, higher frequency strategies more or less than long term, fundamental strategies?
DL:Â In one sense, high frequency trading (HFT) is not possible without big data. However, longer term investors are affected as well: by the trading process and also by the need to keep up with the techniques required to understand unstructured big data when making investment decisions.
In a broader sense, investor confidence in markets is important. Big events like the Flash Crash tend to erode this confidence. Many traders maintain that there are thousands of smaller anomalies that underpin their confidence in the safety and stability of markets. So far we have been lucky and have avoided major cyber security problems. All this combines to undermine market confidence and many people feel this has contributed to reduced volumes.
FG: Are regulators better off attempting to curtail data volumes (Tobin Tax, excess messaging fees) or radically expanding their capacity for data analysis?
DL: A collection of individually stable systems may be unstable in aggregate. Our ability to realistically simulate and test this is not what it could – or should – be.
Even our ability to look back at what happened when things go wrong is limited by data collection from an earlier era. The heads of both the Commodity Futures Trading Commission (CFTC) and Securities and Exchange Commission (SEC) have pointed to this regarding analysis of the Flash Crash.
FG: Have the risk management strategies at large brokers and institutional investors changed to reflect the increased size of financial data? What changes should be made to intraday risk systems to account for the increased volume?
DL: I don’t have enough information on what has changed at the big firms to give a good answer. Systems may have improved but people have to use them properly. Transparency in markets (especially the much larger swaps markets), and what is traded on them are hot political issues (for example, the Swap Execution Facility (SEF) and Financial products Markup Language (FpML)).
When you say increased volume, I assume you mean volume of data, which is a direct consequence of technology. Trading volume has gone down.
FG: Who can ‘get away with’ not looking at the full data set? For low frequency firms, is the consolidated, summarized version sufficient?
DL:Â Higher turnover firms are most affected, but big issues of safety, security and stability of modern markets affect all investors.