David Dight, Senior Software Architect at Liquid Capital, looks at free and open source software development over his 25 years using and developing solutions.
A brief history of FOSS; major developments and definitions
The term FOSS (free and open-source software) has only recently come into common use. Software made available free of charge has been variously referred to as just ‘free’ software, ‘public domain’ software and sometimes ‘freeware’. The 80’s and 90’s saw the proliferation of ‘try-before-you-buy’ software, also known as shareware (or even ‘crippleware’). The term Open Source software was used much later to distinguish it from ‘free’ software reflecting its licensing restrictions. Today we use FOSS to refer to non-commercial and generally free software which may have certain restrictions on copyright, non-commercial use and distribution. One great advantage of this set-up is that it is now used as a development model.
FOSS has been around longer than most people probably imagine, but it wasn’t really prominent until the 1980s. It was largely the domain of hobbyists and enthusiasts until the advent of Richard Stallman’s GNU project. Today there are close to 400 GNU software packages available. One of them, Stallman’s EMACS editor is still popular (though perhaps controversial) and in wide use.
The early 90’s saw the first release of Linus Torvald’s Linux which has now become the most widely used UNIX flavour today — especially in the finance sector. My first exposure to FOSS was with Slackware Linux (at that time released on floppy). Having already used costly SCO Xenix and then SCO Unix for some years, Linux offered the first free community supported UNIX, and our team jumped on it.
Why do organisations choose FOSS?
Lots of reasons; not least of which is it’s free! But just because something is free doesn’t mean it’s necessarily any good, so first let’s look at the other reasons for choosing FOSS.
If you are on the buy-side trade it’s likely you’ll need to talk FIX.
FIX has had a long (informal) link with FOSS. One of the earliest Open Source implementations, QuickFIX still enjoys popularity. It remains popular because of the critical mass of QuickFIX users out there, the low barriers to entry and the abundance of support – both community and commercial. More fundamentally, QuickFIX performs well and is easy to use. Competing with QuickFIX are many commercial FIX offerings and, to be sure, many of them are excellent in terms of the critical discriminators such as low latency and interoperability. So why does QuickFIX continue to capture so much of the market?
Here emerges one of the main reasons FOSS can be the right choice. In my view, it’s about software sovereignty. A typical medium to large buy-side organisation will generally have a team of highly skilled engineers with the capability and experience to develop pretty much anything asked of them. Give them a FOSS package and the right resources and they can modify, extend and enhance it to suit business requirements. In some cases, just having the source available allows developers to quickly fix bugs and get back in production after an outage. In other cases, the source can be instructive, providing insights into techniques and methodologies that influence future designs and implementations.
These are key distinguishing features of FOSS versus commercial offerings (aka closed source). You can’t just change a commercial piece to suit individual business needs and idiosyncrasies. It is usually extremely difficult to get a vendor to add the specific features you want. Even when they do agree to add your feature request it’s often expensive and may take some time, and it’s also unlikely to precisely match your requirements.
Another FOSS product, GNU GCC, is probably one of the most respected FOSS development tools. As a C/C++ compiler it still dominates the field. With GCC it isn’t so much that users want to extend or enhance it, but rather that they want to use a product that has a critical mass of engineers developing and continually improving it. Take the recent changes to C++ with C++11. GCC already supports most of the new language features and supporting libraries described in the standard. Clang (LLVM) has similar support. Releases are frequent and timely.
So, in summary, there is the appealing feature of low cost, the fact that direct access to the source code reduces your risk if you have the capability to take advantage of it, and access to frequent updates and improvements from a vibrant critical mass of developers.
As a developer why would you release on open source?
In my own experience, I started development on fix81 purely from an individual inspiration. I wanted complete control over every aspect of the FIX layer. I worked on it in my own time for more than a year and at the end of this period I thought, “fix8 has evolved into something that could really be of benefit to the FIX community.” I considered the prospect of commercialising and decided instead to release it as open source.
Something worth thinking about here is why so many companies and individual developers freely contribute to these projects. In my view, they are motivated by pure love and enthusiasm for the field. The benefits derived from this enthusiasm speak for themselves. I believe this is the main motivation for developers to release their work to the community – much like myself.
Why FOSS isn’t always the right choice
Of course, FOSS isn’t a panacea. Niche fields often suffer from the lack of available FOSS projects. Often those that are available are immature or just plain bad. Commercial products often fill this void. Sometimes these products will branch a FOSS version to keep the community active and involved.
A critical weakness can be the lack of technical support. Even with an active community there is often insufficient support, leaving organisations exposed and looking for costly alternatives quickly. In my experience though, these situations are infrequent. Redhat was one of the early leaders in developing a commercial support model which went a long way in addressing these problems.
Projects that use poorly researched and misunderstood FOSS components are also commonplace. Just as many organisations actively choose not to use FOSS, others seem to use it by default. Inevitably this can lead to the wrong choices being made. Once a system has been built on the top of a number of ill-suited FOSS pieces it can be difficult to unravel and choose alternate components.
Also, sometimes FOSS projects go cold, occasionally leaving an unsupported, unimproved user base to fend for themselves.
One final reason FOSS might not be chosen is cultural. Banks for example have traditionally relied on a combination of in-house and vendor solutions. Up until fairly recently, FOSS operating systems were not deployed as a part of their IT policy. Seeing how Linux has started replacing Sun/Solaris is a good example of how this has changed.
Uses of FOSS
Many companies embrace the field and use as much FOSS as is appropriate (and legal depending on license restrictions). Amazon2,3 is a well-known example. Facebook4 is another. Clearly some organisations have concerns about proprietary IP so will not contribute enhancements back to the community. More traditional organisations will develop in-house, use vendor solutions and selectively use established FOSS pieces (such as GCC). However, it seems to me that the trend is moving more and more to FOSS (especially Linux and GNU).
Linux is a good example of this trend towards FOSS. From specialised kernel modifications to entire Linux distributions devoted to specific purposes. Linux now powers mobile phones (e.g. Android), embedded systems, wireless routers and even media players (e.g. Western Digital’s WDTV). The world has embraced Linux like no OS before it. This is in stark contrast to the closed source OS model.
Redhat’s JBoss application server is also a great example of the potential of a complete FOSS stack, from OS to application.
What comes next?
We have seen many big organisations building systems on FOSS and as I see it, FOSS will continue to grow, with the finance sector increasingly becoming involved as users and contributors. The success of Linux will likely pave the way for more OS platforms and applications offered as FOSS. It is also likely that we will see more commercial offerings providing FOSS versions (for example Intel’s TBB 5).
My prediction is that FOSS will increasingly replace closed source and vendor supplied solutions. This seems especially likely as the trend to low volatility/low volume in the marketplace pressures companies to seek more cost effective ways to deliver software.6
In finance/IT, organisations considering the development of new systems are now more likely than ever to use FOSS from day one. From the Linux kernel, right the way through to application software, FOSS is finding its way into more organisations.
We can probably also view these changes in use and availability as a general trend towards more openness in toolsets, platforms and environments. Initiatives like the Loadstone Foundation demonstrate this trend.7 More organisations are viewing their own implementation as their primary IP. As this trend progresses I believe we will see more packages being offered with the FOSS model and more packages offered both as FOSS and commercially supported.
