Algorithmic Trading System Architecture
Previously on this blog I have written about the conceptual architecture of an intelligent algorithmic trading system as well as the functional and non-functional requirements of a production algorithmic trading system. Since then I have designed a system architecture which I believe could satisfy those architectural requirements. In this post I will describe the architecture following the guidelines of the ISO/IEC/IEEE 42010 systems and software engineering architecture description standard. According to this standard an architecture description must:
- Contain multiple standardized architectural views (e.g. in UML) and
- Maintain traceability between design decisions and architectural requirements
Software architecture definition
There is still no consensus regarding what a system’s architecture is. In the context of this article, it is defined as the infrastructure within which application components which satisfy functional requirements can be specified, deployed, and executed. Functional requirements are the expected functions of the system and its components. Non-functional requirements are measures through which the quality of the system can be measured.
A system which fully satisfies its functional requirements may still fail to meet expectations if nonfunctional requirements are left unsatisfied. To illustrate this concept consider the following scenario: an algorithmic trading system which you have just bought / built makes excellent trading decisions, but is completely inoperable with the organizations risk management and accounting systems. Would this system meet your expectations?
Conceptual Architecture
A conceptual view describes high level concepts and mechanisms that exist in the system at the highest level of granularity. At this level, the algorithmic trading system follows an event driven architecture (EDA) broken up across four layers, and two architectural aspects. For each layer and aspect reference architectures and patterns are used. Architectural patterns are proven, generic structures for achieving specific requirements. Architectural aspects are cross-cutting concerns which span multiple components.
- Event driven architecture - an architecture which produces, detects, consumes, and reacts to events. Events include real time market movements, complex events or trends, and trading events e.g. submitting an order.

This diagram illustrates the conceptual architecture of the algorithmic trading system
Reference Architectures
To use an analogy, a reference architecture is similar to the blueprints for a load-bearing wall. This blue-print can be re-used for multiple building designs irrespective of what building is being built as it satisfies a set of commonly occurring requirements. Similarly, a reference architecture defines a template containing generic structures and mechanisms which can be used to construct a concrete software architecture that satisfies specific requirements. The architecture for the algorithmic trading system uses a space based architecture (SBA) and a model view controller (MVC) as references. Good practices such as the operational data store (ODS), the extract transform and load (ETL) pattern, and a data warehouse (DW) are also used.
- Model view controller - a pattern which separates the representation of information from the user's interaction with it.
- Space based architecture - specifies an infrastructure where loosely coupled processing units interact with one another through a shared associative memory called space (shown below).
- Space-based architectural conceptual view
- Model View Controller original image: http://goo.gl/TjJ28A
Structural View
The structural view of an architecture shows the components and sub-components of the algorithmic trading system. It also shows how these components are deployed onto physical infrastructure. The UML diagrams used in this view include component diagrams and deployment diagrams. Below is gallery of the deployment diagrams of the overall algorithmic trading system and the processing units in the SBA reference architecture, as well as related component diagrams for each one the layers.
- Algorithmic trading system high level deployment diagram
- SBA processing units deployment diagram
- Order processing layer component diagram
- Automated trader / event processing component diagram
- Data source and pre-processing layer component diagram
- MVC based user interface component diagram
Architectural Tactics
According to the software engineering institute an architectural tactic is a means of satisfying a quality requirement by manipulating some aspect of a quality attribute model through architectural design decisions. A simple example used in the algorithmic trading system architecture is 'manipulating' an operational data store (ODS) with a continuous querying component. This component would continuously analyse the ODS to identify and extract complex events. The following tactics are used in the architecture:
- The disruptor pattern in the event and order queues
- Shared memory for the event and order queues
- Continuous querying language (CQL) on the ODS
- Data filtering with the filter design pattern on incoming data
- Congestion avoidance algorithms on all incoming and outbound connections
- Active queue management (AQM) and explicit congestion notification
- Commodity computing resources with capacity for upgrade (scalable)
- Active redundancy for all single points of failure
- Indexation and optimized persistence structures in the ODS
- Schedule regular data backup and clean-up scripts for ODS
- Transaction histories on all databases
- Checksums for all orders to detect faults
- Annotate events with timestamps to skip 'stale' events
- Order validation rules e.g. maximum trade quantities
- Automated trader components use an in-memory database for analysis
- Two stage authentication for user interfaces connecting to the ATs
- Encryption on user interfaces and connections to the ATs
- Observer design pattern for the MVC to manage views
The above list are just a few design decisions I identified during the design of the architecture. It is not a complete list of tactics. As the system is being developed additional tactics should be employed across multiple levels of granularity to meet functional and non-functional requirements. Below are three diagrams describing the disruptor design pattern, filter design pattern, and the continuous querying component.
- Continuous Querying Component diagram
- Disruptor design pattern class diagram source: http://martinfowler.com/articles/lmax.html
- Filter design pattern class diagram
Behavioural View
This view of an architecture shows how the components and layers should interact with one another. This is useful when creating scenarios for testing architecture designs and for understanding the system from end-to-end. This view consists of sequence diagrams and activity diagrams. Activity diagrams showing the algorithmic trading system's internal process and how traders are supposed to interact with the algorithmic trading system are shown below.
- Algorithmic trader interaction
- End-to-end algorithmic trading process
Technologies and frameworks
The final step in designing a software architecture is to identify potential technologies and frameworks which could be used to realize the architecture. As a general principle it is better to leverage off of existing technologies, provided that they adequately satisfy both functional and nonfunctional requirements. A framework is a realized reference architecture e.g. JBoss is a framework which realizes the JEE reference architecture. The following technologies and frameworks are interesting and should be considered when implementing an algorithmic trading system:
- CUDA - NVidia has a number of products which support high performance computational finance modelling. One can achieve up to 50x performance improvements in running Monte Carlo simulations on the GPU instead of the CPU.
- Apache River - River is a tool-kit used to develop distributed systems. It has been used as a framework for building applications based on the SBA pattern
- Apache Hadoop - in the event that pervasive logging is a requirement, then the use of Hadoop offers an interesting solution to the big-data problem. Hadoop can be deployed in a clustered environment supporting CUDA technologies.
- AlgoTrader - an open source algorithmic trading platform. AlgoTrader could potentially be deployed in the place of the automated trader components.
- FIX Engine - a standalone application which supports the Financial Information Exchange (FIX) protocols including FIX, FAST, and FIXatdl.
Whilst not a technology or a framework, components should be built with an application programming interface (API) to improve interoperability of the system and its components.
Conclusion
The proposed architecture has been designed to satisfy very generic requirements identified for algorithmic trading systems. Generally speaking algorithmic trading systems are complicated by three factors which vary with each implementation:
- Dependencies on external enterprise and exchange systems
- Challenging nonfunctional requirements and
- Evolving architectural constraints
The proposed software architecture would therefore need to be adapted on a case-by-case basis in order to satisfy specific organizational and regulatory requirements, as well as to overcome regional constraints. The algorithmic trading system architecture should be seen as just a point of reference for individuals and organizations wanting to design their own algorithmic trading systems.
For a full copy and sources used please download a copy of my report. Thank you.
-
Great overview, and a good start on the architecture. Your conclusion was apt, and pointed out why algorithmic trading software systems require constant back-testing and tweaking to keep them relevant. Good read!
-
When the data from commodities or fixed income is inaccurate or slow in being received the models could have a hard time calculating especially in the space of a Black Swann event.
-
Thank you very much for this article. I've been thinking about AI in finance since the late 90s, and finally the technologies and APIs are commonly available. Your article and blog is a great help to make those first steps to making the dreams of earlier years come true. Thanks a lot and good luck in your further ventures!
-
Please keep me updated in your progress. I am very interested. Thank you
-
Thank you very much for sharing the post!
-
Thanks for another excellent post. The place else may
just anyone get that type of information in such a perfect way of writing?
I've a presentation next week, and I am on the look for such information. -
Thanks for sharing valuable details.
Comments