Complication and complexity: Need they be feared?

18 September 2019 / Dr Ewan Kirk


How can you distinguish complicated from complex systems? This piece explains why investors should not fear systems that are complicated, but should have justifiable concerns about systems that are complex.

Introduction

Systematic trading is often characterised as being a complicated and complex process. Indeed, a great deal of commentary by press and investors focuses on this perceived complexity. In some ineffable way, systematic trading being complicated indicates that there is something unsavoury or underhand about it. Complicated trading strategies seem to be responsible for market chaos, excess volatility, flash crashes (both down and up)… there is just something that appears to be "bad" about them.

Whilst we would naturally strongly disagree with the characterisation of systematic trading as "bad", in this post, we would like to take issue with the conflation of two terms. In the press and in the industry, the words complex and complicated are often used interchangeably but they are strictly not the same thing. By understanding the difference, we may have more insight into the nature of systematic trading and how it interacts with the financial ecosystem.

Complicated

How do we define something as complicated? If you are a dog, then an alarm clock or the concept of Norway is complicated and incomprehensible. If you've never seen American Football before, it seems complicated and incomprehensible. The Space Shuttle, an iPhone, and the ISDA credit default swap conventions are all complicated. However, they're not incomprehensible. Somebody understands them or at least understands a few of the components and how these components work together. If you repeat the same inputs into the Shuttle, your iPhone and credit default swaps, they behave the same way every time.

Is being complicated an undesirable property of a process or system? This mostly depends on what the system is intended to do. Making something unnecessarily complicated might not be a good idea. We could open cans of beans using computer-controlled laser beams or enter into a full legal multi-party debt contract when we buy a round of drinks at the pub but those would be pointlessly complicated compared to a can opener or informally assuming everybody buys a round eventually.

However, excessive simplicity might not be good either. Managing air traffic around an airport by having a person on top of a tall building with a pair of binoculars is simple compared to the current system of ADS-B, RADAR and a centralised air traffic control infrastructure. But does anybody want a non-complicated air traffic control?

In some cases complicated systems are a good thing or, even more often, they are a result of the requirements for the system. An example of this is telephony. When telephony was invented, users had a big box in their house with a microphone and speaker. This box was connected by a length of copper wire to a local exchange which was manned (or more precisely wo-manned) by operators. You pressed the receiver hook a few times to speak to the operator and told them who you wanted to speak to. The operator connected your line directly to somebody else's line using a patch cable with a 1/4 inch jack plug and there was a copper wire directly connecting your microphone to their speaker and a wire connecting your speaker to their microphone. Simple.

However, there were a number of problems

  • This wasn't very scalable. Imagine how many jack plug holes would have been needed for London or Manhattan.
  • As small local systems became connected to each other within a country and internationally, setting up calls could be exceptionally difficult and time consuming.
  • While a location was often wanted for a business call ("Hello, is that British Airways?"), for personal calls, people mostly wanted to call a person, not a location —"Hello, is that Brian's house? I'd like to speak to Brian".

Over the past 140 years, telephony has evolved into a very complicated system. Mechanical and then electronic direct dial exchanges, undersea cables, and satellites have connected together all the fixed line phones in the world. Digital signal processing techniques allowed operators to multiplex many conversations on a single wire or fibre.

In the early 1980s, mobile telephony took off and added another layer of complication to telephony. Identifying how to connect to a specific phone (which potentially could be in any location in the world), setting up the connection, negotiating protocols, and then billing are all extremely complicated.

Advances in telephony have required handsets and exchanges to become more and more complicated. A microphone, a speaker and a bell has evolved into the iPhone X — gigabytes of operating system and application code run by a microprocessor containing nearly 7 billion transistors. It is debatable whether or not any single person can completely understand an iPhone given how complicated it is. Is being complicated a problem in this case? For people who sometimes struggle with new features and interfaces, it may seem like a bad thing but, overall, the complication has added astonishingly rich functionality to telephones that is considered desirable and probably good.

These types of highly complicated systems have certain common characteristics

  • The complicated system can be subdivided into smaller less complicated parts.
  • This subdivision can be done almost indefinitely until the small component is easily understandable. For example, a single line of computer code or a single transistor.
  • When these simple functional "atoms" are combined, they work together in predictable ways.
  • Feedback between components is limited, predictable and often linear.
  • The reaction of the complicated system to identical inputs are identical and predictable.

Let's look at a mathematical example. Here's an equation which describes a curve:

$$ \left(\left(\frac x7\right)^2\sqrt{\frac{||x|-3|}{|x|-3}}+\left(\frac y3\right)^2\sqrt{\frac{\left|y+\frac{3\sqrt{33}}7\right|}{y+\frac{3\sqrt{33}}7}}-1 \right) $$$$ \qquad \qquad \left(\left|\frac x2\right|-\left(\frac{3\sqrt{33}-7}{112}\right)x^2-3+\sqrt{1-(||x|-2|-1)^2}-y \right) $$$$ \qquad \qquad \left(3\sqrt{\frac{|(|x|-1)(|x|-.75)|}{(1-|x|)(|x|-.75)}}-8|x|-y\right)\left(3|x|+.75\sqrt{\frac{|(|x|-.75)(|x|-.5)|}{(.75-|x|)(|x|-.5)}}-y \right) $$$$ \qquad \qquad \left(2.25\sqrt{\frac{(x-.5)(x+.5)}{(.5-x)(.5+x)}}-y \right) $$$$ \qquad \qquad \left(\frac{6\sqrt{10}}7+(1.5-.5|x|)\sqrt{\frac{||x|-1|}{|x|-1}} -\frac{6\sqrt{10}}{14}\sqrt{4-(|x|-1)^2}-y\right)=0 $$


There's no doubt that this is a pretty complicated equation. It is doubtful that anybody could look at this equation and work out what this equation describes. That being said, each functional "atom" is easy to understand (there's no doubt as to what $\sqrt{33}$ means) and the "atoms" combine in predictable ways (the $+$ operation is pretty well understood).

However, the thing it describes isn't very complicated. It's this:

Figure 1
The Batman Curve

The result of this complicated equation isn't actually that complicated. Now let's look at complex systems which, as we will see, are very different indeed.

Complex

Whilst there are formal definitions of complexity, we can understand them by looking at the complex systems all around us. Their components interact in unpredictable ways and they are difficult or impossible to forecast even though they are often easy to describe.

The classic example of a complex system is the weather. It is easy to describe the state of the weather at any one time but forecasting the weather is a notoriously difficult problem. Even with almost unlimited computing power, any forecast beyond 10 days is little better than using the historical average of temperature and precipitation for that time of year. Butterflies flapping their tiny hurricane-causing wings is a tired old cliché but does encapsulate a sensitivity of the weather to initial conditions.

Let's use a mathematical example and compare our complicated equation above to this simple equation

$$ z_{n+1} = z^{2}_{n} + c $$


How does this equation work? The numbers $z$ and $c$ are "complex". Sadly the word complex means something else here. These numbers are of the form $x + iy$ where $i$ is defined as $\sqrt{-1}$. One can think of these pairs of numbers as defining a point on a plane. For each value of $c$ (an $x,y$ pair on the plane) we start with $z_0=0 + i0$ and the next value $z_1$ is found from the equation. For the first iteration, it is obviously $c$. Now we use the equation again to get $z_2 = c^2 + c$.

As we continue this process indefinitely, there are two things that can happen to $z$

  • $z$ goes to infinity
  • $z$ doesn't go to infinity (or it is "bounded")

We can classify every $c$ as either one which leads to an infinite $z$ or one which leads to a bounded $z$. This seems like it should be simple. If $c$ is a normal positive real number, then this classification is easy: if $c$ > 0.25 or $c$ < -2.0, then the $z$ goes to infinity. If $c$ is between -2.0 and 0.25, then it does not go to infinity. Simple… but when we make the imaginary part of the number non zero and plot the points that are bounded in black, something very complex indeed happens.

Figure 2
The Mandelbrot Set

This sort of emergent complexity arising from relatively simple rules has been an area of active research for more than 40 years and has provided deep insight into dynamical systems.

What's this got to do with finance?

We've produced some pretty graphs but what, if anything, has this to do with systematic trading, finance or algorithms?

In our view, when investors, commentators and practitioners are discussing systematic trading, complicated and complex are used interchangeably… and incorrectly.

Some things in finance are complicated

Systematic trading is very complicated. Or, at least, it should be very complicated. Writing massive libraries of computer code, combining them into systems and then connecting these systems both to themselves and to market venues thousands of miles away is a very complicated task. This is not an endeavour where an Excel spreadsheet and a couple of VBA macros is going to work very well. Whilst a single model running on a single stock in a simple brokerage account may be able to work reliably in Excel, it is rare that a simple model running on one asset is an investable proposition.

As the number of assets, the number of models and the number of funds increases, a system necessarily becomes more complicated. Critically however, the individual components don't necessarily become more complicated. They remain understandable and predictable. Of course there is always the chance that a bug is introduced into the system or that individual components can interact in counter-intuitive ways but careful and rigorous regression testing before adding a new component, model or asset to the system can deal with these problems before they create issues in a live system.

There are many other examples of complicated things in finance. Simple European options were considered complicated and scary for many years. Only the cognoscenti could trade them and anybody who understood the "greeks" was thought of as a mega-brain geek. There was (and is) nothing complex about options, they were (and are) just a little complicated. Other examples include derivatives such as futures, interest rate swaps, CDOs and MBS.

Investors, journalists and even some practitioners find these complicated systems daunting and worrying. The term "Black Box" is used in a pejorative way to describe complicated systems. Some worry that complicated systems are causing market volatility in some way which they can't really explain.

It appears that there is something uniquely frightening about financial investments where the "complicated" part involves mathematics, statistics or computer code. Many private or public equity transactions may involve astonishingly complicated legal documents but for some reason this causes less consternation and fear than a few lines of basic mathematics.

Some things in finance are complex

In contrast to complicated things, complex things are very hard (and often impossible) to forecast. The first and foremost source of complexity in finance is the "market". Here we aren't talking about any specific market like the NYSE or the CME Eurodollar market but rather about an idealised market which could just as easily refer to high frequency trading in IBM on the NASDAQ to an open outcry market in 17th century Amsterdam.

What characteristics do these idealised markets generally have?

  • Multiple participants buying and selling at the same time.
  • Participants are generally heterogeneous. Some may be market makers, some may be hedgers, some may be speculators.
  • Each participant has a goal which is generally known only to them.
  • Some proportion of the market participants are uninformed or "noise" traders.
  • A single security is (generally) traded in a single location.
  • Arbitrage opportunities between identical securities are "rapidly" arbitraged away.

These simple characteristics generate extremely complex behaviour. To a very good approximation for the vast majority of participants in the market it is impossible to forecast the market dynamics over a few seconds, minutes, hours or days.

To give an indication of why it is unlikely that anybody can forecast a market in the short term, let's assume that a trader in the market can forecast just the direction of the S&P500 tomorrow. She doesn't know how much the move is, she just knows the direction of the move in the S&P500 tomorrow. If the prediction is up, she buys the S&P500 on the close of the previous day, holds for one day and then changes her position at the end of the following day on the basis of whether or not she predicts the market will go up or down on the following day.

If this super trader starts with one dollar on the 1st of January 2000 then by the end of 2018 she has something slightly over 100,448,964,442,877,088 dollars and will have had a Sharpe ratio of 14. The trader's wealth of 100 petadollars is approximately 200 times the total global wealth in the world. Since very few, if any, people have 18 year track records this good, we can conclude that even forecasting up or down is really really difficult.

This is a direct result of the complexity of market dynamics. Complex systems are difficult to forecast due to their extreme dependence on initial conditions and non-linear relationships between the components.

So is any trading or financial process doomed to failure? Does complexity ensure that the efficient market hypothesis is true? Slightly surprisingly, the answer to this is no.

Complexity requires complicated processes

For our final example, let's assume that we have a very poor quality forecast of whether or not a security will rise or fall over a year. The probability that this forecast is right is 51%. Making some simple assumptions about the nature of the security, our forecast would enable us to make about 0.25% annually. At its heart, this poor performance is a direct result of the near-complete randomness of markets and, this randomness is a direct result of the highly complex nature of markets in general. Obviously, the process to actually trade this single forecast isn't very complicated. The "system" makes the forecast on the 1st of January (up/down) and on the basis of this, you buy or sell. Then you ignore the market, take the position off on the 31st of December and see what your profit or loss is. Even the most ardent advocate of high quality systems and software in the financial world would agree that there is little need for complicated systems to trade this forecast.

However, for our forecasting technique to be useful, it will have to generate better returns than under one percent a year. An obvious way to do this is to apply the same forecasting technique to other assets. If we can make forecasts on 5000 uncorrelated stocks (this is a big IF) then the return rises to 17% with the same level of risk.

Leaving aside how unlikely it is that you can make 5000 uncorrelated forecasts, making use of imperfect forecasting techniques on many securities theoretically looks like a good idea. Indeed, this is often referred to as the Fundamental Law Of Active Management . If you have a certain degree of "skill" then you should apply that skill to as many things as possible.

The problem of managing one position is a lot easier than the problem of managing 5000. Firstly, of course, one has to collect all the data for each of the 5000 securities. Then one has to make the forecast on these 5000 securities. Then a decision has to be made about how much risk (or assets) should be assigned to each of the securities. This could be as simple as 1/5000 in each asset or could be a complicated optimisation procedure with multiple constraints and which will require a forecasted covariance matrix. After that, the resulting portfolio needs to be analysed to understand its risk characteristics and exposures to various risk factors (which themselves have to be defined and analysed). Once the 5000 trades have been done, things get a little less complicated, although there will be thousands of corporate actions like dividend payments, stock splits, bankruptcies and mergers to handle over the course of the year. If the securities are not all denominated in the domestic currency, then currency hedges and rebalances will have to executed frequently. Finally, on the 31st of December, all the trades and currency hedges will have to be unwound.

This whole process is complicated. So complicated that it requires a significant amount of "specialist" mathematical, statistical and financial quantitative knowledge. If the investment process is going to be robust then it also requires a significant amount of immensely complicated software and arrays of computers and storage to analyse, manage and retain all the information. This may run to millions of lines of code. Writing, testing, managing and deploying these millions of lines of code is also a complicated process requiring specialist knowledge and experience.

Conclusion

So why do we need all these complicated (and hard to explain) techniques and processes? We need complicated processes precisely because the markets and the securities which we trade are complex. They are difficult to forecast and difficult to understand in detail. The markets, the securities and the people who trade in these markets interact in complex non-linear ways. There are the impacts caused by news events arriving at random times which can have unpredictable (literally) effects on the prices of securities. All this complexity needs to be "factored out" from the securities to (hopefully) generate a reasonable forecast of how the security will behave over the next minute or the next year.

It is complicated. However, although there is no simple way to explain every single component part of a systematic trading system the outputs, characteristics and benefits of a complicated systematic trading system can be explained and understood precisely because it is not complex. At Cantab, we are very keen to explain what we do and how we do it to our investors ("a glass box, not a black box"). Necessarily, these explanations have omitted some of the more complicated details but the fundamental characteristics of the processes and how they are likely to add value to an investment portfolio are clear and easily understandable without extensive technical knowledge.

Investors often describe systematic investment processes as "complex" and worry that they can't understand "the complexity". This is a category error. Markets are complex. Investors suffer from exposure to complexity in both systematic and discretionary investments.

However, when investors describe systematic investment processes as complicated, then they are correct. This is not something to fear. In our view, investors should ignore how complicated the investment process is and concentrate on what the expected results of the process are. We believe that investors should also accept that, at some level, they will never understand a complicated systematic investment process. Not because they specifically aren't capable of that (many of our investors are very smart indeed) but because it's unlikely that any one person can understand it all. Whilst each individual component is fully explicable and systematic, the ability to understand every single part of a systematic investment process is probably beyond any one person.

Conversely, the press and broadcast media are full of glib simple explanations of the dynamics of highly complex markets. Our view is that investors would find it instructive to worry more about overly simple descriptions of unfathomable complexity than about investment processes that are complicated.


Source: GAM unless otherwise stated. The information in this document is given for information purposes only and does not qualify as investment advice. Opinions and assessments contained in this document may change and reflect the point of view of GAM in the current economic environment. No liability shall be accepted for the accuracy and completeness of the information. The mentioned financial instruments are provided for illustrative purposes only and shall not be considered as a direct offering, investment recommendation or investment advice. Reference to a security is not a recommendation to buy or sell that security. Past performance is not an indicator of future performance and current or future trends. There is no guarantee that forecasts will be realised. The examples are being provided for illustrative purposes only. The examples provided were selected to assist the reader in better understanding the various trading strategies presented. It does not represent actual performance and it is not a recommendation by the portfolio managers to buy or sell the investments mentioned in the examples provided.


Back to top