As long as you cannot predict an event, it is considered random. However, is it true? What makes something be really random? The absence of a specific pattern? The inability to predict what the next event will be?
Humans have an almost uncontrollable desire to find patterns in things. Our evolutionary process has guaranteed our brain as a fantastic tool to classify things: from recognizing the face of your mother from the time you are born to knowing who you can trust or not (!?)
Regardless of what we have learned to classify or not, the big point is that we are too bad at dealing with the lack of patterns. In the post "Predicting Rare Events and Financial Crises" we left open the question "how to calculate the probability of the occurrence of 17 heads in a row if this never happened in your sample?
Do you think throwing a die or shuffling a deck are random events? Maybe not. The act of rolling a die is ruled by specific mathematical laws and if we knew precisely the rolling force, the direction of the wind, the friction with the air, initial position of the die, the throwing angle, the point of friction with the surface and all the other variables involved, it would be absolutely possible to predict what the outcome of the launch would be. The same rational can be applied to a deck or trading. So, possibly, randomness does not exist - and not even the free will - since everything is the result of a combination of previous events, and changing any one of them, lead us to have a totally different answer.
Any change in the initial conditions of the die will influence the final result. Stopping to look at a bird can prevent you from being hit by a car a few minutes later or make you to meet the love of your life - the famous butterfly effect. If things are like this on a daily basis, why would they be different in the financial markets?
When looking at a chart of any asset, the first thing that happens is to fall into the temptation to find a pattern in the prices. As our brain is a beautiful classification machine, we can find several of them already in the first 5 minutes looking at the chart ... we still have the ones who say: if you look at the chart and in less than 1 minute you find nothing, it's because there's nothing to be done! In fact, it is very easy to look at an already drawn chart and find a lot of things that "seem to have happened since others have happened," or better saying, the famous patterns of the technical analysis. What is really difficult is to find these same patterns live, in real time, while the chart is forming. Here is where the "men are separated from the boys".
Everything that happened in the past chart was a consequence of past events and events that no one would even know would happen, others already partially expected and all the consequences of the others - remember the butterfly effect described above. Everything that will happen from this second will still be influenced by the speed of response to the events that are happening now, what other people are seeing on the same chart and the consequences of other possible infinite variables. Any change in price now influences the present so much that forecasting what will happen in the future is practically impossible, assuming that we do not know all the variables involved.
This discussion brings us closer to chaos theory, as defined in one sentence: systems considered to be dynamic are extremely sensitive to their initial conditions. A personal example: one of the reasons for the 2008 financial crisis was the near-zero US interest rate cut by the Fed Chairman Alan Greenspan, to contain a chain effect caused by the dot-com crisis, the internet business bubble. The crisis of 2008 led the job offers and wages in the US to a huge reduction, a fact that made me choose another place to do my internship. This "new" place was close to a mountain region where I went one day to ski for the first time and I broke my knee! In other words, would I have broken my knee because of the dot-com bubble?
Anyway, does randomness exist or does not? Like everything in life, it depends! By the point of view of the classical physics, no...while by the point of view of the quantum physics, yes. The subatomic universe is completely different from the Newtonian physics. With all this talk of randomness, classification and cause-and-effect relationships, we leave an open question for the next post: Is technical analysis a fallacy?
"I was looking at the markets and had an idea for a new investment method. As soon as I took an historical series of data and my favorite computer program I've started doing my "backtest"! A few hours later, I've said: "It is alive!", "It works!". Can not wait for my plans of wealth. Everything comes to my mind as I contemplate the beautiful return curve of my "backtest"!
Full of confidence, I'm ready for the real life! "
The beginning of this text reflects the daily lives of those whom try, somehow, beat the market. And at this time, the trader may come across three possible scenarios:
1 - Your new theory does not work after the "start";
2 - Your theory works for a while and then never again;
3 - Your theory is still running and your account increases every day.
After reading some works of Sir Karl Raimund Popper - one of the greatest philosophers about history of science - and a lot of time thinking, we say that a theory can be considered a theory if it has been tested and rejected or if it is not yet known if that is wrong .
One might already conclude immediately: a theory is never right? Yes, exactly. Let us recall here the Black Swan logic (book from Nicolas Taleb, a great work). A theory will never be certain because we will never know if all swans are white and we can not infer that all of them are because we did an historical analysis and we have never seen a black swan before. You are following the reasoning? So let us return to the three scenarios traders face after their backtest.
1 - Your new theory does not work soon after the start
This had happened because the historical data had no example of what just happened and it had bucked his new theory. The dynamics of the market, the information available and the relationship between supply and demand was changed right after he "put to the test" his new method. And this has a strong correlation with the following scenario...
2 - Your theory works for a while and then never again
Although it is very similar to scenario 1, the big difference is in the fact that the current situation of the market still follows the same backtest behavior, or a little worse, the theory is developed with overfiting - over-adjusted. So many customization were made to make the theory profitable that it is impossible to generalize any changes in the market. The backtest learnt everything that happened in the past that makes it have a blind vision of the future.
3 - Your theory is still running and your account increases every day
Finally, we might conclude that this is the winning scenario. The theory works! Yes, indeed, but only to the point where it will be disproved. And on this day, the system will fail and you may have devastating consequences. Confidence in the system and the confidence of the trader inflate as the account balance does. And a very full balloon burst more easily and more strongly that a balloon not so full. The difference between this scenario to the others is that the unlikely event has not happened yet.
So can we say that we have no escape?
Yes, we have. You can not look only at the historical result, but you should look on the positive expectation of returns. Your system will go wrong someday, for sure, and by "go wrong" I do not mean a few days of losses. I mean a sequence of losses, with no "apparent explanation." Let's say the system has only 0.1% chance of failure. That is, on average, only 1 in 1,000 times it will not work. First some points that must be understood:
That said, what should we pay attention? The maximum loss that we can have when the system goes wrong, plus the flexibility to update it. Let's focus on the first thing. Knowing the maximum loss will help the trader to make the calculation of the expected return. Using again Nassin Taleb, we quote the example given in his book "Fooled by Randomness". The market may have a 70% chance of going up and provide 1% gains. If it does not go (the other 30%), the losses may reach 10%. Considering that the trader has $1,000, the expectation of the trader in this case is negative by $230. Why? For every 10 times he wins $70.00 in total, but will lose $300, with a final balance of $230.
It may sounds interesting having a winning rate of 70%, but if the trader does not take into account what will happen in the remaining 30%, its might be a beautiful backtest, which began with a beautiful theory, but will not be worth anything.