Does backtesting really work?

Efe Dickson
0
824 Views
Jul 18, 2022

Backtesting is a technique for determining if a model's predictions match actual data. For example, backtesting a risk model often entails determining if actual historical losses on a portfolio differ significantly from the losses anticipated by the model. The model continually underestimates risk if real losses are larger than predicted. The model overestimates the danger if they are lower.

Does it actually work? Both large institutions and small retail traders utilize backtests as tools. Backtesting is a continuous process used by Jim Simons' Medallion Fund, the most successful hedge fund in the world, to create new strategies. Why? A trading strategy can be tested in the past.

There are clear guidelines for when to purchase and when to sell in a backtest. To put it another way, you can write the technique and determine exactly how it has performed in the past. As a result, this is a backtest using old data.

Naturally, this cannot predict the future, but it does indicate if the method has been successful or unsuccessful in the past. Anything that has done poorly in the past is unlikely to perform well in the future. Market cycles, however, influence how well or poorly an approach performs in various markets. This is the reality of trading.

On the other hand, if a backtest demonstrates that your concept has been successfully implemented in the past, it will likely perform better than any idea that has been unsuccessfully implemented. A successful backtest, however, does not always mean that it will be successful in the future. But in our opinion, it's the most accurate sign you can obtain.

This process is followed by a backtest:

1.Find a hypothesis that interests you.

2.Define measurable entrance and exit parameters that are clear and succinct.

3.The market you wish to test should be specified.

4.Set the time period that you wish to test.

5.The approach is coded.

6.Run the plan during the in-sample time.

7.Try the backtest with a sample.

All there is to it is this.

Before investing real money in the technique, paper trade it for a while if the backtest yields a favorable and encouraging outcome. You can save a ton of cash by doing this! In our trading courses, which are based on our 20 years of full-time trading and investing, we have written more about our processes.

Backtesting a trading strategy is effective because it allows you to verify or disprove a trading hypothesis; automate all of your trading based on the results; take advantage of the law of big numbers; reduce behavioral errors; and, in the end, speed up execution times. Backtesting is unquestionably not a time waster. This essay outlines what we believe to be the key justifications for why backtesting is effective, based on our daily backtesting for more than 20 years.

Who Makes Use of Backtesting?

Backtesting may be done by anybody; however, backtesting is often done by institutional investors and money managers. Backtesting makes use of data that might be costly to gather and necessitates complicated modeling.

Institutional traders and investment firms have the personnel and financial capital to incorporate backtesting models into their trading methods. Furthermore, because huge sums of money are at stake, institutional investors are frequently forced to backtest to gauge risk.

Because it eliminates emotions, backtesting a trading technique is effective.

There is ample proof that women make better investors than men, and that individual investors underperform average returns. Behavior errors are the major cause of this:

Investors frequently purchase after a significant decline and sell during a panic. Most of the time, you must take the exact opposite action. Such errors cannot be detected by a backtest, so you must adhere to the trading strategy.

You must trade less frequently than you'd like or prefer in order to stick to the trading strategy. The best approach to maintaining financial separation is to do this.

Women also do better because they put money aside, invest it, and then forget about it. They were not attempting to be brilliant! They are devoid of ego.

When your "intuition" advises you to sell or buy, you are more likely to ignore your system the more closely you follow the markets. Unfortunately, this intuition is frequently flat-out incorrect.

It's improbable that you'll succeed in overriding your systems and plans. How do you know that overruling works if you haven't backtested it?

In conclusion, what is backtesting outside of samples?

When you split your backtest into two sections, in sample and out of sample, you are doing out-of-sample backtesting. The rules, signals, and parameters are created during the in-sample test. You test your rules and signals on untested data in the out-of-sample setting.

Tests outside the sample are required, even if they are (obviously) not infallible. An out-of-sample test's incubation period, which lasts for many months and involves live trading on a demo account, is its last phase.

The goal of performing backtests is to make future predictions. You need to be cautious and persistent when doing this. Before you spend money on a strategy, we think an out-of-sample test is a crucial part of this process.