In real life, many trading strategies based on things in the market don't work (or no longer work). This could be because they were taken away by arbitrage. At least as likely, though, it was a false discovery from the beginning.
Recently I have been asked the most question: Why did my signal get an excellent record in the past year during the war in 2021-2022, but in the backtesting, all of my products should have been completely lost? The answer is simple: Do you know when you will wake up precisely every day next year? The answer is always: No one could and no one can.
I also pointed out a few ways the backtesting tool was unreliable in 2017( Why I don't believe in Backtesting 2017 wrote ). Let's stay on the same topic and talk more about our experiences. Before that, let's talk about why most traders had been educated to believe in backtesting results.
I have got many questions about backtesting recently:
How traders have been educated to believe in backtesting results?
Most new traders are attracted by a beautiful profit curve result from backtesting at the beginning, even I did so from a youtube video from 500USD to 130000USD. That's why I have been working for the past eight years. However, I lost much in the first year because of those beautiful profit curve backtesting results. When I kept learning to code, I understood more about the structure of the whole eco-systems, and I stopped tracing a beautiful backtest result to promote my products.
At the beginning of this article, no one knew what would happen. Although I am one of the sellers on MQL, I understand how those sellers intend to promote their products. The only way to promote their product is by a backtesting result with a low-drawdown and high-profit return curve. That usually would be a nightmare for their clients.
Face the fact: No one knows if the product works or not in the future. Either backtesting or forward testing. Do not be mis-edutcated.
All is about trading structure:
Even if two accounts run simultaneously in the same trading environment (same funds, same server, same time start, same computer) running for a long time will have different results. I tried this five years ago. So how can we judge that we can have the same results in the future based on a single backtest record? Furthermore, if all users simultaneously have orders and prices in the same direction, will this be a good or bad thing? It must be a big fish. I'm sure a good author could have insight into the answer to this question and deal with this significant issue. The following will discuss why the backtest results cannot be trusted. I believe that many developers will not pay attention to the details.
If all users simultaneously have orders and prices in the same direction, will this be a good or bad thing?
What factors might affect the different results from backtesting and live?
The answer is Cycle.
A video to explain a little bit about cycle and drawdown. (start from 07:12 to 09:00 )
As I mentioned in my 2020 article. Market depth is always an inseparable part of the transaction, and the start and end of each transaction must involve > Sequence. For example, if a person has only one mouth, he cannot eat ten apples at once. No matter how fast he eats, he can only take one bite at a time. This is the meaning of market depth. When many buyers or sellers buy and sell simultaneously, their final transaction prices will not be the same. This situation is named Slippage.
Some factors that cannot be easily observed, such as network delays, broker's server disconnection, etc., are also severe problems, resulting in different rhythms of the entire cycle.
Some traders think the delay problem can be solved using a VPS closer to the broker's server. This is a big miseducation because the broker's internal server's quality is a critical factor affecting order execution. It means that you say a word in front of me, but my brain may not react immediately, so my distance from you is not very important. The situation is the same as the distance between the VPS and the dealer. This is called Latency.
The above factors: Sequence, Slippage, and Latency, will exist in the trading environment of Live trading, then even the real account cannot be synchronized, so the backtest record is even more unrealistic.