Issues Related to Back Testing
Back testing a trading strategy is the process of evaluating the profitability of a methodology based on returns calculated from historical data. Many portfolio managers use back testing to prove that a strategy has worked in the past, but they fail to evaluate a number of issues that are inherent in analyzing historical returns.
Back testing can be a way to develop a successful trading strategy, but there are a number of pitfalls, which include: curve fitting, determining liquidity and including commissions. There are a number of pitfalls to back testing that have given quants a false sense of security and eventually led to significant losses.
Any back tested strategy can produce robust results based on the time horizon used to back test and the criteria used by the investment strategy. By refining the criteria on a specific period of historical data, the results can become skewed and produce returns that can never work again.
An investor should be concerned with the liquidity of a security and the ability to transact at the prices that are used for a back test. Without properly compensating for the liquidity of prices, a back test can produce beneficial results that cannot be reproduced in the future.
An issue that needs to be addressed when evaluating a specific trading investment strategy is determining if the strategy will work in the future. Unfortunately, back testing is a method for finding a strategy that might have worked in the past but might never work again. , Back testing finds the best strategy that worked in the past. A strategy that has too many criteria can work when evaluating a specific historical period, but that strategy might fail to work when the trading environment changes.
For example, a strategy that works when a market is trending might fail if the market consolidates for a number of years. Criteria that are fit to a specific period (known as fitting the curve), may not be able to produce robust results when forward tested on live data .
Two other considerations that need to be evaluated when analyzing into a profit-and-loss calculation of a back tested strategy are slippage and commissions.
Slippage is the difference between the estimated transaction cost and the amount actually paid for a specific security. Slippage reduces estimated profits and increases losses, and therefore, needs to be integrated into the estimated returns of a back tested strategy.
Commissions are payments to a broker for transacting and clearing trades, and they need to be accounted for when evaluating back tested returns.
Back testing should also take place over various time frames, to avoid results that only work over a small sample period. Investors should understand that historical results on a trading strategy are not a guarantee that a strategy will work in the future.
In the simplest of terms, a back test is a method of choice for creating a strategy that has worked in the past and might or might not working in the future. There are a number of methods that compensate for these issues including simulations which avoid using historical data to back test a strategy.
