Here’s a nice take on this topic from a blog I recently found: Roll em!
When back-testing futures trading systems we have to make a choice regarding the historical price series. This issue came up for me recently while building a portfolio variance limiter in Trading Blox Builder (TBB). Put aside the issue of whether this is a wise thing to do: I want to use it to illustrate how careful one must be to use the “right” back-adjusted series for back-testing.
First I will quickly review the three main historical prices one is likely to encounter (unadjusted, Panama and Ratio-Adjusted). Then I am going to give an example for each to show that, depending upon the calculation you want to perform, any one of the three could be the “right” one. My variance example will show that sometimes, you even have to use them in combination!
Definitions. I am going to refer to the closing price only. The following applies equally to all prices:
- Unadjusted Close: The actual close of a given contract on the date in question. In this context it refers specifically to the contract we would actually buy or sell (usually the most active) rather than some other month.
- Panama Close: The closing price of a past contract adjusted up or down by a fixed dollar amount to account for the price difference when rolling from that contract to a future contract.
- RAd (Ratio Adjusted) Close: The closing price of the contract adjusted up or down by a fixed multiplier to account for the price difference when rolling from a previous contract to a new contract.
Here’s a simple example:
- The unadjusted close for March 2011 crude was {90.54, 89.03, 87.48, 86.94, 86.71, 86.73} through 2011-02-10. Starting on that date, the unadjusted close for April Crude was {89.94, 89.13, 88.73, 87.57, 87.84}. If we roll positions on 2011-02-10, we have to pay up (89.94 – 86.73) = 3.21 per contract.
- To create a Panama adjusted closing price series we must add 3.21 to each of the March 2011 contract closes: {93.75, 92.24, 90.69, 90.15, 89.92, 89.94*, 89.13, 88.73, 87.57, 87.84} * 2011-02-10 roll date. We have now created a back-adjusted series without the “jump” caused by the contract roll. Notice that the Panama series preserves absolute price changes.
- 89.94 / 86.73 = 1.037. To create a ratio-adjusted contract we must multiply each of the March 2011 contract closes by 1.037 to give us : {93.8910, 92.3251, 90.7178, 90.1578, 89.9193, 89.94*, 89.13, 88.73, 87.57, 87.84} * 2011-02-10 roll date. Again, we have a back-adjusted series without the “jump” caused by the contract roll. Notice that the RAd series preserves the percent price changes.
Here’s the thing: the first closing price of each series is different, so which is the “right” close?
Let’s start simple with some rules of thumb:
- If your calculations involve absolute values (e.g. nominal position sizes) use the unadjusted close.
- If your calculations involve absolute differences in values across bars (e.g. $ profit or loss) use the Panama contracts.
- If your calculations involve multiplicative differences (e.g. % relationships) use RAd contracts.
Here’s an example for the first: You want to know your nominal exposure to crude. It is function of the absolute value of the contract, so you would use unadjusted close. Using our numbers from above, to estimate nominal exposure to oil on the first day of the series, we would use a price of $90.54 because that is what it would have been at the time. Imagine what would happen if you used a Panama contract value that was negative!
For the second, consider a statistic like true range. It involves an absolute difference from one bar to another. If we were to use unadjusted close, we would be right MOST of the time, but horribly wrong on the roll date. In this case the “right” data to use is the panama contract. In the present example, if we were to use the RAd contracts and were position-sizing based on ATR, our back-test would have the wrong position sizes.
As an example of the third, consider the standard deviation of closing price series, or more accurately, the standard deviation of log differences: LN(Pi / Pi-1). Clearly we must use the RAd contracts for this calculation because we are looking at daily percent change – only the RAd contract accurately preserves this relationship. Using a Panama contract could lead to wildly wrong estimates of standard deviation, and you cannot take logs of negative numbers. The back-calculated standard deviations would change every time there was a contract roll in the future!
For my final example I want to consider a situation where we need to combine two different contract types: covariance. Everyone is familiar with the standard portfolio variance estimate:
Since standard deviations (sigmai) and correlation (rhoi, j) are percentage or ratio properties of price, you should use ratio adjusted contracts to calculate them. Since the weights are functions of the actual dollar values of the positions at the time they were held, the correct basis for the weights would be the unadjusted close. So we would be using 2 different series in this calculation.
So, Question: “Which Closing Price?” Answer: “It Depends”. Load them all into your simulation so you have access to whichever you need.
EDIT: I came up with what I think is a better, more general Rule-of-Thumb:
Use the back-adjusted contracts that will forever give you the same results in your simulation as you would have got in real time, regardless of future contract rolls.
Edit 2: Fixed LaTeX implementation
Nice post. I always look at this problem from a ratio vs point-based adjustments angle, but indeed you could also require the unadjusted close…
Ideally, you could generate a price series containing all the information and feed that into your back-testing engine – I presume this is what you must have done and fed to TBB.
Even better, the back-testing engine would also have a rolling mechanism, which would let you feed all individual contracts with the possibility to implement and test more funky rolling mechanisms..
TBB does not allow this unfortunately (I have a feeling PowerST is more powerful is that domain – but too expensive for now to check it out..).
Thank you.
Bear with me here, but at first glance I'm not sure about the creation of an artificial time series as a baseline. Our main concern is data adjustment errors causing different trades than what would have been placed in reality. If we know that these trade descrepancies (adjustment errors) potentially get larger as the data series gets larger. Would it not be easier to just test a specific time period with unadjusted date and see if these trades match with the results of your adjusted data?
ex. Unadjusted contract data from 2000-2001 (or some period where the rolls were smooth.) Then take your adjusted data from 2000 to 2011 and see if the backtest shows the same trades during that 2000-2001 period.
Thanks again for your thoughts. Thad
Anon, thanks for your comment.
In answer to your question: it is all in the details. A simple way to figure out the answer for your specific situation is to test it in a spreadsheet.
Set up a price column or columns and populate with a test price series; you are not trying to simulate actual contracts with rolls, etc., you just wnat a long enough series of numbers for your calculations to work. Set up a "back-adjusted" price column or columns. Set the back-adjusted price equal to the unadjusted price x "ratio" + "offset". Start with "ratio" = 1 and "offset" = 0. Next calculate your indicators and follow them all the way through to the trading decision.
Now you are ready to see which is the correct type of contract to use. If you change "ratio" and your trading signals are unchanged, then you should be using ratio-adjusted contracts. If you change "offset" and your trading decisions are unaffected, then use Panama contracts. Note that even though intermediate results (e.g. indicator values) may change with ratio and / or offset, it is the trading decisions that matter.
Nice Post. Thanks for highlighting an issue that I'm afraid many seem to overlook or just dismiss. Unfortunately, I have been falling into the later. My previous conclusion was that there really is no perfect method.
However, correct me if I'm wrong, the "more correct" path would be to use ratio adjusted data for bollinger breakout type systems and use panama methods for ATR breakout systems. Don't use one for both. Now wondering if it matters for dual/triple moving average and donchian based systems? thanks again for a nice post.