I once was a subscriber to Bob Prechter’s Elliott Wave Theorist, though not for a couple of years. One of his themes at the time (and probably still) was “All the same market”. He commented on the way in which more and more assets were moving with greater correlation than in the past. In this post we perform a reality check on futures market correlation.

I have been exploring portfolio-level correlation filters for my trading system. This is not for risk-control per se, it is aimed at reducing portfolio volatility so I can increase cumulative average growth rate (see Ralph Vince’s “Leverage Space Trading Model”, pp50-54).

Correlations change over time, so I wanted to take a look to see how much and how fast so I can get a handle on how frequently I need to check correlations. This is some interesting data I found …

Correlation Coefficient Heatmaps

Each of the following 3 charts shows a heatmap of the Pearson correlation coefficient matrix of 40 futures contracts (although it includes both ES and SP) over a 1,000 bar period. The colors range from dark red (-1) to white (+1) with zero being orange. The histogram in the top right indicates the distribution of coefficients. The first chart covers 1999 – 2002, the second 2003 – 2006 and the third is 2007 – 2010. Each chart is roughly 3 years.

Baseline

In the first chart we see a lot of orange. The histogram is nicely centered and symmetrical about zero. There are some correlations you would expect (energies; meats; grains; Euro, Swiss and Pound; USD vs other currencies; etc), but generally these contracts show the low correlation for which futures are renowned.

Building

Now we are beginning to see stronger sector correlations, only the dollar is showing any real -ve correlations. The metals now stand out, and the currencies have formed an anti-dollar block. The dollar now shows significant negative correlation to almost half the portfolio. Only the softs remain as an uncorrelated group. The histogram at top right has become skewed to the right, though it is still centered on zero.

All the Same Market?

The final chart sees the evolution continue. It looks like the world of futures vs US$, Yen, the Note and the Bond! The histogram now has a left skew as the mode is solidly in positive territory. This seems like the perfect illustration of Prechter’s point.

Details

These data were derived from the “free” portfolio of futures provided by CSI (it still includes Bellies, which I hope they will soon replace). I constructed a ratio-adjusted continuous contract. I constructed a date-synchronized matrix of closing prices then removed any rows that are not “full” of data. I then calculated the log difference for each period (simple division of the matrix by a version of itself shifted by a row). Then I used R’s “cor” function which produces the 40 x 40 correlation matrix in two shakes of a lamb’s tail. I submitted the results to the “heatmap.2” function in the “gplots” package.

I also spot-checked the significance of a handful of the correlation coefficients (about 100 of them) bootstrapping the series to find a confidence interval and permutating the series to estimate p-values.

In the bootstrap approach I created 500 new series by randomly selecting paired measurements (with replacement) from the subject series. I then calculated the correlation coefficient of each synthetic series. I used the 5th and 95th percentiles of the correlation coefficients to create a confidence interval.

In the permutation approach I created 500 new series by taking one of the series and randomly re-ordering it then I paired it with an original series to create randomly paired data. I then calculated the correlation coefficients of each new series. I compared the distribution of the coefficients from the random series with the coefficient from the actual data to estimate the p-value of the original coefficient.

The majority of correlation coefficients with a magnitude of greater than .4 or so were significant.