How are correlation and cointegration related?

In what ways (and under what circumstances) are correlation and cointegration related, if at all? One difference is that one usually thinks of correlation in terms of returns and cointegration in terms of price. Another issue is the different measures of correlation (Pearson, Spearman, distance/Brownian) and cointegration (Engle/Granger and Phillips/Ouliaris).

-

So, my question is, does anyone know how to generate correlated prices? In order to generate correlated time series you should use copula approach. – user2623 Jun 30 '12 at 8:31
@Dr.Mike If you want to ask a question, you need to click the Ask Question link in the top-right corner and post it there. – chrisaycock♦ Jun 30 '12 at 12:52

6 Answers

This isn't really an answer, but it's too long to add as a comment.

I've always had a real problem with the correlation/covariance of price. To me, it means nothing. I realize that it gets used (abused) in many contexts, but I just don't get anything out of it (over time, price has to generally go up, go down, or go sideways, so aren't all prices "correlated"?).

On the flip side, correlation/covariance of returns makes sense. You're dealing with random series, not integrated random series.

For example, below is the code required to generate two price series that have correlated returns.

A typical plot is shown below. In general, when the red series goes up, the blue series is likely to go up. If you run this code over and over, you'll get a feel for "correlated returns".

```` library(MASS)   

 #The input data     
 numpoi <- 1000  #Number of points to generate     
 meax <- 0.0002  #Mean for x     
 stax <- 0.010   #Standard deviation for x     
 meay <- 0.0002  #Mean for y     
 stay <- 0.005   #Standard deviation for y     
 corxy <- 0.8    #Correlation coeficient for xy     

 #Build the covariance matrix and generate the correlated random results     
 (covmat <- matrix(c(stax^2, corxy*stax*stay, corxy*stax*stay, stay^2), nrow=2))     
 res <- mvrnorm(numpoi, c(meax, meay), covmat)     
 plot(res[,1], res[,2])     

 #Calculate the stats of res[] so they can be checked with the input data     
 mean(res[,1])     
 sd(res[,1])     
 mean(res[,2])     
 sd(res[,2])     
 cor(res[,1], res[,2])     

 #Plot the two price series that have correlated returns      
 plot(exp(cumsum(res[,1])), main="Two Price Series with Correlated Returns", ylab="Price", type="l", col="red")      
 lines(exp(cumsum(res[,2])), col="blue")     
````

If I try to generate correlated prices (not returns), I'm stumped. The only techniques that I am aware of deal with random normally distributed inputs, not integrated inputs.

So, my question is, does anyone know how to generate correlated prices?

I'm out of time, so I'll have to add my cointegration comments later.

Edit 1 (04/24/2011) ================================================

The above deals with the correlation of returns, but as implied in the original question, in the real world it looks like correlation of prices is a more important issue. After all, even if the returns are correlated, if the two price series drift apart over time, my pairs trade is going to screw me. That's where co-integration comes in.

When I look up "co-integration":

http://en.wikipedia.org/wiki/Cointegration

I get something like:

"....If two or more series are individually integrated (in the time series sense) but some linear combination of them has a lower order of integration, then the series are said to be cointegrated...."

What does that mean?

I need some code so I can screw around with things to make that definition meaningful. Here's my stab at a very simple version of co-integration. I'll use the same input data as in the code above.

````#The input data  
numpoi <- 1000    #Number of data points  
meax <- 0.0002    #Mean for x  
stax <- 0.0100    #Standard deviation for x  
meay <- 0.0002    #Mean for y  
stay <- 0.0050    #Standard deviation for y  
coex <- 0.0200    #Co-integration coefficient for x  
coey <- 0.0200    #Co-integration coefficient for y  

#Generate the noise terms for x and y  
ranx <- rnorm(numpoi, mean=meax, sd=stax) #White noise for x  
rany <- rnorm(numpoi, mean=meay, sd=stay) #White noise for y  

#Generate the co-integrated series x and y  
x <- numeric(numpoi)  
y <- numeric(numpoi)  
x[1] <- 0  
y[1] <- 0  
for (i in 2:numpoi) {  
  x[i] <- x[i-1] + (coex * (y[i-1] - x[i-1])) + ranx[i-1]  
  y[i] <- y[i-1] + (coey * (x[i-1] - y[i-1])) + rany[i-1]  
}  

#Plot x and y as prices  
ylim <- range(exp(x), exp(y))  
plot(exp(x), ylim=ylim, type="l", main=paste("Co-integrated Pair (coex=",coex,",  coey=",coey,")", sep=""), ylab="Price", col="red")  
lines(exp(y), col="blue")  
legend("bottomleft", c("exp(x)", "exp(y)"), lty=c(1, 1), col=c("red", "blue"), bg="white")  

#Calculate the correlation of the returns.  
#Notice that for reasonable coex and coey values,  
#the correlation of dx and dy is dominated by  
#the spurious correlation of ranx and rany  
dx <- diff(x)  
dy <- diff(y)  
plot(dx, dy)  
cor(dx, dy)  
cor(ranx, rany)  
````

Notice above, that the "co-integration term" for x and y shows up inside the "for loop":

````x[i] <- x[i-1] + (coex * (y[i-1] - x[i-1])) + ranx[i-1]  
y[i] <- y[i-1] + (coey * (x[i-1] - y[i-1])) + rany[i-1]  
````

A positive `coex` determines how fast `x` will try to reduce the spread with `y`. Likewise, a positive `coey` determines how fast `y` will try to reduce the spread with `x`. You can tweak these values to generate all sorts of plots to see how those co-integration terms `(y[i-1] - x[i-1])` and `(x[i-1] - y[i-1])` work.

After you've played with this a while, notice that it doesn't really answer the correlation of prices issue. It replaces it. So, am I now off-the-hook for the correlation of prices issue?

=========================================================

Obviously, now it's time to put the two concepts together to get a model that is in the ballpark with pairs trading. Below is the code:

````library(MASS)  

#The input data  
numpoi <- 1000    #Number of data points  
meax <- 0.0002    #Mean for x  
stax <- 0.0100    #Standard deviation for x  
meay <- 0.0002    #Mean for y  
stay <- 0.0050    #Standard deviation for y  
coex <- 0.0200    #Co-integration coefficient for x  
coey <- 0.0200    #Co-integration coefficient for y  
corxy <- 0.800    #Correlation coeficient for xy  

#Build the covariance matrix and generate the correlated random results  
(covmat <- matrix(c(stax^2, corxy*stax*stay, corxy*stax*stay, stay^2), nrow=2))  
res <- mvrnorm(numpoi, c(meax, meay), covmat)  

#Generate the co-integrated series x and y  
x <- numeric(numpoi)  
y <- numeric(numpoi)  
x[1] <- 0  
y[1] <- 0  
for (i in 2:numpoi) {  
  x[i] <- x[i-1] + (coex * (y[i-1] - x[i-1])) + res[i-1, 1]  
  y[i] <- y[i-1] + (coey * (x[i-1] - y[i-1])) + res[i-1, 2]  
}  

#Plot x and y as prices  
ylim <- range(exp(x), exp(y))  
plot(exp(x), ylim=ylim, type="l", main=paste("Co-integrated Pair with Correlated Returns (coex=",coex,",  coey=",coey,")", sep=""), ylab="Price", col="red")  
lines(exp(y), col="blue")  
legend("bottomleft", c("exp(x)", "exp(y)"), lty=c(1, 1), col=c("red", "blue"), bg="white")  

#Calculate the correlation of the returns.  
#Notice that for reasonable coex and coey values,  
#the correlation of dx and dy is dominated by  
#the correlation of res[,1] and res[,2]  
dx <- diff(x)  
dy <- diff(y)  
plot(dx, dy)  
cor(dx, dy)  
cor(res[, 1], res[, 2])  
````

You can play around with the parameters and generate all sorts of combinations. Notice that even though these series consistently reduce the spread, you can't predict how or when the spread will be reduced. That's just one reason why pairs-trading is so much fun. The bottom line is, to get in the ballpark with modeling pairs-trading, it requires both correlated returns and co-integration.

A typical example. Exxon (XOM) versus Chevron (CVX), where the above model applies if some additional terms are added.

http://finance.yahoo.com/q/bc?s=XOM&t=5y&l=on&z=l&q=l&c=cvx

So, to answer your question (as just my opinion), price correlation is typically used/abused as an attempt to deal with the longer term divergence/closeness of the paths of the series, when co-integration is what should be used. It is the co-integration terms that limit the drift between the series. Price correlation has no real meaning. Correlation of the returns of the series determine the short term similarity of the series.

I did this in a hurry, so if anyone sees an error, don't be afraid to point it out.

-

6
answer with code is so good.... – nicolas Jul 9 '11 at 11:54

Correlation is much more widely used concept and it has much more "informal" meanings. If we have only two random variables $X$ and $Y$ then correlation is simply a measure of linear dependence between the two variables:

$$corr(X,Y)=\frac{cov(X,Y)}{\sqrt{var(X)var(Y)}}=\frac{EXY-EX\cdot EY}{\sqrt{var(X)var(Y)}}$$

If correlation is -1 or 1 then the two variables are perfectly linearly related, i.e. there exists real numbers $a,b,c$ for which

$$P(aX+bY=c)=1$$

The correlation is called a measure of linear dependence since if we standartize $X$ and $Y$ (subtract the means and divide by standard deviations) then the correlation is the solution for the following

$$cor(X,Y)=argmin_a(E(Y-aX)^2)$$

So if $cor(X,Y)=0$ then you can say that there is no way to explain $Y$ using linear combination of $X$. For Gaussian random variables this have stronger implications, if correlation is zero, then the Gaussian random variables are independent.

When we have time series then we have more than two variables. Each time series is a sequence of random variable $\{X_t,t=1,2,...\}$. Naturally we can calculate correlation between any two time periods $corr(X_t,X_s)$. This gives us a lot of correlations for one time series, which are characterised by correlation function $r(t,s)=corr(X_t,X_s)$. Now if this function depends only on the difference $t-s$, i.e $r(t,s)=r(t-s)$ then the time series $X_t$ are called stationary (to be precise this is called weak stationarity and also requires that $EX_t$ should be constant, to be more precise the definition of stationarity actually involves the covariance not the correlation).

Now if we introduce another time series $Y_t$ we again can define a lot of correlations $corr(X_t,Y_s)$. And again we can define stationarity. Now for stationary series $(X_t,Y_t)$ the correlation $corr(X_t,Y_t)$ does not depend on $t$, so as in simple two random variables case we can talk about linear relationship between $X_t$ and $Y_t$. Note that in time series case we still have a lot of correlations left: $corr(X_t,Y_t+h)$, $h=...,-1,0,1,...$, which can be interpreted as a measure of linearity between past and future values of times series $X_t$ and $Y_t$.

And only now we can introduce the concept of cointegration. The stationary time-series are called integrated of order 0. If the difference of the time series is stationary then such time series are called integrated of order 1.

Integrated time-series are non-stationary, so for example $corr(X_t,Y_t)$ for integrated time series depend on $t$, which is not so nice. If for example we have highly correlated stationary processes knowing the fact that the correlation does not depend on time we can forecast one process with high accuracy knowing the values of the other. This does not hold for integrated time-series. Of course we can difference the integrated time series to get the stationary time series but if we difference them we can only investigate so call short-term dynamics, i.e. what happens now or in the near future (here we measure the time in numbers of time periods, short-term usualy mean 1-10 time periods).

Now finally we can introduce the definition of cointegration. The integrated (of order 1) time-series $X_t$ and $Y_t$ are called cointegrated if their linear combination $aX_t+bY_t$ is stationary. Since stationarity property remains if we multiply the time series by the constant we get that $Y_t-a/(-b)X_t$ is stationary. Which in turns gives us

$$Y_t=cX_t+\varepsilon_t$$

where $\varepsilon_t$ is a stationary time-series and $c$ is apropriate constant. This is again useful for predicting what happens to $Y_t$ if we know $X_t$ and $\varepsilon_t$ does not vary too much. Note that this relationship holds for all $t$ and $c$ can be estimated having the apropriate data since it does not depend on $t$.

The confusion between correlation and cointegration might arise from the fact that for stationary time series $Y_t$ and $X_t$ the exact same relationship holds:

$$Y_t=cX_t+\varepsilon_t$$

where $\varepsilon_t$ is a stationary process. Furthermore if we try to estimate $c$ using the data it can be shown that if we increase the number of data points indefinitely, the estimate for $c$ will converge to some meaningful number, which for example in case where $X_t$ and $Y_t$ are zero mean unit variance time series will be exactly the correlation $corr(X_t,Y_t)$ (which for stationary series is the constant, as discussed previously).

Note that this will not hold for cointegrated time series: although $c$ can be estimated, it in general will not be correlation $corr(X_t,Y_t)$, since $X_t$ and $Y_t$ are integrated (the special particular case is discussed in the link below). The story becomes worse for integrated time series, which are not cointegrated. Then the same estimate for $c$ which has nice properties and interpretations under stationarity and cointegration becomes meaningless. For more mathematical details what exactly happens in this case you can look at my post in stats.SE.

I hope this answer is useful. I intentionally sacrificed some mathematical strictness for better clarity, hopefully not too much.

-

Would you happen to know of a Khan-academy-esque video that shows derivation of the above? – Chloe Mar 6 '12 at 18:56
Derivation of exactly what? Nothing is derived in this post, only the definitions are given. As this post is purely my arrangement of known facts, I do not know about videos where somebody is talking about this. – mpiktas Mar 7 '12 at 2:01

From Quantitative Trading by Ernie Chan :

"Correlation between two price series actually refers to the correlations of their returns over some time horizon (for concreteness, let's say a day). If two stocks are positively correlated, there is a good chance that their prices will move in the same direction most days. However, having a positive correlation does not say anything about the long-term behavior of the two stocks. In particular, it doesn't guarantee that the stock prices will not grow farther and farther apart in the long run even if they do move in the same direction most days. However, if two stocks were cointegrated and remain so in the future, their prices (weighted appropriately) will be unlikely to diverge. Yet their daily (or weekly, or any other time horizon) returns may be quite uncorrelated."

-

4
Correlation between two price series does not actually refer to the correlations of their returns. You can calculate the correlation of two price series, but this is not how people tend to think of correlation between instruments and it's what I meant by "a stumbling block". I would agree that correlation between first differences of a series does not tell you anything about cointegration of their levels, but it doesn't help you understand how the two concepts relate to one another. – Joshua Ulrich Apr 22 '11 at 14:01

Before I try to answer your question we need to establish a difference between what one wants to analyse. It is true that before modern time-series methodologies were developed, researches used "correlation" between prices as a means of analysis. However, since a Price (at a specific moment in time) is 1 value, it makes no sense to compare 2 prices with each other using "correlation" (although there are some attempts: Robinson (2006) http://www.cemmap.ac.uk/wps/cwp107.pdf ) And as already has been pointed out before, most people above mention correlation in context of returns and such.

Most of the time we are interested not in the price but its movement! (i.e. it's a relative/dynamic concept which involves TIME).

In this "time-series" context, co-integration is the right tool to measure relationship between MOVEMENTS in prices.

Let me try to answer your question concretely by elaborating a bit on Johansen co-integration methodology as an illustration. The maximum likelihood estimation is in function of the deterministic term and the stationary effects. In other words we consider the multivariate linear regressions 1 and 2. With basic mathematic transformations it is easy to prove that the Johansen test statistics are directly linked to the angle (mean(t)) between the vectors of residuals in (1) and (2), ut and vt respectively. (If you care to read the full proof feel free to contact me)

In other words, the smaller the angle(t) between the two "error-vectors" ut and vt the more "connected" or integrated the price movements are between time-series 1 and 2. Which is quite intuitive actually...

Also, with some background knowledge, I think this illustration puts cointegration in contrast to correlation

I hope this was not too confusing (without the whole proof) and helped to answer your question to some degree.

-

Ow, I see as a new user I can not add images...  my apologies (will try to add it later when my Reputation score goes up I guess) – Val Jul 6 '11 at 15:47
• Correlation between two financial time series should be calculated as correlation of the returns (or log returns for prices).

• There is absolutely no relationship between correlation of the returns and cointegration. Two correlated time series can be cointegrated or not cointegrated. Two cointegrated time series can be correlated or not correlated.

Everything else is spurious :)

Edit: Sorry for this quick answer to an actually good question. They are definitely 2 different concepts. But that's true that they can be confused, because they seam to capture the same kind of things. For instance, one can do a portfolio allocation algorithm taking into account the correlation (VaR, CAPM), or the cointegration. In these examples, correlation and cointegration are 2 measures of risk/diversification. But they do not mean the same think. Taking only one of the 2 may lead to overestimate or underestimate the risk of the portfolio.

I would say that in this example, correlation is more shorter term (risk over one day, one week, ...), whereas cointegration is more longer term, it is the risk that different assets move together in the long run. I would say that a good algorithm should have the 2 approaches to estimate correctly the diversification. See http://www.carolalexander.org/publish/download/JournalArticles/PDFs/RIBF_16_65-90.pdf

Another example are markets that close at different time. If you look only at correlation of daily returns, you may find a weak correlation even though the markets move together.

-

1
As I said in my comment to @NYCBrit, "I would agree that correlation between first differences of a series does not tell you anything about cointegration of their levels, but it doesn't help you understand how the two concepts relate to one another." In short, this doesn't help answer the question. – Joshua Ulrich Apr 25 '11 at 13:53
• Correlation is a property of collections of observations.
• Cointegration is a property of time series.

The important difference is that temporal observations have one neighbour to their left and one to their right. Collections are like a set — no implicit "neighbour" relationships.

Moving average is an inappropriate statistic to apply to lab experiments or phone survey data. It is appropriate in the analysis of time series.

-

Interesting point to make. If we are comparing two time series, correlation tell us something about the complete time series as a whole, whereas cointegration tells us something about the individual matching points. – Gravitas Sep 18 '11 at 22:03
{"extraction_info": {"found_math": true, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 0, "math_alttext": 0, "mathml": 0, "mathjax_tag": 0, "mathjax_inline_tex": 53, "mathjax_display_tex": 5, "mathjax_asciimath": 0, "img_math": 0, "codecogs_latex": 0, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0}, "config": {"markdown_headings": false, "markdown_code": true, "boilerplate_config": {"ratio_threshold": 0.18, "absolute_threshold": 10, "end_threshold": 15, "enable": true}, "remove_buttons": true, "remove_image_figures": true, "remove_link_clusters": true, "table_config": {"min_rows": 2, "min_cols": 3, "format": "github"}, "remove_chinese": true, "remove_edit_buttons": true, "extract_latex": true}, "english_confidence": 0.9238115549087524, "perplexity_flag": "middle"}