Correlation

Correlation and independence

It is a corollary of the Cauchy–Schwarz inequality that the absolute value of the Pearson correlation coefficient is not bigger than 1. Therefore, the value of a correlation coefficient ranges between −1 and +1. The correlation coefficient is +1 in the case of a perfect direct (increasing) linear relationship (correlation), −1 in the case of a perfect inverse (decreasing) linear relationship (anti-correlation),^[7] and some value in the open interval ${\displaystyle (-1,1)}$ in all other cases, indicating the degree of linear dependence between the variables. As it approaches zero there is less of a relationship (closer to uncorrelated). The closer the coefficient is to either −1 or 1, the stronger the correlation between the variables.

If the variables are independent, Pearson's correlation coefficient is 0. However, because the correlation coefficient detects only linear dependencies between two variables, the converse is not necessarily true. A correlation coefficient of 0 does not imply that the variables are independent^{[citation needed]}.

$${\displaystyle {\begin{aligned}X,Y{\text{ independent}}\quad &\Rightarrow \quad \rho _{X,Y}=0\quad (X,Y{\text{ uncorrelated}})\\\rho _{X,Y}=0\quad (X,Y{\text{ uncorrelated}})\quad &\nRightarrow \quad X,Y{\text{ independent}}\end{aligned}}}$$

For example, suppose the random variable ${\displaystyle X}$ is symmetrically distributed about zero, and ${\displaystyle Y=X^{2}}$. Then ${\displaystyle Y}$ is completely determined by ${\displaystyle X}$, so that ${\displaystyle X}$ and ${\displaystyle Y}$ are perfectly dependent, but their correlation is zero; they are uncorrelated. However, in the special case when ${\displaystyle X}$ and ${\displaystyle Y}$ are jointly normal, uncorrelatedness is equivalent to independence.

Even though uncorrelated data does not necessarily imply independence, one can check if random variables are independent if their mutual information is 0.

Sample correlation coefficient

Given a series of ${\displaystyle n}$ measurements of the pair ${\displaystyle (X_{i},Y_{i})}$ indexed by ${\displaystyle i=1,\ldots ,n}$, the sample correlation coefficient can be used to estimate the population Pearson correlation ${\displaystyle \rho _{X,Y}}$ between ${\displaystyle X}$ and ${\displaystyle Y}$. The sample correlation coefficient is defined as

${\displaystyle r_{xy}\quad {\overset {\underset {\mathrm {def} }{}}{=}}\quad {\frac {\sum \limits _{i=1}^{n}(x_{i}-{\bar {x}})(y_{i}-{\bar {y}})}{(n-1)s_{x}s_{y}}}={\frac {\sum \limits _{i=1}^{n}(x_{i}-{\bar {x}})(y_{i}-{\bar {y}})}{\sqrt {\sum \limits _{i=1}^{n}(x_{i}-{\bar {x}})^{2}\sum \limits _{i=1}^{n}(y_{i}-{\bar {y}})^{2}}}},}$

where ${\displaystyle {\overline {x}}}$ and ${\displaystyle {\overline {y}}}$ are the sample means of ${\displaystyle X}$ and ${\displaystyle Y}$, and ${\displaystyle s_{x}}$ and ${\displaystyle s_{y}}$ are the corrected sample standard deviations of ${\displaystyle X}$ and ${\displaystyle Y}$.

Equivalent expressions for ${\displaystyle r_{xy}}$ are

${\displaystyle {\begin{aligned}r_{xy}&={\frac {\sum x_{i}y_{i}-n{\bar {x}}{\bar {y}}}{ns'_{x}s'_{y}}}\\[5pt]&={\frac {n\sum x_{i}y_{i}-\sum x_{i}\sum y_{i}}{{\sqrt {n\sum x_{i}^{2}-(\sum x_{i})^{2}}}~{\sqrt {n\sum y_{i}^{2}-(\sum y_{i})^{2}}}}}.\end{aligned}}}$

where ${\displaystyle s'_{x}}$ and ${\displaystyle s'_{y}}$ are the uncorrected sample standard deviations of ${\displaystyle X}$ and ${\displaystyle Y}$.

If ${\displaystyle x}$ and ${\displaystyle y}$ are results of measurements that contain measurement error, the realistic limits on the correlation coefficient are not −1 to +1 but a smaller range.^[8] For the case of a linear model with a single independent variable, the coefficient of determination (R squared) is the square of ${\displaystyle r_{xy}}$, Pearson's product-moment coefficient.

Example

Consider the joint probability distribution of X and Y given in the table below.

${\displaystyle \mathrm {P} (X=x,Y=y)}$

y x	−1	0	1
0	0	⁠1/3⁠	0
1	⁠1/3⁠	0	⁠1/3⁠

For this joint distribution, the marginal distributions are:

${\displaystyle \mathrm {P} (X=x)={\begin{cases}{\frac {1}{3}}&\quad {\text{for }}x=0\\{\frac {2}{3}}&\quad {\text{for }}x=1\end{cases}}}$

${\displaystyle \mathrm {P} (Y=y)={\begin{cases}{\frac {1}{3}}&\quad {\text{for }}y=-1\\{\frac {1}{3}}&\quad {\text{for }}y=0\\{\frac {1}{3}}&\quad {\text{for }}y=1\end{cases}}}$

This yields the following expectations and variances:

${\displaystyle \mu _{X}={\frac {2}{3}}}$

${\displaystyle \mu _{Y}=0}$

${\displaystyle \sigma _{X}^{2}={\frac {2}{9}}}$

${\displaystyle \sigma _{Y}^{2}={\frac {2}{3}}}$

Therefore:

${\displaystyle {\begin{aligned}\rho _{X,Y}&={\frac {1}{\sigma _{X}\sigma _{Y}}}\mathrm {E} [(X-\mu _{X})(Y-\mu _{Y})]\\[5pt]&={\frac {1}{\sigma _{X}\sigma _{Y}}}\sum _{x,y}{(x-\mu _{X})(y-\mu _{Y})\mathrm {P} (X=x,Y=y)}\\[5pt]&={\frac {3{\sqrt {3}}}{2}}\left(\left(1-{\frac {2}{3}}\right)(-1-0){\frac {1}{3}}+\left(0-{\frac {2}{3}}\right)(0-0){\frac {1}{3}}+\left(1-{\frac {2}{3}}\right)(1-0){\frac {1}{3}}\right)=0.\end{aligned}}}$