# How to compute covariance and correlation coefficients (in Python, using pandas and NumPy)

See all solutions.

Covariance is a measure of how much two variables “change together.” It is positive when the variables tend to increase or decrease together, and negative when they upward motion of one variable is correlated with downward motion of the other. Correlation normalizes covariance to the interval $[-1,1]$.

## Solution

We will construct some random data here, but when applying this, you would use your own data, of course.

1
2
3
4
5
import pandas as pd
import numpy as np
df = pd.DataFrame(np.random.rand(10,5))
df.columns = [ 'col1','col2','col3','col4','col5' ]

col1 col2 col3 col4 col5
0 0.488293 0.151749 0.485939 0.278562 0.998647
1 0.405459 0.766983 0.915349 0.099784 0.518523
2 0.312085 0.498104 0.526030 0.745883 0.292882
3 0.313217 0.826840 0.254793 0.942009 0.456271
4 0.657147 0.024847 0.769884 0.140779 0.427270

If you have two pandas Series, you can compute the covariance of just those two variables. Note that every column in a DataFrame is a pandas series.

1
np.cov( df['col1'], df['col2'] )

1
2
array([[ 0.04524431, -0.02545402],
[-0.02545402,  0.12901528]])


You can also compare all of a DataFrame’s columns among one another, each as a separate variable.

1
df.cov()

col1 col2 col3 col4 col5
col1 0.045244 -0.025454 0.005095 -0.015552 -0.006827
col2 -0.025454 0.129015 0.009857 0.062661 -0.013753
col3 0.005095 0.009857 0.084701 -0.048114 0.014510
col4 -0.015552 0.062661 -0.048114 0.087198 -0.023934
col5 -0.006827 -0.013753 0.014510 -0.023934 0.057866

The Pearson correlation coefficient can be computed with np.corrcoef in place of np.cov.

1
np.corrcoef( df['col1'], df['col2'] )

1
2
array([[ 1.        , -0.33316075],
[-0.33316075,  1.        ]])


And pandas DataFrames have a built in method to do this for all numeric columns.

1
df.corr()

col1 col2 col3 col4 col5
col1 1.000000 -0.333161 0.082300 -0.247604 -0.133423
col2 -0.333161 1.000000 0.094296 0.590780 -0.159177
col3 0.082300 0.094296 1.000000 -0.559850 0.207259
col4 -0.247604 0.590780 -0.559850 1.000000 -0.336937
col5 -0.133423 -0.159177 0.207259 -0.336937 1.000000