🚀 Data Science Roadmap 2026

📘 Phase 2: Mathematics for Data Science

📖 Topic 8: Covariance and Correlation

Welcome back! 👋

In the previous lesson, you learned about Range, Percentiles, Quartiles, IQR, and the Five-Number Summary.

Now let's learn two extremely important concepts for understanding relationships between variables:

• Covariance

• Correlation

These concepts are used extensively in Exploratory Data Analysis (EDA), feature selection, machine learning, and statistical analysis.

🔹 1. Why Do We Need Covariance and Correlation?

Suppose you're analyzing student data:

• Hours Studied | Exam Score

• 2 | 50

• 4 | 60

• 6 | 70

• 8 | 80

• 10 | 90

You can observe that as study hours increase, exam scores also increase.

But how can we mathematically measure this relationship?

That's where covariance and correlation come in.

🔹 2. What is Covariance?

Covariance measures the direction in which two variables change together.

It tells us whether two variables tend to increase or decrease together.

Three possibilities:

• Positive Covariance: When one variable increases, the other tends to increase. X ↑ → Y ↑. Example: Study hours ↑ → Exam score ↑

• Negative Covariance: When one variable increases, the other tends to decrease. X ↑ → Y ↓. Example: Product price ↑ → Demand ↓

• Covariance Near Zero: There is little or no linear relationship between the variables. X ↑ → No consistent change in Y

🔹 3. Covariance Formula

For population data:

• Cov(X,Y) = Sum of (Xi - Mean X) ** (Yi - Mean Y) / N

Where:

• Xi = Individual X value

• Yi = Individual Y value

• Mean X = Mean of X

• Mean Y = Mean of Y

• N = Number of observations

The calculation essentially asks: When X is above or below its average, is Y also above or below its average?

🔹 4. Simple Covariance Example

Consider:

• X = [1, 2, 3]

• Y = [2, 4, 6]

Means:

• Mean(X) = 2

• Mean(Y) = 4

Now calculate deviations:

• X | X - Mean X | Y | Y - Mean Y | Product

• 1 | -1 | 2 | -2 | 2

• 2 | 0 | 4 | 0 | 0

• 3 | 1 | 6 | 2 | 2

Sum of products: 2 + 0 + 2 = 4

Population covariance: Cov(X,Y) = 4 / 3 = 1.33

So covariance is positive. That makes sense because Y increases whenever X increases.

🔹 5. The Problem with Covariance

• Covariance tells us the direction of a relationship, but its magnitude depends on the units of the variables.

• For example: Height in centimeters, Weight in kilograms

• Changing centimeters to meters can change the numerical value of covariance.

• Therefore, covariance isn't always easy to interpret or compare.

• This leads us to correlation.

🔹 6. What is Correlation? ⭐

• Correlation measures both the direction and strength of a linear relationship between two variables.

• Unlike covariance, correlation is standardized.

• Its value always lies between: -1 <= r <= 1

🔹 7. Interpreting Correlation

• r = +1: Perfect positive linear relationship. X ↑ → Y ↑

• r = -1: Perfect negative linear relationship. X ↑ → Y ↓

• r = 0: No linear relationship.

• Important: r = 0 does not necessarily mean there is no relationship at all. A strong nonlinear relationship can still exist.

🔹 8. Correlation Strength

A rough interpretation: