🚀 Data Science Roadmap 2026

📘 Phase 2: Mathematics & Statistics for Data Science

📖 Topic 13: Law of Large Numbers (LLN)

The Law of Large Numbers is a fundamental concept in probability and statistics.

As the number of observations increases, the sample average tends to get closer to the true population average, provided the observations satisfy appropriate conditions.

This is why collecting more representative data makes estimates more reliable.

🔹 1. What Is LLN?

P(Heads) = 0.5 for a fair coin

• 10 tosses: 7 Heads → 7/10 = 0.70

• 100 tosses: 54 Heads → 54/100 = 0.54

• 10,000 tosses: Proportion → ∼0.50

More trials → observed average approaches expected value.

🔹 2. Simple Example

True avg weight = 70 kg

• Sample 5 → 74 kg

• Sample 50 → 71 kg

• Sample 500 → 70.3 kg

• Sample 5,000 → 70.05 kg

🔹 3. LLN Does NOT Mean Perfect

LLN does NOT mean every large sample = exact population mean. It means convergence, not guaranteed equality. Mean might be 99.8 instead of 100, but close.

🔹 4. LLN and Probability

If P(Success) = 0.20

• 10 trials → 30% observed

• Many trials → tends to 20%

🔹 5. Two Main Versions

1) Weak LLN: Sample average converges in probability. The probability of being far from true mean becomes very small.

2) Strong LLN: Sample average converges almost surely, with probability 1.

For Data Science, focus on the core idea.

🔹 6. LLN vs CLT - Very Important

LLN → Accuracy

Where does sample mean go? → Toward population mean μ.

CLT → Distribution

What does distribution of sample means look like? → Approximately Normal.

🔹 7. Casino & Gambler's Fallacy

LLN does NOT mean: "If you lost, you must win next."

After H,H,H,H,H → P(Tails) next is still 0.5.

LLN is about long-run averages, not next trial.

🔹 8. LLN in Data Science

• Averages: Avg revenue, spending, delivery time - more data = more stable

• Conversion Rate: 10 visitors → 20% is noisy. 100,000 visitors → stable

• A/B Testing: Needs adequate sample size

• ML: Tiny eval sets = unstable metrics. Larger sets = reliable

🔹 9. LLN Does NOT Fix Bias

More data is NOT automatically better data.

If you survey only an expensive private club to estimate city income, even 1M samples = biased.

Large + Biased = Biased Estimate

Large + Representative = Reliable

🔹 10. Python Demo

import numpy as np
import matplotlib.pyplot as plt

np.random.seed(42)
tosses = np.random.choice([0, 1], size=10000)
running_average = np.cumsum(tosses) / np.arange(1, len(tosses) + 1)

plt.plot(running_average)
plt.axhline(0.5, linestyle="--")
plt.xlabel("Number of Tosses")
plt.ylabel("Proportion of Heads")
plt.title("Law of Large Numbers")
plt.show()

🔹 11. Common Mistakes

❌ Large sample = exact value → No, it tends toward it

❌ LLN guarantees next outcome → No, long-run only

❌ More data removes bias → No

❌ LLN = CLT → No

❌ Small samples useless → No, just more uncertain

🔹 12. Interview Answer

The Law of Large Numbers states that, under suitable conditions, as independent observations increase, the sample average converges toward the population expected value. It explains why larger representative samples give more stable estimates.

🎯 Key Takeaways

✅ LLN = long-run convergence of average to E

✅ More representative obs = more stable

✅ Does not predict next outcome

✅ Does not remove bias - representativeness matters

✅ LLN → Convergence, CLT → Normality[X]

🎯 Double Tap ❤️ For More