๐ Phase 2: Mathematics & Statistics for Data Science
๐ Topic 16: Bayesian Statistics โ Prior, Likelihood & Posterior
Bayesian Statistics is an important approach to statistical inference.
It provides a framework for updating our beliefs about an unknown quantity when new evidence becomes available.
The central idea is:
Start with prior information, observe new data, and update your belief to obtain a posterior distribution.
Bayesian methods are widely used in: Machine Learning, Classification, Medical diagnosis, Spam detection, Risk analysis, Recommendation systems, A/B testing, Natural Language Processing.
๐น 1. What Is Bayesian Statistics?
Suppose a company wants to determine whether a customer is likely to purchase a product.
Before seeing any new information, we may already have some historical knowledge about the customer's purchase probability.
Then we observe new information: Previous purchases, Website activity, Product views, Time spent on the website.
We can combine the previous information with the new evidence. This produces an updated belief. That is the basic idea of Bayesian Statistics.
๐น 2. Bayes' Theorem
Bayesian inference is based on Bayes' Theorem.
The simple form is:
P(A | B) = [P(B | A) ร P(A)] / P(B)
Where:
P(A | B) = Probability of A given B
P(B | A) = Probability of B given A
P(A) = Prior probability of A
P(B) = Probability of observing B
In Bayesian terminology:
Posterior โ Likelihood ร Prior
This is one of the most important relationships to remember.
๐น 3. Prior Probability
The prior represents our initial belief about a parameter or hypothesis before observing the new data.
For example: Suppose historical data shows that approximately 10% of customers purchase a particular product. Before analyzing today's customer behavior, we might use: Prior probability = 10%
The prior can come from: Historical data, Previous experiments, Domain knowledge, Earlier studies, Expert knowledge
๐น 4. Likelihood
The likelihood tells us how compatible the observed data is with a particular hypothesis or parameter value.
Suppose we observe that a customer: Visited the product page 10 times, Added the product to the cart, Returned to the website multiple times
We can ask:
How likely is this behavior if the customer is actually going to purchase?
This information contributes to the likelihood.
๐น 5. Posterior Probability
The posterior is our updated belief after considering the observed data.
In simple terms: Prior + Evidence โ Posterior
For example: Before observing new behavior: Purchase probability = 10%. After observing strong purchase-related behavior: Updated probability = 35%. The 35% represents our updated belief based on the evidence and prior information.
๐น 6. The Bayesian Process
Bayesian inference can be thought of as a cycle:
Step 1: Start with a Prior - What did we believe before seeing the new data?
Step 2: Collect Data - Observe new evidence.
Step 3: Calculate Likelihood - How compatible is the evidence with different possibilities?
Step 4: Update - Combine prior and likelihood.
Step 5: Obtain Posterior - The posterior becomes our updated belief.
๐น 7. Simple Example: Medical Testing