Linear regression is the simplest supervised ML model that finds relationships between features and labels.
Mathematically it looks like:
y'=b+w1*x1 + w2*x2 + ... + wn*xn
where
- y' - predicted value
- b - bias (calculated during training)
- wn - weight for a feature (calculated during training)
- xn - feature value (input to the model)
Loss for that type of model is usually calculated as a mean squared error(MSE) or mean absolute error (MAE):
- MSE is sensitive to outliers and adjusts the model toward them.
- MAE minimizes the absolute differences, making it less sensitive to outliers.
Training steps:
1. Calculate the loss with the current weight and bias.
2. Determine the direction to move the weights and bias that reduce loss.
3. Move the weight and bias values a small amount in the direction that reduces loss.
4. Return to step one and repeat the process until the model can't reduce the loss any further.
Example:
The model needs to predict taxi ride prices based on features like distance and ride duration. Past ride prices can be used as labels.
The model formula:
y'=b+w1*distance + w2*ride_duration
The goal is to find values for b, w1, and w2 that minimize the MSE for the given labels. A well-trained model should converge after limited number of iterations, where the loss cannot be optimized anymore.
Use Cases:
✏️ Predicting Outcomes. Forecast values based on multiple inputs, e.g., taxi fares, apartment rentals, or flight prices.
✏️ Discovering Relationships. Reveal how variables are related and how changes in one variable affect the whole result.
✏️ Processes Optimizations. Optimize processes by understanding the relationships between different factors.
Studying linear regression made me realize why I learned linear algebra and statistics at university 😄. I really had some fun with the math and dynamic examples.
References:
- Google ML Crash Course: Linear Regression
- Understanding Multiple Linear Regression in ML
#aibasics