Binary Data Classification

In the previous #aibasics post, I briefly explained the basics of machine learning with Linear Regression. Today let's talk about another type of task - binary data classification. Typical example is determining whether an email is spam or not spam.

Key steps for binary classification:

1. Predict Probability. Take a Logistic Regression model that predict probability (mathematically it returns values between 0 and 1). For example, the probability of an input email being either spam or not spam. If the model predicts 0.72, this means there is a 72% chance the email is spam and 28% chance the email is not spam.

2. Set a Classification Threshold . The classification threshold determines how to assign a binary label (e.g., spam or not spam) based on the predicted probability. For example, the model predicts that a given email has a 75% chance of being spam. Does it mean the email is spam? Actually, no. If the threshold is set at 0.8, then email will be classified as not spam.

3. Evaluate the Model Using a Confusion Matrix. To measure how good our model is, we need to summarize the number of correct and incorrect predictions using confusion matrix:
- True Positive (TP): Correctly predicted positive cases.
- False Negative (FN): Positive cases incorrectly predicted as negative.
- False Positive (FP): Negative cases incorrectly predicted as positive.
- True Negative (TN): Correctly predicted negative cases.

4. Measure Classification Quality. The following metrics are used to define the effectiveness of the result model:
- Accuracy. The proportion of all classifications that were correct, whether positive or negative.
- Recall. The proportion of all actual positives that were classified correctly as positives.
- False Positive Rate. The proportion of all actual negatives that were classified incorrectly as positives.
- Precision. The proportion of all the model's positive classifications that are actually positive

The classification threshold and quality metrics should be adjusted based on the cost of errors for particular domain. If marking important emails as spam is costly, you may increase the threshold to reduce false positives. Conversely, if missing spam emails is more problematic, you may lower the threshold to prioritize catching them.

References:
- Google ML Course: Logistic Regression
- Google ML Course: Classification
- Confusion matrix in machine learning

#aibasics