Beginners in ML often hear:
Always normalize the data.
And they start scaling everything, then the model quality... drops.
➡️ Why does this happen?
Because normalization isn't always necessary.
What does normalization actually do?
it brings the features to the same scale.
For example:
age → 18–60
salary → 1000–100000
After scaling:
the values become comparable and the training becomes more stable
When normalization is really needed?
It's especially important for models that are sensitive to scale:
Logistic Regression, Linear Regression, SVM, KNN, Neural Networks
Without scaling, such models may work worse
or train unstably.
And now the most important thing, Trees usually don't need scaling.
These are Random Forest, XGBoost, LightGBM, CatBoost
Why? Because trees make splits: feature < threshold
And it doesn't matter to them: whether it's 0.5 or 5000 and the scale hardly matters
How normalization can worsen the model?
1. It adds noise
Sometimes scaling blurs the distributions, amplifies outliers, worsens separability Especially on bad data.
2. It breaks interpretability
It used to be: income = 5000
Now it's: income = -0.73
It's harder to explain this to the business.
3. Incorrect scaling = leakage
A classic mistake: scaling on the entire dataset, then splitting
The test has already "leaked" into the train.
4. CatBoost can get worse
CatBoost works well with: categorical features, original distributions
Sometimes extra preprocessing just gets in the way.
The most important insight, Scaling isn't a "data improvement" tool. It's a tool for a specific model.
What to do in practice?
A simple rule: linear models / distance-based → scaling is needed, trees → usually not needed
Normalization isn't always useful, for some models it's useless, and sometimes even harmful.
••••••••••••••••••••••••••••••••••••••
🤖 Data & ML | @DataXplore