Z-scoring for simple and fast anomaly detection
Anomaly detection is a fairly common problem that covers many scenarios, from financial fraud to computer network failures. Some problems require complex machine learning models, but most often some simpler and cheaper methods are sufficient. For example, you have sales data over a period of time where you want to flag days with abnormally high volumes or highlight customers with abnormally high credit card swipes for risk testing.
For such cases, a simple statistical method of marking outliers, called Z-scoring, will do. The score is equal to the difference between the current and mean values, divided by the standard deviation. Z-scoring assumes the classical normal distribution of random variables. Converting nominal scale values to a logarithmic scale will improve the ability of most ML models to discern relationships and improve the ability of Z-scores to flag outliers.