🙌🏻Mathematics for the Data Scientist, Part 1: Benford's Law
Benford's (Newcomb-Benford) law of the first digit describes the probability of occurrence of a certain first significant digit in distributions of quantities taken from real life. This mathematical law is true for many distributions, it allows you to predict the frequency of occurrence of the second and third digits in the dataset.
For the first time this law was discovered by the American astronomer Simon Newcome in 1881, analyzing the degree of wear and tear of book pages. And in 1938, physicist Frank Benford made similar conclusions based on the results of the analysis of tables on the characteristics of rivers, chemical compounds and house numbers in the city directory. An analysis of numbers showed that one is the first significant digit with a probability of not 1/9, as it seems at first glance, but about 1/3.
Benford's Law applies to sets of numbers that can grow exponentially, i.e. the rate of growth of a value is proportional to its current value, for example, stock balances in warehouses, stock prices, population size, length of rivers, area of countries.A set of numbers satisfies Benford's law if the first digit d (𝑑∈1,…, 9) occurs in the equation. Using this distribution, you can predict which digit occurs most frequently in the dataset. The law usually does not apply for distributions with given minimum or maximum values, as well as those that cover only one or two orders of magnitude. Also Benford's law does not apply to texts. The sample size for the law of the first digit should be sufficient to apply statistical methods. In practice, the first digit law is applied in applications for detecting fraud in tax forms, election results, economic performance and accounting data.