In statistical modeling, apparent violations of distributional assumptions (e.g., normality, Poisson) may result from heterogeneity we haven't accounted for.

In plain English, these violations may result from outliers, for example. Outliers can tell us a lot about are data (e.g., why most people are this way and not that way). They may also be hinting that something is amiss with our data.

Violations of distributional assumptions may also tell us we haven't thought about the problem enough! For example, we may be trying to force a one-size-fits-all model on our data. But different types of consumers may have different motivations, so a regression model that aims at the middle may miss most of them. People also transition in and out of different "states" because of marriage, promotions, childbirth, etc.

One reason predictive analytics often goes wrong is because of people working in factory-like (or sweatshop-like) environments being forced to crank out as many models as possible as quickly as possible.

Well-trained analysts given more time to think about the data and business problem can extract more value from the same data. So the problem may lie more with the execution than the idea itself.

✴️ @AI_Python_EN