Dirichlet Neural Networks, a Bayesian Approach

Epistemic uncertainty relates to the spread of the categorical distribution p ( π | x ∗ , D ) on the simplex, which corresponds to α 0 = ∑ C c = 1 α c for a Dirichlet distribution. Aleatoric uncertainty is linked to the position of the mean on the simplex. Equipped with this configuration, we would like Dirichlet network to yield the desired behaviors shown in the figure below:

When the model is confident in its prediction, it should yield a sharp distribution centered on one of the corners of the simplex (Fig a.). For an input in a region with high degrees of noise or class overlap (aleatoric uncertainty), it should yield a sharp distribution focused on the center of the simplex, which corresponds to being confident in predicting a flat categorical distribution over class labels (Fig b.). Finally, for out-of-distribution inputs, the Dirichlet network should yield a flat distribution over the simplex, indicating large epistemic uncertainty (Fig c.).

https://chcorbi.github.io/posts/2020/11/dirichlet-networks/