Sebastian Raschka considered some experiments testing one popular hypothesis about batch size "Does batch size must be the power of 2?" in his new paper "No, We Don't Have to Choose Batch Sizes As Powers Of 2" (https://sebastianraschka.com/blog/2022/batch-size-2.html). He concluded that setting batch size as powers of 2 is not necessary, because it doesn't slow down the training or inference significantly. However, the batch size is still a hyperparameter, which depends on many factors: model architecture, loss function, learning rate, etc, so we should spend some time tuning it carefully despite some arguments, which recommend increasing or decreasing batch size.