According to common belief, neural networks' main advantage over traditional ML algorithms is that NNs learn features by themselves while in the traditional ML, you handcraft features. This is not exactly true. Well, it's true for vanilla feed-forward NNs consisting only of fully connected layers. But those are very hard to train for high dimensional inputs like images.

When you use a convolutional neural network, you already use two types of handcrafted features: convolution filters and pooling filters.

The designer of the convolutional NN for image classification has looked into the input data (this is what traditional ML engineers do to invent features) and decided that patches of pixels close to each other contain information that could help in classification, and at the same time reduce the number of NN parameters.

The same reasoning is used when we classify texts using bag-of-words features. We look at the data and decide that individual words and n-grams of words would be good features to classify a document. This reduces the number of input features while allowing us to accurately classify documents.

BTW, the way convolutional filters apply (sum of element-wise multiplications, spanning over channels resulting in one number) is a hell of a feature!
Burkov

❇️ @AI_Python_EN