Many machine learning algorithms use kernels: the support vector machine, the principal component kernel, and others. Their task is to calculate the dot product in some transformed feature space, usually of high dimensionality, without explicitly transitioning to this space.
The idea is this: instead of explicitly constructing the mapping φ(x) to the new space and then calculating ⟨φ(X), φ(Y)⟩, the kernel function k(X, Y) is used, which immediately returns the result of this dot product.
An example with a polynomial kernel:
k(X, Y) = (1 + XᵀY)²
Let:
X = (x1, x2)
Y = (y1, y2)
If we expand the expression, it turns into the dot product of two vectors in a higher-dimensional space (in this case — 6 dimensions). At the same time, the coordinates themselves in this space are not explicitly calculated.
Hence the meaning of the "trick": the result is calculated in a high-dimensional space without explicitly constructing the vectors themselves in this space.
The Gaussian kernel (RBF) enhances this effect: it corresponds to working in an infinite-dimensional feature space, while the calculations remain finite and compact due to the form of the kernel function.
The mathematics behind the RBF kernel → link
••••••••••••••••••••••••••••••••••••••
🤖 Data & ML | @DataXplore