30 Days of Data Science Series: https://t.me/datasciencefun/1708
Let's learn Support Vector Machine in detail
Concept: Support Vector Machines (SVM) are supervised learning models used for classification and regression tasks. The goal of SVM is to find the optimal hyperplane that maximally separates the classes in the feature space. The hyperplane is chosen to maximize the margin, which is the distance between the hyperplane and the nearest data points from each class, known as support vectors.
For nonlinear data, SVM uses a kernel trick to transform the input features into a higher-dimensional space where a linear separation is possible. Common kernels include:
- Linear Kernel
- Polynomial Kernel
- Radial Basis Function (RBF) Kernel
- Sigmoid Kernel
## Implementation Example
Suppose we have a dataset that records features like petal length and petal width to classify the species of iris flowers.
# Import necessary libraries
import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.svm import SVC
from sklearn.metrics import accuracy_score, confusion_matrix, classification_report
import matplotlib.pyplot as plt
import seaborn as sns
# Example data (Iris dataset)
from sklearn.datasets import load_iris
iris = load_iris()
X = iris.data[:, 2:4] # Using petal length and petal width as features
y = iris.target
# Splitting the data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)
# Creating and training the SVM model with RBF kernel
model = SVC(kernel='rbf', C=1.0, gamma='scale', random_state=0)
model.fit(X_train, y_train)
# Making predictions
y_pred = model.predict(X_test)
# Evaluating the model
accuracy = accuracy_score(y_test, y_pred)
conf_matrix = confusion_matrix(y_test, y_pred)
class_report = classification_report(y_test, y_pred)
print(f"Accuracy: {accuracy}")
print(f"Confusion Matrix:\n{conf_matrix}")
print(f"Classification Report:\n{class_report}")
# Plotting the decision boundary
def plot_decision_boundary(X, y, model):
h = .02 # step size in the mesh
x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1
y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1
xx, yy = np.meshgrid(np.arange(x_min, x_max, h), np.arange(y_min, y_max, h))
Z = model.predict(np.c_[xx.ravel(), yy.ravel()])
Z = Z.reshape(xx.shape)
plt.contourf(xx, yy, Z, alpha=0.8)
sns.scatterplot(x=X[:, 0], y=X[:, 1], hue=y, palette='bright', edgecolor='k', s=50)
plt.xlabel('Petal Length')
plt.ylabel('Petal Width')
plt.title('SVM Decision Boundary')
plt.show()
plot_decision_boundary(X_test, y_test, model)
#### Explanation of the Code
1. Importing Libraries
2. Data Preparation
3. Train-Test Split
4. Model Training: We create an
SVC model with an RBF kernel (kernel='rbf'), regularization parameter C=1.0, and gamma parameter set to 'scale', and train it using the training data.5. Predictions: We use the trained model to predict the species of iris flowers for the test set.
6. Evaluation: We evaluate the model using accuracy, confusion matrix, and classification report.
7. Visualization: Plot the decision boundary to visualize how the SVM separates the classes.
#### Decision Boundary
The decision boundary plot helps to visualize how the SVM model separates the different classes in the feature space. The SVM with an RBF kernel can capture more complex relationships than a linear classifier.
SVMs are powerful for high-dimensional spaces and effective when the number of dimensions is greater than the number of samples. However, they can be memory-intensive and require careful tuning of hyperparameters such as the regularization parameter \(C\) and kernel parameters.
Credits: t.me/datasciencefun
ENJOY LEARNING 👍👍