30 Days of Data Science Series: https://t.me/datasciencefun/1708
Let's learn about k-Means Clustering today
Concept: k-Means is an unsupervised learning algorithm used for clustering tasks. The goal is to partition a dataset into \( k \) clusters, where each data point belongs to the cluster with the nearest mean. It is an iterative algorithm that aims to minimize the variance within each cluster.
The steps involved in k-Means clustering are:
1. Initialization: Choose \( k \) initial cluster centroids randomly.
2. Assignment: Assign each data point to the nearest cluster centroid.
3. Update: Recalculate the centroids as the mean of all points in each cluster.
4. Repeat: Repeat steps 2 and 3 until the centroids do not change significantly or a maximum number of iterations is reached.
#### Implementation Example
Suppose we have a dataset with points in 2D space, and we want to cluster them into \( k = 3 \) clusters.
# Import necessary libraries
import numpy as np
import pandas as pd
from sklearn.cluster import KMeans
import matplotlib.pyplot as plt
import seaborn as sns
# Example data
np.random.seed(0)
X = np.vstack((np.random.normal(0, 1, (100, 2)),
np.random.normal(5, 1, (100, 2)),
np.random.normal(-5, 1, (100, 2))))
# Applying k-Means clustering
k = 3
kmeans = KMeans(n_clusters=k, random_state=0)
y_kmeans = kmeans.fit_predict(X)
# Plotting the clusters
plt.figure(figsize=(8,6))
sns.scatterplot(x=X[:, 0], y=X[:, 1], hue=y_kmeans, palette='viridis', s=50, edgecolor='k')
plt.scatter(kmeans.cluster_centers_[:, 0], kmeans.cluster_centers_[:, 1], s=300, c='red', label='Centroids')
plt.xlabel('Feature 1')
plt.ylabel('Feature 2')
plt.title('k-Means Clustering')
plt.legend()
plt.show()
## Explanation of the Code
1. Libraries: We import necessary libraries like
numpy, pandas, sklearn, matplotlib, and seaborn.2. Data Preparation: We generate a synthetic dataset with three clusters using normal distributions.
3. k-Means Clustering: We create a
KMeans object with \( k=3 \) clusters and fit it to the data. The fit_predict method assigns each data point to a cluster.4. Plotting: We scatter plot the data points with colors indicating the assigned clusters and plot the centroids in red.
#### Choosing the Number of Clusters
Selecting the appropriate number of clusters (\( k \)) is crucial. Common methods to determine \( k \) include:
- Elbow Method: Plot the within-cluster sum of squares (WCSS) against the number of clusters and look for an "elbow" point where the rate of decrease sharply slows.
- Silhouette Score: Measures how similar an object is to its own cluster compared to other clusters. Higher silhouette scores indicate better-defined clusters.
## Elbow Method Example
# Elbow Method to find the optimal number of clusters
wcss = []
for i in range(1, 11):
kmeans = KMeans(n_clusters=i, random_state=0)
kmeans.fit(X)
wcss.append(kmeans.inertia_)
plt.figure(figsize=(8,6))
plt.plot(range(1, 11), wcss, marker='o')
plt.xlabel('Number of clusters')
plt.ylabel('WCSS')
plt.title('Elbow Method')
plt.show()
## Evaluation Metrics
- Within-Cluster Sum of Squares (WCSS): Measures the compactness of the clusters. Lower WCSS indicates more compact clusters.
- Silhouette Score: Measures the separation between clusters. Values range from -1 to 1, with higher values indicating better-defined clusters.
#### Applications
k-Means clustering is widely used in:
- Market Segmentation: Grouping customers based on purchasing behavior.
- Image Compression: Reducing the number of colors in an image.
- Anomaly Detection: Identifying outliers in a dataset.
k-Means is efficient and easy to implement but can be sensitive to the initial placement of centroids and the choice of \( k \). It works well for spherical clusters but may struggle with non-spherical or overlapping clusters.
Best Data Science & Machine Learning Resources: https://topmate.io/coding/914624
Credits: t.me/datasciencefun
ENJOY LEARNING 👍👍