Data Analysis Books | Python |… on Telegram: “Exploratory Data Analysis (EDA) EDA is the proce…”

Exploratory Data Analysis (EDA)

EDA is the process of analyzing datasets to summarize key patterns, detect anomalies, and gain insights before applying machine learning or reporting.

1️⃣ Descriptive Statistics
Descriptive statistics help summarize and understand data distributions.

In SQL:

Calculate Mean (Average):

SELECT AVG(salary) AS average_salary FROM employees; 
Find Median (Using Window Functions) SELECT salary FROM ( SELECT salary, ROW_NUMBER() OVER (ORDER BY salary) AS row_num, COUNT(*) OVER () AS total_rows FROM employees ) subquery WHERE row_num = (total_rows / 2);

Find Mode (Most Frequent Value)

SELECT department, COUNT(*) AS count FROM employees GROUP BY department ORDER BY count DESC LIMIT 1;

Calculate Variance & Standard Deviation

SELECT VARIANCE(salary) AS salary_variance, STDDEV(salary) AS salary_std_dev FROM employees;

In Python (Pandas):

Mean, Median, Mode

df['salary'].mean() df['salary'].median() df['salary'].mode()[0]

Variance & Standard Deviation

df['salary'].var() df['salary'].std()

2️⃣ Data Visualization

Visualizing data helps identify trends, outliers, and patterns.

In SQL (For Basic Visualization in Some Databases Like PostgreSQL):

Create Histogram (Approximate in SQL)

SELECT salary, COUNT(*) FROM employees GROUP BY salary ORDER BY salary;

In Python (Matplotlib & Seaborn):

Bar Chart (Category-Wise Sales)

import matplotlib.pyplot as plt 
import seaborn as sns 
df.groupby('category')['sales'].sum().plot(kind='bar') 
plt.title('Total Sales by Category') 
plt.xlabel('Category') 
plt.ylabel('Sales') 
plt.show()

Histogram (Salary Distribution)

sns.histplot(df['salary'], bins=10, kde=True) 
plt.title('Salary Distribution') 
plt.show()

Box Plot (Outliers in Sales Data)

sns.boxplot(y=df['sales']) 
plt.title('Sales Data Outliers') 
plt.show()

Heatmap (Correlation Between Variables)

sns.heatmap(df.corr(), annot=True, cmap='coolwarm') plt.title('Feature Correlation Heatmap') plt.show()

3️⃣ Detecting Anomalies & Outliers

Outliers can skew results and should be identified.

In SQL:

Find records with unusually high salaries

SELECT * FROM employees WHERE salary > (SELECT AVG(salary) + 2 * STDDEV(salary) FROM employees);

In Python (Pandas & NumPy):

Using Z-Score (Values Beyond 3 Standard Deviations)

from scipy import stats df['z_score'] = stats.zscore(df['salary']) df_outliers = df[df['z_score'].abs() > 3]

Using IQR (Interquartile Range)

Q1 = df['salary'].quantile(0.25) 
Q3 = df['salary'].quantile(0.75) 
IQR = Q3 - Q1 
df_outliers = df[(df['salary'] < (Q1 - 1.5 * IQR)) | (df['salary'] > (Q3 + 1.5 * IQR))]

4️⃣ Key EDA Steps

Understand the Data → Check missing values, duplicates, and column types

Summarize Statistics → Mean, Median, Standard Deviation, etc.

Visualize Trends → Histograms, Box Plots, Heatmaps

Detect Outliers & Anomalies → Z-Score, IQR

Feature Engineering → Transform variables if needed

Mini Task for You: Write an SQL query to find employees whose salaries are above two standard deviations from the mean salary.

Here you can find the roadmap for data analyst: https://t.me/sqlspecialist/1159

Like this post if you want me to continue covering all the topics! ❤️

Share with credits: https://t.me/sqlspecialist

Hope it helps :)

#sql