Data cleaning is the process of detecting and correcting inaccurate, incomplete, or inconsistent data to improve data quality for analysis and modeling. It is a crucial step in any data science workflow.
Handling Missing Values
df.isnull().sum() # Check missing values
df.dropna() # Remove rows with missing values
df.fillna(0) # Replace missing values
Removing Duplicate Data
df.duplicated() # Identify duplicates
df.drop_duplicates() # Remove duplicates
Correcting Data Types
df.dtypes #identify data types
df["age"] = df["age"].astype(int) #convert age column to integer data type
df["date"] = pd.to_datetime(df["date"]) #convert date column to date data type
Renaming Columns
df.columns = df.columns.str.lower().str.replace(" ", "_")Handling Inconsistent Data
df["gender"] = df["gender"].str.lower() #convert to lower case
df["name"] = df["name"].str.strip()
Clean data leads to more accurate analysis and reliable models. Python’s pandas library simplifies cleaning tasks such as handling missing values, duplicates, incorrect types, and inconsistencies.