▎Common Data Cleaning Terms

1. Data Cleaning: The process of identifying and correcting inaccuracies, inconsistencies, and errors in a dataset to improve its quality and reliability for analysis.

2. Missing Values: Data points that are absent or not recorded in a dataset; handling missing values is crucial for accurate analysis.

3. Outliers: Data points that deviate significantly from the rest of the dataset; identifying and addressing outliers is important to prevent skewed results.

4. Data Imputation: The method of replacing missing values with substituted values, which can be based on statistical methods, such as mean, median, or mode, or predictive models.

5. Normalization: The process of adjusting values in a dataset to a common scale, often to eliminate units of measurement or to reduce skewness.

6. Standardization: A technique used to center and scale data by transforming it to have a mean of zero and a standard deviation of one, making it suitable for comparison.

7. Deduplication: The process of identifying and removing duplicate records from a dataset to ensure each entry is unique.

8. Data Transformation: The process of converting data from one format or structure into another, often to improve compatibility with analytical tools or models.

9. Data Validation: The process of checking data for accuracy and quality before it is processed or analyzed, ensuring it meets predefined criteria.

10. Data Type Conversion: Changing the data type of a variable (e.g., from string to integer) to ensure consistency and compatibility in analysis.

11. String Manipulation: Techniques used to modify or extract information from text data, including trimming, concatenation, and pattern matching.

12. Categorical Encoding: The process of converting categorical variables into numerical format, such as one-hot encoding or label encoding, to facilitate analysis.

13. Data Profiling: The examination of data sources to understand their structure, content, relationships, and quality; often used to identify issues that need cleaning.

14. Anomaly Detection: The identification of unusual patterns or deviations in data that may indicate errors or significant events requiring further investigation.

15. Data Aggregation: The process of summarizing data points into a single value, such as calculating averages or totals, often used for reporting purposes.

16. Data Filtering: The process of removing unwanted or irrelevant data points from a dataset based on specific criteria or conditions.

17. Data Enrichment: The process of enhancing existing data by adding additional information from external sources to provide more context or insights.

18. Schema Validation: Ensuring that the structure of the dataset adheres to a predefined schema, including the correct data types and relationships between entities.

19. Data Sampling: The selection of a subset of data points from a larger dataset for analysis, often used when working with large datasets to reduce processing time.

20. Data Pipeline: A series of processes through which raw data is collected, cleaned, transformed, and made ready for analysis or storage in a database.