Financial Management and Business Data Analytics · Data Processing, Organisation, Cleaning and Validation

Data Cleaning Techniques and Data Quality Issues

Updated 10 October 2026 · Fact-checked

Data cleaning (cleansing) is finding and fixing errors in a dataset before analysis. Typical errors are missing values, duplicates, outliers and inconsistent formats. You detect each problem, choose a fix (delete, correct, impute, standardise), document it, and re-check quality on accuracy, completeness, consistency, validity, timeliness and uniqueness.

Understand Data Cleaning and Data Quality Issues

Data is rarely perfect when it arrives. It is typed by people, exported from different systems and merged from many sources. Analysis done on faulty data gives faulty answers, however good the method. This is often called garbage in, garbage out.

Data cleaning (also called data cleansing or scrubbing) is the process of detecting, correcting or removing inaccurate, incomplete, duplicate or badly formatted records. It sits in the data preparation stage, before analysis and reporting.

The common problems are:
- Missing values: blank cells, for example a customer's GST number or a month's sales not recorded.
- Duplicates: the same record entered twice, for example one invoice booked two times.
- Outliers: values far from the rest, for example a salary of ₹25,00,000 in a list where others are about ₹25,000. An outlier may be an error or a genuine rare value.
- Inconsistent formats: dates as 05/06/2026 and 5-Jun-26, or names as 'Infosys Ltd' and 'Infosys Limited'.
- Invalid or wrong entries: negative quantity, age of 250, text in a numeric field, spelling errors.

Data quality is judged on dimensions. Accuracy: values match reality. Completeness: required data is present. Consistency: same fact is the same across records and systems. Validity: values follow defined format and range rules. Timeliness: data is up to date. Uniqueness: no unwanted duplicates.

Cleaning is judgement, not only mechanics. Deleting a record loses information. Filling a blank adds an assumption. You should always choose the fix that suits the data and record what you did.

Key rules to remember

Mean imputation: Replacement value = Σx ÷ n (of the available values); Suits numeric data without extreme outliers. Use the median when outliers exist.
Outlier fences (IQR rule): Lower fence = Q1 − 1.5 × IQR; Upper fence = Q3 + 1.5 × IQR; IQR = Q3 − Q1; A value outside the fences is flagged as a possible outlier. Flagged does not mean wrong.
Z-score: z = (x − mean) ÷ standard deviation; A common rule of thumb flags |z| greater than 3. It is a convention, not a law.
Completeness: Completeness % = (Records with required fields filled ÷ Total records) × 100; A simple measure of the completeness dimension.
Duplicate rate: Duplicate % = (Duplicate records ÷ Total records) × 100; Count only the extra copies, not the first occurrence.

How to solve Data Cleaning and Data Quality Issues questions

Use this order for any question on identifying or fixing data problems.

1Read the data or scenario and list each problem you can see: missing, duplicate, outlier, format, invalid.
2Name the quality dimension each problem breaks (completeness, uniqueness, accuracy, consistency, validity).
3Decide whether the issue is a true error or a genuine value. Check against source documents if possible.
4Pick a technique: delete, correct from source, impute (mean, median, mode), cap, or standardise format.
5Justify the choice in one line, for example median because an outlier would distort the mean.
6Apply the technique and show the calculation if numbers are involved.
7State how you will validate the result (range checks, duplicate checks, re-count) and document the change.

Quickest way: Spot, label, fix, justify

When to use it: For MCQs and short-answer questions where you must match a problem to its remedy.

Spot the symptom: blank = missing, repeated row = duplicate, extreme value = outlier, mixed styles = inconsistent format.
Label it with the matching quality dimension.
Recall the default fix: impute or collect, remove extra copy, investigate then keep or cap, standardise.
If a mean is asked and an outlier is present, think median.
Eliminate options that say 'always delete' or 'ignore'; they are rarely correct.

Common mistakes in Data Cleaning and Data Quality Issues

Deleting every record with a missing value.
It looks like the quickest clean-up.
Fix: Delete only when few records are affected and the loss does not bias results. Otherwise impute or collect the missing data.
Removing all outliers automatically.
Students treat outliers as errors.
Fix: Investigate first. A large genuine sale is valid data. Remove or correct only confirmed errors.
Using the mean to fill blanks when extreme values exist.
Mean is the default measure students remember.
Fix: Use the median for skewed data or data with outliers, and the mode for categories.
Confusing accuracy with consistency.
Both sound like 'correct data'.
Fix: Accuracy means true to reality. Consistency means the same across records or systems. Data can be consistent but wrong.
Counting the first occurrence as a duplicate.
Students count all copies of a repeated record.
Fix: If a record appears 3 times, there are 2 duplicates to remove and 1 to keep.
Cleaning without documenting changes.
Focus stays on the answer, not the process.
Fix: Always mention keeping a log or backup of the original data so changes can be audited.

Worked examples

Example 1

Monthly sales (₹ lakh) of a Pune distributor for six months are: 12, 14, blank, 13, 15, 16. (a) Fill the blank by mean imputation. (b) Find the completeness percentage of the original data.

Show the solution

Available values: 12, 14, 13, 15, 16. Number of values n = 5.
Sum = 12 + 14 + 13 + 15 + 16 = 70.
Mean = 70 ÷ 5 = 14. So the blank is filled with 14.
Completeness = 5 ÷ 6 × 100 = 83.33%.

Answer: (a) Imputed value is ₹14 lakh. (b) Completeness of the original data is about 83.33%.

Example 2

A dataset of employee monthly salaries (₹ thousand) has the values: 20, 22, 24, 25, 26, 28, 30, 31, 32, 150. Using the IQR rule with Q1 = 24 and Q3 = 31, identify any outlier and suggest the treatment.

Show the solution

IQR = Q3 − Q1 = 31 − 24 = 7.
1.5 × IQR = 1.5 × 7 = 10.5.
Lower fence = 24 − 10.5 = 13.5.
Upper fence = 31 + 10.5 = 41.5.
Values below 13.5 or above 41.5 are flagged. Only 150 lies outside the fences.
Treatment: verify 150 against payroll records. If it is a typing error (for example 15.0), correct it. If it is genuine (a senior executive), keep it but use the median for summaries, or analyse it separately.

Answer: The value ₹150 thousand is an outlier (fences are 13.5 and 41.5). Verify it at source, then correct, keep or cap it as justified, and prefer the median while it remains.

Exam tips

Learn the six quality dimensions with a one-line meaning and one example each; definition-plus-example questions are easy marks.
In MCQs, match the symptom to the problem name first, then the fix. Avoid options with 'always' or 'never'.
In written answers, use a small table-like list: problem, example, technique. This earns step marks.
If numbers are given, show the formula line before calculating, for imputation, fences or percentages.
Mention validation and documentation at the end of any descriptive answer.

Practice questions from Data Processing, Organisation, Cleaning and Validation

Data Cleaning and Data Quality Issues in other exams

The same ground in other exams, if you are preparing for more than one or want another angle on it.

Data Cleaning and Data Quality Issues: frequently asked questions

What is data cleaning with an example?

Data cleaning is correcting or removing faulty records before analysis. For example, a sales file has 'Mumbai', 'mumbai' and 'Bombay' for one city and the same invoice twice. You standardise the city name and remove the extra invoice.

How do you handle missing values?

First find why the values are missing and how many there are. You can delete the record, fill with mean, median or mode, or collect the data again from the source. Choose the method that causes least bias and record it.

Should outliers always be removed?

No. An outlier may be a data entry error or a real rare event. Check against the source. Remove or correct it only if it is an error, otherwise keep it and use measures such as the median.

What are the main data quality dimensions?

The main ones are accuracy, completeness, consistency, validity, timeliness and uniqueness. Accuracy means true values, completeness means nothing required is missing, and consistency means the same data agrees across places.