NewThe ManyPI MCP Server is now live

Data cleaning basics

Why real data is messy, the five problems you meet most often and how to fix each one before you count anything.

Guide30 minBachelor +2CC BY 4.0

What you will learn

  • Name the five most common problems in a data set
  • Choose a fix for each
  • Explain why cleaning comes before counting

A count is only as good as the rows behind it. If one customer appears three times, every average that includes them is wrong. Cleaning means finding these problems before you calculate.

The five problems you meet most

Problem

Example

Fix

Duplicates

The same person entered twice

Choose a key column, remove repeats

Inconsistent spelling

Munich, München, munich

Agree on one form and replace the others

Extra spaces and capitals

" Anna@Example.com "

Trim spaces, compare in lower case

Missing values

An empty country field

Fill it from another source, or mark it unknown

Wrong types

12,5 stored as text

Convert to a number with a point

Sort into groups

Which problem is it?

Duplicate

Inconsistent spelling

Missing value

In which order?

Trim and normalise first, then remove duplicates: two rows that differ only in a space are only found once the spaces are gone. Try it with the contact list worksheet and the free CSV Deduplicator.

Reuse this material

CC BY 4.0

You may copy, adapt and share it, also commercially, as long as you credit ManyPI and link back.

"Data cleaning basics" by ManyPI (https://manypi.com/edu/data-cleaning-basics), licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)