Remove duplicates from a CSV file

Drop a CSV below to remove duplicate rows. It is properly parsed (quoted commas and multi-line fields included), deduped on the columns you choose, and the order is preserved. Processed in your browser; the file never leaves your machine.

Drop a .xlsx / .csv file here or

Processed locally. Check your Network tab: no uploads.

Why not just sort -u?

sort -u and uniq work on lines, not CSV records: they reorder your file, can't limit matching to key columns, and corrupt records that contain quoted line breaks. A CSV-aware dedupe keeps the header, keeps order, and matches on the fields you pick. For scripts, the API does the same over HTTP; for spreadsheets, see removing duplicates in Excel, or removing duplicates from a legacy .xls file if your export predates 2007.

FAQ

Frequently asked questions

How do I remove duplicate rows from a CSV without Excel?

Drop the file into the tool above. It parses the CSV in your browser (quoted fields, commas, and line breaks handled correctly) and downloads a deduplicated copy. Nothing is uploaded.

Can I dedupe on specific columns?

Yes. After loading the file, click the column chips (e.g. just the email column). Rows are duplicates when the selected columns match.

What about the command line?

On macOS/Linux, `sort -u file.csv` removes exact duplicate lines but destroys row order and breaks on quoted multi-line fields. `awk '!seen[$0]++'` preserves order but is still line-based, not CSV-aware. This tool is CSV-aware and order-preserving.

Is there an API for this?

Yes. POST your CSV to the free Dedupe API and get the cleaned file back. See the API docs.

Last updated