# Dedupe.dev

> Free in-browser tools that remove duplicates from Excel, CSV, text lists, and
> JSON arrays. Files are processed entirely in the browser and never uploaded.

Full text of every page: https://dedupe.dev/llms-full.txt

## Tools

- [Remove Duplicates from Excel, CSV, Text & JSON Free | Dedupe.dev](https://dedupe.dev/): Drop an Excel, CSV, text list, or JSON array below and get a deduplicated copy in seconds. Files are processed entirely in your browser. Nothing is uploaded.
- [Remove Duplicates in Excel: Free Online Tool (No Upload)](https://dedupe.dev/excel): There are two fast ways to remove duplicates from an Excel sheet: Excel's built-in Data → Remove Duplicates button (exact matches only), or the free tool below. Drop your .xlsx or .csv, pick which columns define a duplicate, and download a cleaned copy. Your file is processed in your browser and never uploaded.
- [Remove Duplicates from CSV Online: Free, CSV-Aware, No Upload](https://dedupe.dev/csv): Drop a CSV below to remove duplicate rows. It is properly parsed (quoted commas and multi-line fields included), deduped on the columns you choose, and the order is preserved. Processed in your browser; the file never leaves your machine.
- [Remove Duplicate Lines Online: Free, Order-Preserving](https://dedupe.dev/text): Paste any list below (emails, URLs, keywords, log lines) and get it back with duplicates removed, original order preserved. Runs in your browser; nothing is uploaded.
- [Dedupe a JSON Array Online, by Key Path, Order-Preserving](https://dedupe.dev/json): Paste a JSON array below and dedupe it, either whole-object or by a key path like user.id. Order is preserved and the first occurrence is kept. All in your browser; use the API for pipelines.
- [Contact List Dedupe for CRM Exports: Free, In Your Browser](https://dedupe.dev/contacts): Drop your contact CSV below and dedupe on the email column: email is the identifier that survives name formatting, and exact matching handles hundreds of thousands of rows instantly. Your contact data is processed in your browser. It is never uploaded, and no CRM access is required. For name-based matching ("Smith, Jane" vs "jane smith"), use fuzzy matching in the API.
- [XLS Remove Duplicates: Free Tool for Legacy Excel Files (No Upload)](https://dedupe.dev/xls): .xls is the binary Excel format Microsoft used before 2007, and it still arrives from older accounting, payroll, and reporting systems. Two ways to clean it: Excel's built-in Data → Remove Duplicates, or the tool below, which reads legacy .xls as well as .xlsx and lets you choose which columns define a duplicate. Parsing happens in your browser; the file is never uploaded.
- [Dedupe API: HTTP Deduplication for CSV, Lines & JSON](https://dedupe.dev/api): POST https://dedupe.dev/api/dedupe with a JSON body and get the deduplicated result back. Three modes are available: lines, rows (CSV), and json. First occurrence kept, order preserved. Metered in units, not calls: one unit is one row of up to 1 KB, and a free key gets 1,000 a month. CORS-open, and nothing you send is stored.
- [Changelog | Dedupe.dev](https://dedupe.dev/changelog): What has actually changed, newest first. The tools are free and run in your browser; nothing here requires an account.
- [Check your email | Dedupe.dev](https://dedupe.dev/pro-thanks): We sent you a one-time sign-in link. It is valid for 30 minutes and signs you in on this device. No password to set, and nothing to install.

## Guides

- [How to Delete Duplicate Photos on iPhone (Built-in, No App Needed)](https://dedupe.dev/guides/iphone-duplicate-photos): You don't need an app: iOS 16 and later finds duplicates for you. Open Photos → Albums → scroll down to Utilities → Duplicates, then tap Merge on each pair (or Select → Merge All). Merging keeps the highest-quality copy and moves the rest to Recently Deleted.
- [Remove Duplicates in Google Photos: What Works in 2026](https://dedupe.dev/guides/google-photos-duplicates): Google Photos already prevents exact duplicate uploads, so the same file won't be stored twice. What it won't do is merge near-duplicates (bursts, edits, resized copies) that are already in your library. Those you remove manually: search or scroll a date, long-press to multi-select, delete, then empty Trash to reclaim storage.
- [Find & Delete Duplicate Files on Mac: Free Methods That Work](https://dedupe.dev/guides/mac-duplicate-files): macOS has no built-in duplicate file finder (Photos handles pictures only). The practical route: use a Finder Smart Folder sorted by size to spot obvious repeats, or run dupeGuru, which is free, open-source, and scans by file content so renamed copies are still caught. Review matches before deleting; send to Trash, don't hard-delete.
- [Remove Duplicates in Google Sheets: UNIQUE, Data Cleanup & Beyond](https://dedupe.dev/guides/remove-duplicates-google-sheets): Google Sheets has this built in, two ways: =UNIQUE(range) returns a deduplicated copy of your data, and Data → Data cleanup → Remove duplicates deletes duplicate rows in place, keeping the first occurrence. Both match exactly, so for messy data (extra spaces, capitalization, reordered names) you'll want a normalizing tool.
- [Excel Formulas to Find & Remove Duplicates: COUNTIF, COUNTIFS, UNIQUE](https://dedupe.dev/guides/remove-duplicates-excel-formula): Put =COUNTIF($A$2:$A$1000,A2)>1 in a helper column: every TRUE is a value that appears more than once, ready to filter, review, and delete. On Excel 365, =UNIQUE(A2:C1000) goes one step further and spills a deduplicated copy of the whole range.
- [Delete Duplicate Photos on Android: Files by Google, No Extra App](https://dedupe.dev/guides/android-duplicate-photos): You likely already have the tool: Files by Google is preinstalled on most Android phones and finds duplicates for you. Open Files → Clean, look for the Duplicate files suggestion, review what's selected, and delete. No third-party cleaner app needed.
- [How to Dedupe CRM Contacts: Salesforce & HubSpot Cleanup Workflow](https://dedupe.dev/guides/dedupe-crm-contacts): First, use what your CRM ships with: Salesforce duplicate rules and HubSpot's Manage duplicates both catch obvious pairs. For the backlog they miss, the workflow that works is export → clean → re-import: export contacts to CSV, dedupe on the email column with the contacts tool (in your browser, nothing uploaded), and re-import. Name variants email misses are a job for the API's fuzzy matching.
- [Delete Duplicate Photos: iPhone, Android, Google Photos, Mac & PC](https://dedupe.dev/guides/duplicate-photos): Start with the honest part: a browser cannot reach your camera roll, so no website can clean your photo library in place, including this one. Anything web-based means uploading your photos to somebody's server first. The good news is that every major platform already ships a duplicate finder, and this page points you at the right one.
- [What Does Dedupe Mean? Definition, Methods and Real Examples](https://dedupe.dev/guides/what-is-dedupe): Dedupe is short for deduplicate: to remove repeated entries from a set of data so that each distinct thing appears exactly once. It works as a verb ("dedupe the mailing list") and as a name for the process. The longer form, deduplication, means the same thing and is the word you will see in storage and backup documentation.
- [How to Remove Duplicates in a Notion Database (No Native Dedupe)](https://dedupe.dev/guides/remove-duplicates-notion): Notion has no built-in dedupe command, so this is a two-part job: find the duplicates, then delete the extra pages. The trap to know about first is that importing a cleaned CSV back into Notion creates new pages rather than updating existing ones, so a careless round trip doubles your database instead of fixing it.
- [Remove Duplicate Emails From a List Before Import (Case, Plus Tags)](https://dedupe.dev/guides/remove-duplicate-emails-list): Normalise before you compare: lowercase and trim every address, because Bob@Example.com and bob@example.com are the same mailbox but a plain exact-match dedupe keeps both. Then decide how to treat plus-addressing (user+tag@) and, for Gmail specifically, dots in the local part, which are ignored on delivery.
- [Remove Duplicate Rows in SQL: DISTINCT, GROUP BY, ROW_NUMBER, DELETE](https://dedupe.dev/guides/remove-duplicates-sql): To read a deduplicated result, SELECT DISTINCT over only the columns that define a duplicate. To delete duplicates while keeping one row per group, rank the rows with ROW_NUMBER() OVER (PARTITION BY key ORDER BY tiebreaker) in a CTE and delete everything ranked above 1.
- [Remove Duplicates in Python with pandas: drop_duplicates, subset, keep](https://dedupe.dev/guides/remove-duplicates-python-pandas): df.drop_duplicates() drops rows that repeat across every column; df.drop_duplicates(subset=["email"], keep="last") dedupes on a key column and picks which row survives. Inspect before you delete with df[df.duplicated(subset=["email"], keep=False)], which returns every member of every duplicate group rather than just the repeats.
- [Find Duplicate Files on Windows Free: PowerShell Get-FileHash](https://dedupe.dev/guides/find-duplicate-files-windows): Windows 10 and 11 have no built-in duplicate file finder, and Storage Sense does not do this job. The free native route is PowerShell: hash every file with Get-FileHash, group by the hash, and any group with more than one member is a set of byte-identical copies, whatever their filenames say.
- [Remove Duplicate Contacts in Outlook & Microsoft 365: The Real Method](https://dedupe.dev/guides/remove-duplicates-outlook-contacts): First, the correction that saves an hour: Outlook's Clean Up button works on email conversations, not on contacts, so it will not help here. For contacts, take the merge prompts Outlook offers as you save, and clear the backlog by exporting to CSV, deduping the file, and importing it back with a duplicate option that replaces rather than adds.
- [How to Find Duplicates in Excel: Highlight Them Before You Delete](https://dedupe.dev/guides/find-duplicates-in-excel): Select your data and use Home → Conditional Formatting → Highlight Cells Rules → Duplicate Values. Excel colours every value that occurs more than once and deletes nothing, which is the point: you get to see the problem before you decide what to do about it. For a list you can filter and count, add a helper column with =COUNTIF($A$2:$A$1000,A2)>1 instead.
- [How to Use Dedupe.dev: A Walkthrough of Every Tab and Option](https://dedupe.dev/guides/how-to-use-dedupe-dev): The tool has three tabs: Excel / CSV for spreadsheet files, Text lines for a pasted list, and JSON for an array of objects. Everything runs in the page, so no file is uploaded, the first occurrence of each duplicate is kept, and the order of what survives is unchanged.
- [How to Clean a CSV Before Importing It Into a CRM or Database](https://dedupe.dev/guides/clean-csv-before-import): Two jobs, in this order. First make the file valid, because encoding, inconsistent column counts and a mismatched header are what actually cause an importer to reject it. Then make it clean: dedupe on the column that identifies a record, which for people is the email address and almost never the name.
- [Duplicate Removal Guides: Excel, Sheets, SQL, pandas, CRM & Photos](https://dedupe.dev/guides): Every guide starts with the method built into the platform you already have, whether that's a formula, a menu, or a preinstalled app. Only then does it cover where our free in-browser tools genuinely help. No fake urgency, no app installs where none are needed.
- [Dedupe a Google Sheet Automatically: Apps Script + API](https://dedupe.dev/guides/google-sheets-dedupe-api): For a one-off, use Data → Data cleanup or the Sheets guide. This page is the automated version: a
thirty-line Apps Script that adds a Dedupe.dev menu to your sheet and runs exact or fuzzy
matching through the API, so recurring cleanups become one click, or a time-driven
trigger with no clicks at all.
- [Dedupe in Zapier, Make and n8n: One HTTP Call, No App](https://dedupe.dev/guides/dedupe-zapier-make-n8n): The API is one POST, so no-code tools need no native app: Zapier's
Webhooks action, Make's HTTP module, and n8n's HTTP Request node each call it
directly. This page has the exact settings for all three and a
downloadable n8n workflow you can import as-is.
- [n8n Dedupe Workflows: Two Ready-Made Imports](https://dedupe.dev/guides/n8n-dedupe-workflow): Two workflows you can import as-is: a minimal
dedupe call (trigger, sample data, one HTTP request), and a
clean-list pipeline that deduplicates a mailing
list, runs email validation on the survivors, and outputs only the
addresses that passed. Import, paste your API key, replace the sample-data node with your source.
- [Data Cleaning Tools: What to Use for Which Problem](https://dedupe.dev/guides/data-cleaning-tools): Pick the tool by the problem, not the category. For a one-off list with duplicates, a focused browser tool finishes in seconds with nothing installed. For messy, exploratory cleanup — inconsistent spellings you need to see and cluster — OpenRefine is still unbeaten and free. For work that must be repeatable, pandas or dbt tests. Dedicated data quality platforms are for teams monitoring pipelines continuously, and are considerable overkill for a file.
- [Entity Resolution and Record Linkage, Explained Plainly](https://dedupe.dev/guides/entity-resolution): Entity resolution is deciding which records refer to the same real-world thing when nothing in the data says so directly. "J. Smith, 12 Oak St" and "Jane Smith, 12 Oak Street" are two rows and one person; concluding that is the whole problem. Deduplication is entity resolution inside a single dataset; record linkage is the same job across two, and comes from statistics rather than software. All three names describe one question: same thing, or different?

## Comparisons

- [Compare Dedupe.dev: Dedupely, Insycle, Dedupe.io, OpenRefine](https://dedupe.dev/compare): Every page here says plainly when the other tool is the better pick, because a comparison
that always concludes "buy us" is an ad. The short version: connected CRM apps win on in-place
merging, ML systems win on ambiguous record linkage, OpenRefine wins on hands-on control, the
built-ins win when duplicates are exact, and we win on being instant, local, and automatable.
- [Dedupe.dev vs Dedupely: CRM App or Local Dedupe?](https://dedupe.dev/compare/dedupely): Different architectures for the same mess. Dedupely connects to your CRM (HubSpot and
others) through OAuth, finds duplicate records, and merges them in place. Dedupe.dev never
sees your CRM: you export to CSV, deduplicate in your browser or over the API, and re-import. Pick by
what you trust and what you need merged.
- [Dedupe.dev vs Insycle: Platform or Focused Tool?](https://dedupe.dev/compare/insycle): Insycle is a data-management platform that lives inside HubSpot, Salesforce or
Pipedrive: dedupe plus standardization, bulk edits and scheduled automations, priced as a
subscription. Dedupe.dev does one job without touching your CRM: free local dedupe in the
browser, exact and fuzzy matching over the API. Platform versus focused tool is the real choice.
- [Dedupe.dev vs Dedupe.io: Threshold or Trained Model?](https://dedupe.dev/compare/dedupe-io): Similar names, different machinery. Dedupe.io is machine-learning entity resolution:
you label example pairs, it trains a model, and it clusters probable matches. Dedupe.dev is
deterministic: exact matching or normalization-plus-edit-distance with a threshold you control. For
genuinely ambiguous record linkage the model wins; for lists a human can eyeball, the threshold is
faster, cheaper, and explainable.
- [Dedupe.dev vs OpenRefine: Ten Seconds or Full Control?](https://dedupe.dev/compare/openrefine): OpenRefine is a free, open-source data-cleaning environment: install it, load your
dataset, and use faceting and clustering to find and merge near-duplicates with full interactive
control. Dedupe.dev is the ten-second version: drop a file in the browser, or POST it to the
API from a script. Both keep your data local for the interactive case; the difference is depth
versus immediacy.
- [Dedupe.dev vs Excel and Google Sheets Built-In Dedupe](https://dedupe.dev/compare/excel-google-sheets): Use the built-ins first: Data → Remove Duplicates in Excel and Data cleanup
in Google Sheets remove exact duplicates in place, free, no new tool. Reach past them when the
duplicates are not exact (case, punctuation, reordered names), when you want a cleaned copy instead
of an edited original, or when the job repeats often enough to automate.

## Trust and legal

- [Privacy Policy | Dedupe.dev](https://dedupe.dev/privacy): The tools on this site process your file in your browser. There is no upload endpoint for them, so there is no copy of your file to keep. What the server does hold is an account record and job metadata: counts, timings, modes and column names. This page says exactly what that means, and how to take it back or destroy it.
- [Terms of Service | Dedupe.dev](https://dedupe.dev/terms): The agreement between you and Zen Acquire LLC for using dedupe.dev. Written to be read: short sentences, no defined term you have to scroll back for, and no clause that says something different from what the software does.
- [Security at Dedupe.dev: Client-Side by Design](https://dedupe.dev/security): The strongest security property this service has is architectural: for the browser tools, your file never reaches a server, so there is no server-side copy to breach. That is a claim you can verify in about 60 seconds with your own browser, and this page tells you exactly how.
- [Contact Dedupe.dev](https://dedupe.dev/contact): One address for everything: support@dedupe.dev. It is read by a person, not a ticket robot, and there is no form on this page to fill in.

## Machine-readable API reference

- [Dedupe API reference in Markdown](https://dedupe.dev/api.md): the complete /api reference as plain Markdown. Endpoint, authentication, the three modes with every option and default, every error status with its exact response body, the billing meter, rate limits, recipes, and runnable samples in curl, Node, Python, Go and PHP.
- [Dedupe API OpenAPI 3.1 description](https://dedupe.dev/openapi.json): POST /api/dedupe described for tooling. Bearer and x-api-key security schemes, request schemas for all three modes, response schemas, and every error response with examples.

## Facts

- All processing is client-side JavaScript; there is no upload endpoint for the tools.
- First occurrence is kept and original row order is preserved everywhere.
- The in-browser tools are free, unmetered, and need no signup.
- The HTTP API is metered in rows: 1,000 a month free with a key, more on paid plans.
- The server stores METADATA ONLY: email, plan, hashed API key, and per-job counts, timings, match mode and column names. Never file contents or row values. See https://dedupe.dev/security and https://dedupe.dev/privacy.
- Users can export every stored row as JSON or CSV and delete their account outright, from https://dedupe.dev/app/settings.
- Last updated July 25, 2026.