What OpenRefine is
OpenRefine (formerly Google Refine) is the standard open-source power tool for messy tabular data. It runs locally, so nothing is uploaded, and its clustering (key collision with several keying functions, nearest neighbor with edit distance) finds near-duplicate values with more configurability than our single threshold. Around the dedupe it adds facets, transformations in its own expression language, and reconciliation. It is the tool we would point a data journalist at.
Where the approaches split
OpenRefine is an environment; we are an operation. The cost of the environment is install, a learning curve, and hands-on time per dataset: every cluster is reviewed and merged interactively, which is a feature for careful one-off work and a chore for the hundredth weekly export. The operation costs you control: the browser tools are exact-match with column selection, the API adds threshold fuzzy matching, and neither will ever show you a cluster for review. Use OpenRefine when the dataset deserves an afternoon. Use us when the answer should exist before your coffee cools, or when no human is in the loop at all because a pipeline is doing the cleaning.
How Dedupe.dev works, for contrast
Two halves. The browser tools are free and unmetered: your file is parsed and deduplicated by JavaScript in the tab, nothing is uploaded, and you can watch the empty Network panel while it runs. The HTTP API is for automation: exact or fuzzy matching, metered in rows (1,000 a month free with a key, 100,000 on Starter at $19/mo, 300,000 on Pro at $49/mo), with fuzzy requests up to 50,000 rows each. Duplicates are removed, never merged: the first occurrence survives, and re-importing the cleaned file is your step.
FAQ
Frequently asked questions
Is OpenRefine free?
Yes, genuinely: free and open source, it runs on your own machine (data stays local, like our browser tools), and it does far more than dedupe: faceting, clustering, transformations, reconciliation against external sources.
Why use Dedupe.dev if OpenRefine exists?
Time to done and automation. OpenRefine is an environment you install and learn; its clustering is powerful but is a hands-on, review-each-cluster workflow. Dropping a file on a browser tool takes ten seconds and no install, and the API automates the same operation in a pipeline, which OpenRefine is not built for.
Can OpenRefine do fuzzy matching?
Yes, well: its key-collision and nearest-neighbor clustering methods are a superset of what most people need, with interactive review of every cluster before merging. If you enjoy that control and the job is one big messy dataset, OpenRefine is the better instrument.
Last updated