Remove Duplicate Lines
Text Tools
Strip duplicate lines from any text. Case-sensitive matching, whitespace trimming, keep-first or keep-last, and optional sorting.
Runs entirely in your browserAbout Remove Duplicate Lines
Deduplicating lines is one of those small chores that comes up constantly — cleaning up a list of emails, deduping log entries, normalizing a CSV column, or finding the unique items in any line-separated dataset. Doing it by hand is tedious; doing it with a one-liner on the command line works but isn't always handy.
This tool gives you the four options that actually matter: case sensitivity (treat `Apple` and `apple` as the same or different), whitespace trimming (so ` apple` and `apple` match), keep-first or keep-last (which occurrence wins when there are duplicates — matters for ordered data), and optional alphabetical sorting. A live readout shows how many lines were kept and how many removed, so you can confirm the operation did what you expected. Everything runs in your browser.
How to use
- 1
Paste your lines
Drop the text into the input area. One item per line.
- 2
Set the comparison rules
Toggle Case sensitive to differentiate `Apple` from `apple`. Toggle Trim whitespace to ignore leading/trailing spaces. Together these decide what counts as a duplicate.
- 3
Pick keep direction
Keep First preserves the order of first appearance. Keep Last keeps the last occurrence — useful when later entries supersede earlier ones.
- 4
Optionally sort alphabetically
Toggle Sort to alphabetize the output after deduplication.
- 5
Grab the result
Use Copy or Download (.txt). The label shows X kept · Y removed.
Examples
Case-insensitive deduplication
With case sensitivity off, Apple and apple are the same item.
Input
apple
Banana
apple
APPLE
bananaOutput
apple
BananaKeep last, sorted
Last occurrence wins; output is alphabetized.
Input
bravo
alpha
charlie
alpha
deltaOutput
alpha
bravo
charlie
deltaFrequently asked questions
Does it handle very large files?+
Yes — tens of thousands of lines deduplicate instantly. For millions of lines you might notice a brief pause as the browser processes the input.
How is order preserved?+
When Keep First is selected (default), the first occurrence wins and the original order is maintained. Keep Last reverses this. Toggle Sort to alphabetize afterward.
What happens to empty lines?+
Empty lines are treated like any other value — duplicates of empty lines collapse to one empty line. If you don't want any empty lines in the output, trim them first.
Can I deduplicate by a substring?+
Not in this tool — it compares whole lines. For substring-based deduplication (e.g. unique email addresses from a 'Name <email>' list), pre-process the data first.
Is my text sent anywhere?+
No. All processing happens in your browser.
Related tools
Word Counter
Count characters, words, sentences, and paragraphs with live reading-time estimates.
Case Converter
Convert text between upper, lower, title, camelCase, snake_case, kebab-case, and more.
Text Diff Checker
Compare two pieces of text and highlight differences inline or side by side.
Lorem Ipsum Generator
Generate placeholder text in classic, hipster, or corporate dialects.
Markdown Editor
Write Markdown with a live preview pane, formatting toolbar, autosaved drafts, and .md / .html export.