Remove duplicate lines

Keep the first occurrence of each line and remove later matches without reordering the list.

Runs in your browser. Your text is not uploaded.

Worked example

Before

pear
apple
pear

After

pear
apple

The second “pear” is omitted; the first copy and the final line ending remain.

Exactly what this changes

  • The default comparison uses every character exactly as entered. Letter case and surrounding spaces matter, so “Pear” and “pear” remain separate values, as do “pear” and “ pear”. The retained line is always the first original occurrence.
  • Order stays intact. PlainSift walks from the first line to the last, records each comparison value, and omits only later matches. It does not alphabetize the result, select the shortest spelling, or rewrite the line that remains.
  • Blank lines participate in the same comparison as text lines. With exact comparison selected, the first blank line remains and later blank lines are removed. A final line ending is tracked separately and remains present when the source had one.

Options and edge cases

  • Turn off case sensitivity when capitalized and lowercase forms should count as the same value. Select trimmed comparison to ignore surrounding whitespace only while checking; the saved first line itself is not trimmed.
  • Combine the two options to compare a trimmed, lowercase key while preserving the exact first spelling and spacing. No Unicode normalization is applied, so composed and combining-character forms stay distinct.

Edge case: This operation compares whole lines, not words within a line. Two lines that merely contain similar text are not duplicates, and no punctuation or internal whitespace is removed.