Text Diff & Compare Tool

A diff tool is two decisions in a trenchcoat: which line of the second text corresponds to which line of the first, and what counts as the same line. Most tools make both silently.

The first is a Myers diff, the algorithm from the 1986 paper that git diff and Python difflib are built on, with patience anchors (the idea behind git diff --patience) for texts that are mostly different. That matters at size. The version this replaced built a full table of every line against every line, which is the square of the input in both time and memory whatever the two texts hold: measured as the comparison alone, 5,000 lines took 428 milliseconds and 10,000 took two seconds against a hundred million cells, and through the page it was far worse because the comparison also re-ran on every render rather than on every change. The same comparisons now take 8 milliseconds at 5,000 lines and 25 at 50,000, and 50,000 lines used to make the browser throw an allocation failure and take the whole tool down.

The second decision is five independent switches, each stated on the page. Line endings: a file written on Windows ends each line with a carriage return and a line feed, and comparing it against the same file written anywhere else marks every line as different unless you say the ending is not part of the line. Whitespace: exactly as written, ignoring space at the ends, or ignoring all of it, which is what makes a reindent invisible. Case: folded the Unicode way rather than by lower-casing, so the German sharp s equals ss, which lower-casing does not manage. Blank lines: left out of the comparison, and still shown, with the numbers they have in the file. Accents: an accented letter has two spellings in Unicode that look identical and are different strings, and NFC normalisation makes them equal.

A setting changes what is COMPARED and never what is shown. That sounds obvious and is easy to get wrong: dropping blank lines from an array and then using the array index as the line number renumbers every line after the first blank one, which is what happened here before this pass.

Inside a changed line you can highlight the difference by word or by character, and characters are grapheme clusters, so an emoji is one character rather than two halves of a surrogate pair. A block that moved rather than changed is pointed out: a line diff has no way to express a move, so both halves are found afterwards and labelled. The result carries a check that every line of both texts is accounted for, and the page shows it. The export is a real unified diff that patch can apply, over the whole comparison rather than the rows on screen; it used to be a file called diff.patch holding lines like "~ old (arrow) new" that nothing could apply.