WorldMaker/tokdiff
View on GitHub ↗ ★ 6 Python
Tokenizer-based character diff tool
14 mentions on Hacker News by 1 person: 0 stories, 14 comments · first 2015-11-09 · latest 2026-02-16
20082026
-
…as many benefits as a git mergetool than it might in a patch theory VCS like darcs or pijul.) [1] https://github.com/WorldMaker/tokdiff
comment by WorldMaker · 2026-02-16 · in SCM as a database for the code -
…a syntax highlighting tuned lexer to build character-level diffs showed a lot of great promise: https://github.com/WorldMaker/tokdiff
comment by WorldMaker · 2025-07-11 · in Diffsitter – A Tree-sitter based AST difftool to get meaningful semantic diffs -
…passing by on the way to that ideal mountain (lexer-based diffs rather than parser-based diffs): https://github.com/WorldMaker/tokdiff
comment by WorldMaker · 2022-07-05 · in What comes after Git -
…"blank email" that needs a greetings and salutation and maybe a description of the attachment.) [0] https://github.com/WorldMaker/tokdiff
comment by WorldMaker · 2022-07-05 · in What comes after Git -
…yet such as work-in-progress code (again because we use them all the time in text editors). https://github.com/WorldMaker/tokdiff
comment by WorldMaker · 2021-06-30 · in Ignoring bulk change commits with Git blame (2019) -
…further with it now that one of the patch algebra VCSes has finally adopted a "chunk" option. [1] https://github.com/WorldMaker/tokdiff (ETA: Suppose the first step is that it is time to get the band back together and update the prototype…
comment by WorldMaker · 2020-11-09 · in Pijul: Towards 1.0 -
…did, this tool would potentially be quite powerful there.) It's a neat toy/experiment though. [1] https://github.com/WorldMaker/tokdiff
comment by WorldMaker · 2020-07-21 · in Syntax highlighting is a waste of an information channel -
…as a storage format which the experiment was partly to prove both ideas could be useful.) [1] https://github.com/WorldMaker/tokdiff
comment by WorldMaker · 2020-01-07 · in Is Git Irreplaceable? (2019) -
…highlighting tokenizers, which perform well and interoperate well with character-based diffs: https://github.com/WorldMaker/tokdiff )
comment by WorldMaker · 2018-07-30 · in Elastic TabStops: A Better Way to Indent and Align Code (2017) -
…someone actually use it for source control, but the core idea is simple enough for the taking. [1] https://github.com/WorldMaker/tokdiff
comment by WorldMaker · 2018-07-09 · in Version Control Before Git with CVS -
…tool, but still to date I've yet to really see it come into general usage outside my prototype toy ( http://github.com/WorldMaker/tokdiff ).
comment by WorldMaker · 2017-04-17 · in Rope Science – Computer science concepts behind the Xi editor -
…like Markdown creates okay diffs in a line-based diff tool. I did some work with tokenized diffs: https://github.com/WorldMaker/tokdiff That particular tool/experiment uses the Pygments tokenizer used for syntax highlighting and produces…
comment by WorldMaker · 2017-03-29 · in Making a good diff algorithm -
…diffs for something like git, though I've suggested trying it before. Python code, if interested: https://github.com/WorldMaker/tokdiff
comment by WorldMaker · 2016-10-17 · in Noms – A versioned, forkable, syncable database -
…The tokenized diffs were cleaner and faster than the equivalent non-tokenized character diff. https://github.com/WorldMaker/tokdiff if anyone wants to play with it.)
comment by WorldMaker · 2015-11-09 · in EditorConfig