Show HN: OmniDiff – Opensource, super fast, content aware Git difftool, in Rust (github.com)
242 points by FabCH 21 hours ago
I have been researching what the ultimate diff tool would look like, based on actual diffs in open source repositories.
OmniDiff can diff: code (25 languages), config, images, PDFs, archives, cursors, fonts...
It's fast. It can diff C++ files of tens of thousands of lines in under 100ms, on a 2023 ThinkPad T14s.
It's open source. AGPLv3.
It's based on real diffs. I looked at the 7400 repositories that are listed in Gentoo packages, sampled 2000 diffs and solved them manually to get a good understanding of what is needed, before putting together the diff pipelines for each content type. Incidentally, this also forms the basis of the CI pipeline, so the robustness of the codebase is quite high.
I'm still developing it. It's v0.3.0 at this time. But I have been using it daily as my git difftool for months now, and it works well.
AI Disclosure: Yes, I do use AI. No, I do not vibecode.
blahgeek 5 hours ago
> can diff 99.95% of code changes
I’m not sure if I’m understanding this correctly : so for the remaining 0.05% of code changes, this tool would just… miss it? That seems horrible for a diff tool :O
FabCH 3 hours ago
It will crash.
But so will every other tool, because it is things like a C++ header file with 128MB of PNG data hardcoded as an array.
Editors don’t even open those files.
xeyownt 3 hours ago
Fair enough.
But then I would rephrase that because it's confusing, and not doing justice to your tool.
FabCH 3 hours ago
grey-area 2 hours ago
Perhaps it could just check the length of files and not crash?!? Don’t you think that’s sensible?
FabCH an hour ago
dmitrygr 2 hours ago
> Editors don’t even open those files.
vi opens multi-GB files just fine.
And for multi-TB files, "less -n" works.
You need to broaden your horizons about editors. Not all of then are javascript-based slopfests
FabCH 2 hours ago
z0al an hour ago
How does it compare to Difftastic[1]? Both built on top of TreeSitter and solve similar problems.
FabCH 35 minutes ago
Comparing is slightly complicated but roughly speaking:
Difftastic only diffs code, which is only 2/3 of all diffs.
When only measured on stuff difftastic supports:
Difftastic is perfect on 64% of diffs and near-perfect on 17% more.
Omnidiff is perfect on 90% of diffs and near-perfect on 5% more.
Speed is
p50 - 35ms difftastic vs 4ms OmniDiff p99 - 637ms vs 420ms
Difftastic is MIT vs AGPL.
z0al 19 minutes ago
> Difftastic only diffs code, which is only 2/3 of all diffs.
Care to elaborate? What does Omnidiff offer for non-code files?
FabCH 3 minutes ago
BobbyTables2 7 hours ago
For me the biggest issue is “git rebase” merge conflicts because the surrounding context was modified (especially near beginning/end of functions where the context includes unrelated stuff), or similar in a markdown document.
(If commit A adds a line to the end of a bullet list in markdown and commit B does the same, they will conflict even though resolution is trivial for a human)
XYen0n an hour ago
You could try https://mergiraf.org/
eviks an hour ago
That would also be trivial for a syntax-tree-based tool, right?
zem 5 hours ago
the resolution depends on a deeper understanding of the contents of the list and whether they are ordered according to some scheme (in which case you will need to look at the two lines and see which one goes first)
gslepak 10 hours ago
Is there a way to use it with git without having to change your global git config? I.e. `git diff | omnidiff` or something?
You can do this with delta: `git diff | delta --side-by-side`
forgotpwd16 8 hours ago
https://github.com/ivankovic/omnidiff#usage
OmniDiff also works directly with git diff and git log -p, through GIT_EXTERNAL_DIFF. This path needs no difftool config:
GIT_EXTERNAL_DIFF=omnidiff git diffnh2 9 hours ago
Piping the output of `git diff` might withhold information from the tool that it needs for its feature set, while `delta` is just a different way to display any diff output.
davvid 14 hours ago
Nicely done OP. Does it do 3-way diffs for merges as well? The instructions only mention the two-way `git difftool` configuration so maybe that'll be a future feature?
Any interest in getting the setup upstreamed[1] so that users only need to `git config --global diff.tool omnidiff` to enable it by default? Disclosure: I created `git difftool` a long time ago, and am always happy when builtin support for useful tools are added to it. Feel free to reach out if you'd like help making that happen.
FabCH 14 hours ago
At the moment the algorithm for code diffing is a particularly efficient implementation of APTED which is a tree diffing algorithm that doesn’t natively construct anything useful for a three way diff. I have a notebook with a bunch of failed attempts at a good algorithm for a three way merge, but so far nothing particularly impressive.
It’s a long term goal to solve, but it’s the hardest one.
I’ll definitely reach out to you to upstream. Thank you for the kind offer!
catlifeonmars 9 hours ago
How do you handle degenerate trees? I’m curious what heuristics are used in practice to eg backtrack or whatever. I’m not very well versed in TED but superficially the problem seems to resemble wave function collapse.
FabCH 3 hours ago
enriquto 11 hours ago
cool to see fresh Rust software with copyleft license !
I feel safer, as a user, enjoying the freedoms that the AGPL provides me. (No evil middleman will be able to give me non-free versions of this program.)
eviks an hour ago
But an evil midllman will relatively easily be able to give you non-free version, so that feeling should be fleeting
FabCH an hour ago
As long as they open source all modifications and follow AGPL to the letter so that anyone can self-host the version they are selling, I’m ok with that.
AGPL totally allows commercial use. You just have to give back.
lejalv 2 hours ago
So much this. Thanks OP!
nickjj 16 hours ago
Delta has a really nice way to show diffs with its diff-so-fancy mode, it's more similar to GitHub's UI where not only the line is colored up differently but there's a different shade of color to highlight the characters.
There's a screenshot and video of it here that I wrote a while back: https://nickjanetakis.com/blog/awesome-git-diffs-with-delta-...
In OmniDiff's readme and web version I didn't see a way to do that.
In a different comment you mentioned OmniDiff is much better than Delta and you mentioned "About a third of diffs are significantly better.". Can you provide some examples of these? I've never seen Delta fail a diff.
Also Delta has a package for Arch in its official repos. For folks who just want a "it works" experience, that's a pretty big win. With Delta the same package can be installed in the same way on Debian, Arch and macOS with Brew using standard package managers. With OmniDiff, there's 3 different ways to install it (manual deb package on Debian, manual package compile on Arch and brew install on a Mac). Just something to think about, as someone who has public dotfiles that work in those 3 environments, I'd skip OmniDiff based on this alone.
FabCH 15 hours ago
OmniDiff is character based by default. It will only highlight the changed characters. I suppose I could add an option to additionally highlight the line as well. Thanks for the feedback!
As for packaging. Unfortunately AUR is closed at the moment due to spam. I can add brew commands for Linux, happily.
In terms of “better”. It’s actually somewhat complex, but a simple way to think about it is that if you move a function up or down in the same class, text-based diffs paint that as “Deleted here, Inserted there” where ideally you would want it to say “Moved”.
bita_nidir 15 hours ago
Wow, I've often wished I had a tool like this. Thanks!
Since most people do click-and-run most of the time (based on decades long research, n=1), I'll just add paste the link for more examples: https://ivankovic.github.io/omnidiff/showcase/content.html
FabCH 15 hours ago
andunie 13 hours ago
This looks really good. It does look like you put a ton of effort into it, I'll give it a try.
I feel like the moved code cases would look great in https://github.com/andunieee/twain, I was looking into how I could slurp your code and make it work there (although for some reason I have the impression that I had already seen this repository in the past, I can't remember where).
andunie 10 hours ago
I don't know what I did wrong, but using this as a git difftool (following the config) I didn't get any of the cool things from the demos, I got a weird single-column diff output that only showed the new side of the changes, not a diff.
FabCH 3 hours ago
Perhaps the terminal is too small? What happens when you hit tab or “v”?
arkenflame 16 hours ago
Really well done, I like this a lot. It produces much more minimal diffs than other tools I've tried.
One feature I miss from difftastic is showing diffs side-by-side on wide terminals, and unified when narrow.
FabCH 16 hours ago
It should… it considers wide to be 200+ columns.
You can force it with “v”.
I am not super set on that value, if the community has a better default for “wide”, happy to modify it.
cocoto 18 hours ago
How does it compare with diffstatic?
FabCH 18 hours ago
Comparing is slightly complicated but roughly speaking:
Difftastic only diffs code, which is only 2/3 of all diffs.
When only measured on stuff difftastic supports:
Difftastic is perfect on 64% of diffs and near-perfect on 17% more.
Omnidiff is perfect on 90% of diffs and near-perfect on 5% more.
Speed is
p50 - 35ms difftastic vs 4ms OmniDiff p99 - 637ms vs 420ms
Difftastic wins on license. MIT vs AGPL.
LVB 16 hours ago
> Difftastic wins on license. MIT vs AGPL.
I'm a little confused on that, since in other comments you've mentioned that AGPL was a deliberate choice for <reasons>. So, what's the definitions of "winning" here?
FabCH 16 hours ago
folmar 10 hours ago
anilgulecha 18 hours ago
You have a good core idea here. You're pretty much killing your project in infancy with the AGPL+donation license model. Meanwhile, some other projects will take the core idea of "language aware differ" and release BSD, and one of those will win, including perhaps making it into git/future version controls.
IMO, you should seek to make this a standard that wins, and hopefully other opportunities that come will be where you make commercial hay. My unsolicited 2c.
FabCH 18 hours ago
I really do care about copyleft and AGPL. It’s a political choice. I specifically chose it because I know from direct experience that big tech is allergic to it.
I actually don’t care about the commercial license. It’s more of an experiment for my own amusement than an actual attempt at building a commercial project.
If it inspires someone to build another one and we still end up with a better differ, I’m ok with that world. As long as the open source community has a better differ.
Note that there already are at least 3 that I am aware of. But they do perform worse on speed and accuracy.
ipsi 18 hours ago
FWIW, there's probably no point in using a copyleft licence if AI is involved[1][2], unless you've got pretty solid proof AI only "assisted" rather than doing the bulk of the work. Otherwise, you don't have copyright on the generated code, and without copyright the licence is meaningless, AFAIK.
This is not to say you can't or shouldn't use AI, but it's important to understand the tradeoffs you're making.
[1] https://www.scotusblog.com/cases/thaler-v-perlmutter/ [2] https://www.jenner.com/en/news-insights/client-alerts/us-cop...
FabCH 18 hours ago
noir_lord 12 hours ago
shock 15 hours ago
I support the choice of AGPL. I wish more software used it. Whenever I use AI to write code, in my experience it invariably chooses the MIT license. For example, for all the ansible roles that I've written using AI it put the MIT license in the meta file, unless otherwise prompted. It makes me wonder if it comes from RLHF.
IshKebab 17 hours ago
> Meanwhile, some other projects will take the core idea of "language aware differ" and release BSD, and one of those will win, including perhaps making it into git/future version controls.
Diffsitter and Difftastic have been around for a while, both MIT, so I doubt it.
paaloeye 20 hours ago
How is it better or worse compared to https://github.com/dandavison/delta ?
FabCH 20 hours ago
It’s much much better with moved code.
Delta adds color to the standard git diff algorithm.
Omnidiff uses a completely different algorithm.
About a third of diffs are significantly better.
eviks an hour ago
Do you have any examples illustrating how it's better?
paaloeye 20 hours ago
Okay, I'll give it a try then.
paaloeye 20 hours ago
Older submission: https://news.ycombinator.com/item?id=49891989
FabCH 20 hours ago
Not quite.
In the meantime, I added support for all other non-code file formats. Hence the name change.
It’s quite a big chunk of functionality.
smallnix 13 hours ago
Very cool! I'd love to see support for figma or Photoshop, perhaps it's already possible via export to an image format, but native would be cool.
FabCH 13 hours ago
Depending on how proprietary their formats are, it could be done.
burner-account 13 hours ago
I am sorry, I looked through the examples and I am not sure I prefer the output of omnidiff... Seeing how everyone else over here is instead overly enthusiastic makes me feel extremely dumb :D
FabCH an hour ago
Don't feel dumb. You are just an outlier on the visual preferences.
I actually do TWO solutions for each test case: a minimal diff and a maximal diff. Pressing 'm' in the TUI pops up a settings page where you can choose how the diff works.
I am actively researching how many ways there are to "paint" the same diff and user preferences and non-uniqueness of diffs is something I am super interested in.
If you would like to, it would be super useful for me to know what diff you don't like and why. It would help make the diff better for users with different visual preferences.
keepupnow 7 hours ago
[dead]
paravz 10 hours ago
[dead]
keepupnow 7 hours ago
[dead]
Surac 18 hours ago
[flagged]
hebetude 18 hours ago
[flagged]
cassianoleal 17 hours ago
> with licenses nobody wants to use
Nobody wants to use AGPL-v3?
fishgoesblub 11 hours ago
Some people think there's no point if you're not allowed to steal the code and not give back
roarcher 6 hours ago
alexpadula 17 hours ago
You know hebetube, my mother once said "if you have nothing nice to say: don't say it". You saying those things probably aren't going to make them turn over the license. "Project dead before it started" like who writes those kinds of things over someones work they spent time on. I'd love for you to explain your logic.