I made a build visualizer to understand Bun's compile times (lalitm.com)
107 points by lalitmaganti 13 hours ago
anaqin 10 hours ago
Good write up! I was hoping it was going to conclude with the author getting buns zig build way faster than Rusts but a good deep dive nevertheless
lalitmaganti 9 hours ago
Having spent ~3 weeks of evenings and weekends on this, I was a bit burned out to keep trying to optimize especially because all of this code is just a dead end anyway...
The fix seemed to be to split up Bun's Zig module into ~100 pieces just like they've done on Rust but how representative that would have been on anything Bun might of shipped is very much in question!
pfg_ 4 hours ago
The issue with splitting up Bun's Zig module is that Rust is designed for multiple crates compiling separate objects and linking them together in the end, but for Zig you would need to manually export and import C abi functions and you can't make use of language features like slices, generics, non-extern structs, and others anymore. In Zig, every module is compiled in the same compilation unit. Even the standard library is compiled in the same unit as your app.
bloaf 7 hours ago
The next questions in my head are:
A) Is there any performance penalty on the compiled result due to being optimized in separate pieces
B) If not, why isn't the optimizer smart enough to split them up internally and optimize in parallel?
lalitmaganti 6 hours ago
anaqin 8 hours ago
Your post went into way more depth than I was expecting when I first opened up it up. You clearly spent a lot of time on it and I appreciated reading the tale so thank you. Given all the drama around AK / Bun, it just would have been so funny to see someone not involved in the project solve one of the main motivations for the move to rust.
I believe there is a fork of bun from 1.3 that a different community is maintaining. I wonder if they’ve been able to get the compile times down by modularizing the build?
listic 4 hours ago
rahkiin 9 hours ago
Can you force a non-parallel build of those rust crates to make the comparison fairer?
lalitmaganti 7 hours ago
Retro_Dev 9 hours ago
xcc3641 2 hours ago
Full LTO on the WebKit link is the serial part. Does your visualizer split link time from codegen?
lalitmaganti 2 hours ago
Yes the whole article is about exactly that :)
You can also use --compiler-traces to understand the inside of the linking process as well; these annotations are provided by lld.
t43562 9 hours ago
It's like Electric Insight, which was a fantastic if proprietary tool.
There's an enormous amount of analysis you can do with tools like this from estimating how much faster the build would be with more cores (i.e. is it worth adding more) to things like a diff of one build against another: why is one build bad and the other good - what tasks had different parameters or weren't in both builds?
emelski 2 hours ago
Hey, I created ElectricInsight! It's nice to hear somebody besides me remembers it fondly.
lalitmaganti 7 hours ago
Diffing is something I definitely want to support. I've filed https://github.com/LalitMaganti/buildprof/issues/12 to add support for this, I have ideas!
flimflamm 10 hours ago
It would be interesting to see what would be good input for LLMs to do the optimizations / trials on that automatically. Is the visual representation best or something else?
lalitmaganti 10 hours ago
It's actually trivially easy to add a "automated report" feature to buildprof due to its architecture (just a few SQL queries on top of Perfetto's trace processor). And I'm sure AI could hill climb the build time based on this.
But I still think the visual view is invaluable for human understanding and for the "why is this even happening" problems which I think AI is still bad at seeing.
Conlectus 9 hours ago
This is just how hacker news is now, huh?
“I did this cool useful thing”
“Great! Please describe how I can get a machine to replace you.”
applfanboysbgon 5 hours ago
Excellent write-up, love to see profiling work on compile times. I have some rudimentary tooling I threw together myself but nothing that looks as comprehensive and insightfully presented as this, I'll be giving it a try!
applfanboysbgon an hour ago
Update: this tool allowed me to almost immediately diagnose a compile time mystery I encountered yesterday that I hadn't yet been able to identify with my own tools. Great work!