Better Gaussian Splatting in Julia (pxl-th.github.io)
106 points by pxl-th 4 days ago
Cieric 4 hours ago
I'm still very fascinated by gaussian splatting, but the issue I always run into is the need to use colmap to generate the camera positions and get the initial point cloud. Which colmap effectively still needs Nvidia for dense point clouds last I checked. I'm curious though does anyone have anything else they've tried that worked well? I've tried a few different things (most of the time getting stuck on just compiling the thing) so I'm curious if anyone has any suggestions that I should focus more heavily on getting working. I have some poor quality videos I'd like to try and use and I'm willing to put in some manual work, but it feels like it's all automatic or nothing. Thanks for any suggestions that any of you have.
corysama 2 hours ago
I know there have been papers and research projects about GS without COLMAP and GS from videos. But, I don't have links handy.
Best I can do is link you to https://x.com/RadianceFields and https://radiancefields.com/ They have all the news about GS every day.
Cieric 2 hours ago
Thanks, I've been exploring a lot of the papers and seeing what I can get working. Some of it really has been interesting, but I just never seem to be able to get it to work well. I might just need to throw myself at it all a bit more. Thanks for the links though, I'll check them out.
ghostly_s 2 hours ago
TFA mentions this project has backends for :
AMD GPU (AMDGPU.jl) NVIDIA GPU (CUDA.jl) MacBook GPU(Metal.jl)
Cieric an hour ago
I'm sorry, but the article doesn't cover anything about the preprocessing steps that are required for 90% of gaussian splat implementations. That preprocessing step is almost always colmap which takes pictures or videos and uses SfM to figure out the camera parameters and their position in the 3d scene, it then does basic point reconstruction and dense point reconstruction. That info is what gaussian splats take in, it converts the points from a colmap project into gaussians and then optimizes them through differentiable rendering using backprop to optimize the gaussians to better match the images at the calculated positions of the camera from colmap. Looking over the project files also shows that they consume colmap projects, my question wasn't about the gaussian splat implementation, it was about the defacto standard colmap preprocessing step that is also required.
hdz 8 hours ago
Having this work on Apple metal is huge for the accessibility around training splats. I remember two years ago I needed an RTX Nvidia GPU to run things like splatfacto on nerfstudio. Very neat optimizations as well like the depth map and sky dome, 3dgs has come a long way. Awesome!
mourner 7 hours ago
Oh, a fellow Ukrainian! Was pleasantly surprised to see that fountain on HN — I was just walking around it a few days ago, very nice and peaceful spot this time of year.
pxl-th an hour ago
Дякую! :)
fredley 8 hours ago
I thought the picture on your home page was very cool, and then I moused over it. Absolutely incredible.
miguelraz 5 hours ago
Fantastic demos in Julia, what a treat.
xchip 4 hours ago
Hey Anton, great work as always :)
pxl-th an hour ago
Hi Christian! Thank you! :)
thimotedupuch 8 hours ago
Great work
aithena 9 hours ago
this is amazing!
chrisjj 7 hours ago
Better? A turn on the spot shows a wreck everywhere but forward.
yorwba 5 hours ago
Those are the areas that are occluded in the input images. If you look at the bushes, you can see that the front is clean while the back is a mess. Probably the starting position could be chosen a bit better.
ghostly_s 2 hours ago
This raises the question, do any of these projects have workflows for combining images taken from different positions? Or does that just work out of the box? (I have no machines atm with anything better than 10yo eGPUs ;~<)
yorwba 24 minutes ago
jaccola 2 hours ago
chrisjj 4 hours ago
The path to left and right is occluded? By what?
ChrisClark 3 hours ago
pwf 5 hours ago
do you know how gaussian splatting works?