Anyone else experimenting with Gaussian Splats – Videography and Pictures

I’ve been experimenting with making Gaussian Splats of our robotic. It’s a comparatively new sort of photo-scanning that isn’t mesh or geometry, however tens of millions of little blobs. It creates a 3d object (merchandise or setting) that has very small particulars captured, like wires, zipties, or hair (in case your robotic has hair…) and works with clear objects, and even view dependent reflections.

I’m probably not certain how helpful it’s, however I can’t deny that it’s cool. Maybe present it off within the pit if the robotic isn’t presently there, or simply usually preserve a scan of earlier robots as an archive. CAD fashions are nice, however they don’t have the wiring, the zip ties, the tape, the battle harm. They’re a super model of the robotic, however a Gaussian Splat captures it as it’s. This is mainly the last word robotic reveal.

Here’s our 2026 Unnamed Robot: 2102 – 2026 Robot Feb 22 – SuperSplat

It’s bought some animated monitoring, however you may simply seize it and rotate round and zoom nevertheless you need. I’ve an concept I’m doing to strive later in the present day to see if I can get full 360 protection of the underside too…

A couple of different splats I made:

The course of entails taking a video whereas orbiting across the robotic at totally different heights and getting near get particulars. It extracts images from it (a whole lot often), then does calculations evaluating them in an effort to get the placement, depth, and relation to one another to create a degree cloud, after which coaching it to get the best colours and sizes of the blobs. There are plenty of totally different variables you may tweak to get higher outcomes, however with my NVIDIA RTX 3090 graphics card, I can get respectable ends in about an hour of processing if I step down the decision, or I can use greater decision pictures and extra coaching and it’ll take many hours. At a sure level you simply get diminishing returns, and the one method to get higher high quality is to alter the video you seize, or change the processing so much. Aside from my very own laptop {hardware} and the electrical energy, this complete factor is free and self hosted (besides the splat viewer web site, which is free as nicely). It’s additionally very spectacular how, making it wants processing energy, however viewing it’s pretty light-weight (works in a browser).

Here’s the breakdown of the steps and know-how I’m utilizing for it (I run Linux on my desktop, and a variety of it’s constructed from nightly supply builds):

• Frame extraction: ffmpeg pulls particular person nonetheless frames from the video at common intervals, weighted towards the components of the video with extra close-up element.
• Blur filtering: Each body is scored for sharpness utilizing a Laplacian variance check — the underside 10% blurriest frames are discarded earlier than any additional processing.
• Feature extraction: SuperPoint, a neural community educated particularly for this job, identifies distinctive visible landmarks in every photograph. SuperPoint handles reflective, metallic, and low-texture surfaces rather more reliably than different strategies.
• Feature matching: LightGlue, one other neural community, figures out which landmarks seem throughout a number of images, linking overlapping frames collectively. It pairs with SuperPoint and considerably outperforms conventional matchers on tough surfaces just like the robotic’s chassis.
• Global reconstruction: COLMAP’s international mapper calculates all digicam positions and a tough 3D level cloud concurrently from the matched landmarks, with GPU-accelerated bundle adjustment for accuracy.
• Orientation alignment: Manhattan World Alignment rotates the 3D mannequin so flooring are flat and partitions are upright, giving it a pure orientation. Not strictly required, however useful for viewing.
• Gaussian Splat coaching: Brush locations tens of millions of tiny translucent blobs (Gaussians) in 3D house and repeatedly tweaks their form, coloration, and transparency till renders match the unique images. (I additionally tried LichtFeld Studio — it’s what was used for the 2025 splat — however Brush was considerably sooner.)

Honestly, the know-how is bettering so quick that higher instruments have come out and I’ve switched to them a number of occasions since I’ve began on this a couple of weeks in the past, it’s exhausting to maintain up.

I consider I can nonetheless get much more element if I dial this in additional. I’ve seen other splats with insane levels of detail, and I can’t inform if the distinction is dear cameras (versus my digicam’s cellphone), a greater processing pipeline, or simply that FRC robots space actually exhausting to seize this manner.

Is anybody else doing something like this, or ? Got any ideas for higher outcomes? Anyone know something attention-grabbing I can truly do with this?