Gracia Captures Realistic Volumetric Scenes With Gaussian Splats (Logan Lemmon And Kazii Botashev)

The most impressive demo I tried at Augmented World Expo in Long Beach earlier this year was a set of scenes captured by Gracia.

The scenes use Gaussian splatting to deliver volumetric video that far surpasses anything else I’ve seen in the space. The scenes ran on Apple Vision Pro in passthrough mode so I was looking around at the bustling convention center show floor with a musical performer inserted at 1:1 scale anchored to the floor. Vision Pro’s leading display resolution combined with Gracia’s impressive representation made it hard for my brain to discern who in front of me was physically present and who was being teleported in as the moving Gaussian splat.

Lead Gracia researcher Kazii Botashev joined from Kazakhstan with the U.S.-based head of partnerships Logan Lemmon in Venice to speak on the Good VR Podcast covering the rise of Gaussian splatting. It’s currently the leading technology for representing a scene volumetrically and our conversation covers their popularization from around 2023 to the question of how they might reduce the technical overhead of using a hundred or more cameras in making these scenes.

“The main rule is that [the] bigger [the] number of cameras is, [the] better quality that you can achieve,” Botashev said. “So the general direction that all the research currently working on is decreasing number of cameras.”

Lemmon suggested that the first impacts of 4DGS, the shorthand term for Gaussian splats that move over time, are likely to be felt across visual effects. Gracia recently worked with Muse and lead singer Matt Bellamy to capture him and represent him using the tech for a music video.

“The VFX world is just now starting to understand Gaussian splatting,” Lemmon said. “You can capture a 4D sequence and you can process it with us, and then you can slice that sequence into PLYs, which are very, very large point cloud files that you can have interpreted by Nuke or Maya or any sort of 3D software. We just recently worked with the band Muse and the lead singer Matt Bellamy… our partners over at Radiant Images and us, we produced this music video alongside some really wonderful people out of Kazakhstan. The result was a 2D render of Matt Bellamy in a music video, but creators had full three-dimensional control over where the camera was placed, the animation of each of the points of his point cloud. They could explode him, they could zoom into his head, they could place him anywhere in the scene at a very high quality in this very stylized look. So we’re seeing more and more use cases for VFX, and we’re going to start seeing virtual production as well become a major vertical for 4DGS utilization.”

You can donate any amount via Stripe to support the work of Good Virtual Reality and thank you to my paid Substack subscribers for supporting independent journalism.

https://buy.stripe.com/7sY8wPeTAcnG3eD1vZao800

Get full access to Good Virtual Reality at www.goodvirtualreality.com/subscribe