GENIE: Gaussian Encoding for Neural Radiance Fields Interactive Editing
Neural Radiance Fields (NeRF) and Gaussian Splatting (GS) have recently
transformed 3D scene representation and rendering. NeRF achieves high-fidelity
novel view synthesis by learning volumetric representations through neural
networks, but its implicit encoding makes editing and physical interaction
challenging. In contrast, GS represents scenes as explicit collections of
Gaussian primitives, enabling real-time rendering, faster training, and more
intuitive manipulation. This explicit structure has made GS particularly
well-suited for interactive editing and integration with physics-based
simulation. In this paper, we introduce GENIE (Gaussian Encoding for Neural
Radiance Fields Interactive Editing), a hybrid model that combines the
photorealistic rendering quality of NeRF with the editable and structured
representation of GS. Instead of using spherical harmonics for appearance
modeling, we assign each Gaussian a trainable feature embedding. These
embeddings are used to condition a NeRF network based on the k nearest
Gaussians to each query point. To make this conditioning efficient, we
introduce Ray-Traced Gaussian Proximity Search (RT-GPS), a fast nearest
Gaussian search based on a modified ray-tracing pipeline. We also integrate a
multi-resolution hash grid to initialize and update Gaussian features.
Together, these components enable real-time, locality-aware editing: as
Gaussian primitives are repositioned or modified, their interpolated influence
is immediately reflected in the rendered output. By combining the strengths of
implicit and explicit representations, GENIE supports intuitive scene
manipulation, dynamic interaction, and compatibility with physical simulation,
bridging the gap between geometry-based editing and neural rendering. The code
can be found under (https://github.com/MikolajZielinski/genie)
689473b7741a16f544fbd06f
Mikołaj Zieliński
claimed_verified
2025-08-07T10:37:26.983Z
653064c7800ac851b9b6ac29
/avatars/f8167661d1c80e8a1bcf409c618d63a0.svg
Mikołaj Zieliński
user
MikolajZ
689473b7741a16f544fbd070
Krzysztof Byrski
689473b7741a16f544fbd071
Tomasz Szczepanik
689473b7741a16f544fbd072
Przemysław Spurek
claimed_verified
2025-08-13T07:27:55.844Z
670a4a2b64cab942c09769c3
https://cdn-avatars.huggingface.co/v1/production/uploads/670a4a2b64cab942c09769c3/gJzfbZJROl1Gd043wMcDw.png
Przemysław Spurek
user
przem85
GENIE: Gaussian Encoding for Neural Radiance Fields Interactive Editing
Neural Radiance Fields (NeRF) and Gaussian Splatting (GS) have recently
transformed 3D scene representation and rendering. NeRF achieves high-fidelity
novel view synthesis by learning volumetric representations through neural
networks, but its implicit encoding makes editing and physical interaction
challenging. In contrast, GS represents scenes as explicit collections of
Gaussian primitives, enabling real-time rendering, faster training, and more
intuitive manipulation. This explicit structure has made GS particularly
well-suited for interactive editing and integration with physics-based
simulation. In this paper, we introduce GENIE (Gaussian Encoding for Neural
Radiance Fields Interactive Editing), a hybrid model that combines the
photorealistic rendering quality of NeRF with the editable and structured
representation of GS. Instead of using spherical harmonics for appearance
modeling, we assign each Gaussian a trainable feature embedding. These
embeddings are used to condition a NeRF network based on the k nearest
Gaussians to each query point. To make this conditioning efficient, we
introduce Ray-Traced Gaussian Proximity Search (RT-GPS), a fast nearest
Gaussian search based on a modified ray-tracing pipeline. We also integrate a
multi-resolution hash grid to initialize and update Gaussian features.
Together, these components enable real-time, locality-aware editing: as
Gaussian primitives are repositioned or modified, their interpolated influence
is immediately reflected in the rendered output. By combining the strengths of
implicit and explicit representations, GENIE supports intuitive scene
manipulation, dynamic interaction, and compatibility with physical simulation,
bridging the gap between geometry-based editing and neural rendering. The code
can be found under (https://github.com/MikolajZielinski/genie)
GENIE combines NeRF's photorealistic rendering with Gaussian Splatting's editable and structured representation, enabling real-time, locality-aware editing and integration with physics-based simulation.
Neural Radiance Fields Gaussian Splatting volumetric representations neural networks implicit encoding explicit collections Gaussian primitives real-time rendering faster training intuitive manipulation interactive editing physics-based simulation Gaussian Encoding for Neural Radiance Fields Interactive Editing feature embedding NeRF network Ray-Traced Gaussian Proximity Search RT-GPS multi-resolution hash grid locality-aware editing scene manipulation dynamic interaction