IBGS: Image-Based Gaussian Splatting
Hoang Chuong Nguyen, Wei Mao, Jose M. Alvarez, miaomiao Liu
3D Gaussian Splatting (3DGS) has recently emerged as a fast, high-quality method for novel view synthesis (NVS). However, its use of low-degree spherical harmonics limits its ability to capture spatially varying color and view-dependent effects like specular highlights. Existing work augments Gaussians with either a global texture map, which struggles with complex scenes, or per-Gaussian textures, which introduce high memory overhead and limit resolution. We propose Image-Based Gaussian Splatting, an efficient alternative that leverages high-resolution source images for fine details and view-specific color modeling. Specifically, we model each pixel color as a combination of a base color from standard 3DGS rendering and a learned residual, inferred from neighboring training images. This promotes accurate surface alignment and enables rendering images of high-frequency details and accurate view-dependent effects. Experiments on standard NVS benchmarks show that our method significantly outperforms prior Gaussian Splatting approaches in rendering quality, without increasing the storage footprint.