Generalizable Hand-Object Modeling from Unconstrained RGB Images via 3D Gaussians
Xingyu Liu, Pengfei Ren, Qi Qi, Haifeng Sun, Zirui Zhuang, Jing Wang, Jianxin Liao, Jingyu Wang
Recent advances in hand–object interaction modeling have employed implicit representations, such as Signed Distance Functions (SDF), Neural Radiance Fields (NeRF), and 3D Gaussian Splatting (3DGS), to reconstruct hands and objects with arbitrary topology and photorealistic detail. However, these methods often rely on dense 3D surface annotations, or are tailored to short clips constrained in motion trajectories and scene contexts, limiting their generalization to diverse environments and movement patterns. In this work, we present HOGS, an adaptively perceptive 3DGS-based framework for generalizable hand–object modeling from unconstrained monocular RGB images. By integrating photometric cues from the visual modality with the physically grounded structure of 3D Gaussians, HOGS disentangles inherent geometry from transient lighting and motion-induced appearance changes. This endows implicit representations with the ability to generalize to unseen environments and dynamic motion patterns. Experiments on two challenging datasets demonstrate that HOGS outperforms state-of-the-art methods in monocular hand-object reconstruction and photo-realistic rendering.