VeGaS: Video Gaussian Splatting
Implicit Neural Representations (INRs) employ neural networks to approximate
discrete data as continuous functions. In the context of video data, such
models can be utilized to transform the coordinates of pixel locations along
with frame occurrence times (or indices) into RGB color values. Although INRs
facilitate effective compression, they are unsuitable for editing purposes. One
potential solution is to use a 3D Gaussian Splatting (3DGS) based model, such
as the Video Gaussian Representation (VGR), which is capable of encoding video
as a multitude of 3D Gaussians and is applicable for numerous video processing
operations, including editing. Nevertheless, in this case, the capacity for
modification is constrained to a limited set of basic transformations. To
address this issue, we introduce the Video Gaussian Splatting (VeGaS) model,
which enables realistic modifications of video data. To construct VeGaS, we
propose a novel family of Folded-Gaussian distributions designed to capture
nonlinear dynamics in a video stream and model consecutive frames by 2D
Gaussians obtained as respective conditional distributions. Our experiments
demonstrate that VeGaS outperforms state-of-the-art solutions in frame
reconstruction tasks and allows realistic modifications of video data. The code
is available at: https://github.com/gmum/VeGaS.
673c8d3a44ea7d8aeaa3ef3e
Weronika Smolak-Dyżewska
673c8d3a44ea7d8aeaa3ef3f
Dawid Malarz
admin_assigned
2024-11-19T16:26:14.135Z
6303d84b7373aacccd867994
/avatars/c600374518284ba9c50899dde00bdf28.svg
Dawid Malarz
user
MalarzDawid
673c8d3a44ea7d8aeaa3ef40
Kornel Howil
claimed_verified
2024-11-19T13:09:06.082Z
65fc24b804769daf21a98fa6
/avatars/d7faf536940809823d56a442f55e9fb9.svg
Kornel
user
kornelhowil
673c8d3a44ea7d8aeaa3ef41
Jan Kaczmarczyk
claimed_verified
2024-11-20T10:08:59.307Z
6309f63cb70f9eb690f6d4aa
/avatars/341e45c84887577916b5ee99516a0e34.svg
Jan Kaczmarczyk
user
JasiekKaczmarczyk
673c8d3a44ea7d8aeaa3ef42
Marcin Mazur
673c8d3a44ea7d8aeaa3ef43
Przemysław Spurek
admin_assigned
2024-11-19T16:26:57.859Z
670a4a2b64cab942c09769c3
https://cdn-avatars.huggingface.co/v1/production/uploads/670a4a2b64cab942c09769c3/gJzfbZJROl1Gd043wMcDw.png
Przemysław Spurek
user
przem85
VeGaS: Video Gaussian Splatting
Implicit Neural Representations (INRs) employ neural networks to approximate
discrete data as continuous functions. In the context of video data, such
models can be utilized to transform the coordinates of pixel locations along
with frame occurrence times (or indices) into RGB color values. Although INRs
facilitate effective compression, they are unsuitable for editing purposes. One
potential solution is to use a 3D Gaussian Splatting (3DGS) based model, such
as the Video Gaussian Representation (VGR), which is capable of encoding video
as a multitude of 3D Gaussians and is applicable for numerous video processing
operations, including editing. Nevertheless, in this case, the capacity for
modification is constrained to a limited set of basic transformations. To
address this issue, we introduce the Video Gaussian Splatting (VeGaS) model,
which enables realistic modifications of video data. To construct VeGaS, we
propose a novel family of Folded-Gaussian distributions designed to capture
nonlinear dynamics in a video stream and model consecutive frames by 2D
Gaussians obtained as respective conditional distributions. Our experiments
demonstrate that VeGaS outperforms state-of-the-art solutions in frame
reconstruction tasks and allows realistic modifications of video data. The code
is available at: https://github.com/gmum/VeGaS.
The Video Gaussian Splatting (VeGaS) model uses a novel Folded-Gaussian distribution to enable realistic modifications of video data and performs well in frame reconstruction tasks.
Implicit Neural Representations INRs 3D Gaussian Splatting 3DGS Video Gaussian Representation VGR Folded-Gaussian distributions 2D Gaussians