LDM3D-VR: Latent Diffusion Model for 3D VR
Latent diffusion models have proven to be state-of-the-art in the creation
and manipulation of visual outputs. However, as far as we know, the generation
of depth maps jointly with RGB is still limited. We introduce LDM3D-VR, a suite
of diffusion models targeting virtual reality development that includes
LDM3D-pano and LDM3D-SR. These models enable the generation of panoramic RGBD
based on textual prompts and the upscaling of low-resolution inputs to
high-resolution RGBD, respectively. Our models are fine-tuned from existing
pretrained models on datasets containing panoramic/high-resolution RGB images,
depth maps and captions. Both models are evaluated in comparison to existing
related methods.
6549e852fc80ab2d8ca40a20
Gabriela Ben Melech Stan
claimed_verified
2023-11-09T09:20:20.208Z
63174cfddc97a974718f373a
/avatars/f5a1d5d90abf8aafe8864bc4f1b076cc.svg
gabriela ben melech
user
gabsm
6549e852fc80ab2d8ca40a21
Diana Wofk
claimed_verified
2024-09-04T07:26:28.561Z
66d740a519214d743cc30335
/avatars/bc0839af30a183d5fae0a353d02733e2.svg
Diana Wofk
user
dwofk
6549e852fc80ab2d8ca40a22
Estelle Aflalo
admin_assigned
2023-11-07T16:13:42.992Z
64575266900b28f7841090e5
/avatars/13dc0a370bb24271653c352897a2673e.svg
Estelle Aflalo
user
estellea
6549e852fc80ab2d8ca40a23
Shao-Yen Tseng
claimed_verified
2023-11-07T17:12:46.489Z
6316524d76c93113fb1572ac
/avatars/39824232871f0a4b402d4e89435da58f.svg
Shaoyen Tseng
user
shaoyent
6549e852fc80ab2d8ca40a24
Zhipeng Cai
claimed_verified
2023-11-07T20:23:35.699Z
6419fe794577db917f6d17fc
/avatars/ed10371a0082c5e02d32b76217200207.svg
zhipeng cai
user
zcai
6549e852fc80ab2d8ca40a25
Michael Paulitsch
admin_assigned
2023-11-07T16:14:06.607Z
645e367e1db05f131837ded6
/avatars/c8882aa99f14fe872dabd7f2d25b4d38.svg
Michael Paulitsch
user
Paulitsch
6549e852fc80ab2d8ca40a26
Vasudev Lal
admin_assigned
2023-11-07T16:14:13.593Z
6454021413dba495b2dfebd1
/avatars/16f019f7c321aa24000364a50a1975fe.svg
Vasudev Lal
user
vasudevlal
LDM3D-VR: Latent Diffusion Model for 3D VR
Latent diffusion models have proven to be state-of-the-art in the creation
and manipulation of visual outputs. However, as far as we know, the generation
of depth maps jointly with RGB is still limited. We introduce LDM3D-VR, a suite
of diffusion models targeting virtual reality development that includes
LDM3D-pano and LDM3D-SR. These models enable the generation of panoramic RGBD
based on textual prompts and the upscaling of low-resolution inputs to
high-resolution RGBD, respectively. Our models are fine-tuned from existing
pretrained models on datasets containing panoramic/high-resolution RGB images,
depth maps and captions. Both models are evaluated in comparison to existing
related methods.
LDM3D-VR, comprising LDM3D-pano and LDM3D-SR, generates high-resolution RGBD from textual prompts and low-resolution inputs, demonstrating state-of-the-art performance in virtual reality applications.
latent diffusion models RGBD virtual reality LDM3D-VR LDM3D-pano LDM3D-SR pretrained models datasets depth maps captions