MVPaint: Synchronized Multi-View Diffusion for Painting Anything 3D
Texturing is a crucial step in the 3D asset production workflow, which
enhances the visual appeal and diversity of 3D assets. Despite recent
advancements in Text-to-Texture (T2T) generation, existing methods often yield
subpar results, primarily due to local discontinuities, inconsistencies across
multiple views, and their heavy dependence on UV unwrapping outcomes. To tackle
these challenges, we propose a novel generation-refinement 3D texturing
framework called MVPaint, which can generate high-resolution, seamless textures
while emphasizing multi-view consistency. MVPaint mainly consists of three key
modules. 1) Synchronized Multi-view Generation (SMG). Given a 3D mesh model,
MVPaint first simultaneously generates multi-view images by employing an SMG
model, which leads to coarse texturing results with unpainted parts due to
missing observations. 2) Spatial-aware 3D Inpainting (S3I). To ensure complete
3D texturing, we introduce the S3I method, specifically designed to effectively
texture previously unobserved areas. 3) UV Refinement (UVR). Furthermore,
MVPaint employs a UVR module to improve the texture quality in the UV space,
which first performs a UV-space Super-Resolution, followed by a Spatial-aware
Seam-Smoothing algorithm for revising spatial texturing discontinuities caused
by UV unwrapping. Moreover, we establish two T2T evaluation benchmarks: the
Objaverse T2T benchmark and the GSO T2T benchmark, based on selected
high-quality 3D meshes from the Objaverse dataset and the entire GSO dataset,
respectively. Extensive experimental results demonstrate that MVPaint surpasses
existing state-of-the-art methods. Notably, MVPaint could generate
high-fidelity textures with minimal Janus issues and highly enhanced cross-view
consistency.
672a3b03d0448c8bae6dc97b
Wei Cheng
claimed_verified
2024-11-05T16:09:12.427Z
64b914c8ace99c0723ad83a9
https://cdn-avatars.huggingface.co/v1/production/uploads/64b914c8ace99c0723ad83a9/B4gxNByeVY_xaOcjwiN1j.jpeg
Wei Cheng
user
wchengad
672a3b03d0448c8bae6dc97c
Juncheng Mu
672a3b03d0448c8bae6dc97d
Xianfang Zeng
672a3b03d0448c8bae6dc97e
Xin Chen
672a3b03d0448c8bae6dc97f
Anqi Pang
672a3b03d0448c8bae6dc980
Chi Zhang
672a3b03d0448c8bae6dc981
Zhibin Wang
672a3b03d0448c8bae6dc982
Bin Fu
672a3b03d0448c8bae6dc983
Gang Yu
claimed_verified
2024-11-15T09:37:39.626Z
63417332c5565a4b8d43a0d8
https://cdn-avatars.huggingface.co/v1/production/uploads/63417332c5565a4b8d43a0d8/MmnYG7Wu2Z_lqCBvTfJmy.png
Gang Yu
user
skicy
672a3b03d0448c8bae6dc984
Ziwei Liu
672a3b03d0448c8bae6dc985
Liang Pan
claimed_verified
2024-11-05T16:09:10.438Z
64ef30fb0de94d31b7921175
/avatars/d4f4c7be460befc1cedec13bbf9db972.svg
pan
user
pldeqiushui
MVPaint: Synchronized Multi-View Diffusion for Painting Anything 3D
Texturing is a crucial step in the 3D asset production workflow, which
enhances the visual appeal and diversity of 3D assets. Despite recent
advancements in Text-to-Texture (T2T) generation, existing methods often yield
subpar results, primarily due to local discontinuities, inconsistencies across
multiple views, and their heavy dependence on UV unwrapping outcomes. To tackle
these challenges, we propose a novel generation-refinement 3D texturing
framework called MVPaint, which can generate high-resolution, seamless textures
while emphasizing multi-view consistency. MVPaint mainly consists of three key
modules. 1) Synchronized Multi-view Generation (SMG). Given a 3D mesh model,
MVPaint first simultaneously generates multi-view images by employing an SMG
model, which leads to coarse texturing results with unpainted parts due to
missing observations. 2) Spatial-aware 3D Inpainting (S3I). To ensure complete
3D texturing, we introduce the S3I method, specifically designed to effectively
texture previously unobserved areas. 3) UV Refinement (UVR). Furthermore,
MVPaint employs a UVR module to improve the texture quality in the UV space,
which first performs a UV-space Super-Resolution, followed by a Spatial-aware
Seam-Smoothing algorithm for revising spatial texturing discontinuities caused
by UV unwrapping. Moreover, we establish two T2T evaluation benchmarks: the
Objaverse T2T benchmark and the GSO T2T benchmark, based on selected
high-quality 3D meshes from the Objaverse dataset and the entire GSO dataset,
respectively. Extensive experimental results demonstrate that MVPaint surpasses
existing state-of-the-art methods. Notably, MVPaint could generate
high-fidelity textures with minimal Janus issues and highly enhanced cross-view
consistency.
MVPaint, a 3D texturing framework, addresses local discontinuities and multi-view inconsistencies in Text-to-Texture generation by employing synchronized multi-view generation, spatial-aware 3D inpainting, and UV refinement techniques, achieving high-fidelity textures.
synchronized multi-view generation spatial-aware 3D inpainting UV refinement UV-space super-resolution spatial-aware seam-smoothing Objaverse T2T benchmark GSO T2T benchmark Janus issues cross-view consistency