Jointly Generating Multi-view Consistent PBR Textures using
  Collaborative Control
Multi-view consistency remains a challenge for image diffusion models. Even
within the Text-to-Texture problem, where perfect geometric correspondences are
known a priori, many methods fail to yield aligned predictions across views,
necessitating non-trivial fusion methods to incorporate the results onto the
original mesh. We explore this issue for a Collaborative Control workflow
specifically in PBR Text-to-Texture. Collaborative Control directly models PBR
image probability distributions, including normal bump maps; to our knowledge,
the only diffusion model to directly output full PBR stacks. We discuss the
design decisions involved in making this model multi-view consistent, and
demonstrate the effectiveness of our approach in ablation studies, as well as
practical applications.
67079b9754069379bd86c10c
Shimon Vainer
admin_assigned
2024-10-10T09:46:53.809Z
63f9e95db31627d24009de1a
/avatars/53e6782c38bd1babca03389ae15a7d2e.svg
Shimon Vainer
user
esx2ve
67079b9754069379bd86c10d
Konstantin Kutsy
admin_assigned
2024-10-10T09:46:59.437Z
64636800d4d34b01f45cc40d
/avatars/f423e37b8461f3bda11021b3abf80ee1.svg
Konstantin Kutsy
user
bostadynamics
67079b9754069379bd86c10e
Dante De Nigris
67079b9754069379bd86c10f
Ciara Rowles
claimed_verified
2024-10-10T09:29:55.515Z
63357214eb6132ca653020e7
https://cdn-avatars.huggingface.co/v1/production/uploads/63357214eb6132ca653020e7/A_imoyFDx30wgi0guG_s8.png
Ciara
user
CiaraRowles
67079b9754069379bd86c110
Slava Elizarov
admin_assigned
2024-10-10T09:47:07.631Z
639b56244bbcd176745a6e6d
https://cdn-avatars.huggingface.co/v1/production/uploads/639b56244bbcd176745a6e6d/a1DRanMtKUNJNcrt3ZmOg.jpeg
Slava Elizarov
user
SlavaElizarov
67079b9754069379bd86c111
Simon Donné
Jointly Generating Multi-view Consistent PBR Textures using
  Collaborative Control
Multi-view consistency remains a challenge for image diffusion models. Even
within the Text-to-Texture problem, where perfect geometric correspondences are
known a priori, many methods fail to yield aligned predictions across views,
necessitating non-trivial fusion methods to incorporate the results onto the
original mesh. We explore this issue for a Collaborative Control workflow
specifically in PBR Text-to-Texture. Collaborative Control directly models PBR
image probability distributions, including normal bump maps; to our knowledge,
the only diffusion model to directly output full PBR stacks. We discuss the
design decisions involved in making this model multi-view consistent, and
demonstrate the effectiveness of our approach in ablation studies, as well as
practical applications.
A collaborative control diffusion model for PBR text-to-texture addresses multi-view inconsistency by directly modeling full PBR stacks, enabling better alignment across views.
multi-view consistency image diffusion models Text-to-Texture PBR image probability distributions normal bump maps collaborative control PBR stacks ablation studies