ORIGEN: Zero-Shot 3D Orientation Grounding in Text-to-Image Generation
We introduce ORIGEN, the first zero-shot method for 3D orientation grounding
in text-to-image generation across multiple objects and diverse categories.
While previous work on spatial grounding in image generation has mainly focused
on 2D positioning, it lacks control over 3D orientation. To address this, we
propose a reward-guided sampling approach using a pretrained discriminative
model for 3D orientation estimation and a one-step text-to-image generative
flow model. While gradient-ascent-based optimization is a natural choice for
reward-based guidance, it struggles to maintain image realism. Instead, we
adopt a sampling-based approach using Langevin dynamics, which extends gradient
ascent by simply injecting random noise--requiring just a single additional
line of code. Additionally, we introduce adaptive time rescaling based on the
reward function to accelerate convergence. Our experiments show that ORIGEN
outperforms both training-based and test-time guidance methods across
quantitative metrics and user studies.
67e9eebe1f495035ca228ded
Yunhong Min
claimed_verified
2025-03-31T08:11:49.474Z
66ee81b676a8038cb42c8caa
/avatars/9b4c5ded9c94788c35ce7ffbc2f8d24b.svg
Yunhong Min
user
myhong
67e9eebe1f495035ca228dee
Daehyeon Choi
claimed_verified
2025-03-31T08:11:51.359Z
6616702547ea6347974667e5
/avatars/6bf95e50ba19df163ef89867ed63fecc.svg
Daehyeon Choi
user
daehyeonchoi
67e9eebe1f495035ca228def
Kyeongmin Yeo
admin_assigned
2025-03-31T08:26:31.020Z
659e42cfb65ee9ee1fd11e61
/avatars/a9220d099f32800fc43ae79bb519c1e9.svg
Kyeongmin Yeo
user
32V
67e9eebe1f495035ca228df0
Jihyun Lee
67e9eebe1f495035ca228df1
Minhyuk Sung
admin_assigned
2025-03-31T08:26:40.432Z
631f432b5ba8c026340a7890
https://cdn-avatars.huggingface.co/v1/production/uploads/631f432b5ba8c026340a7890/9PK7A_TRMpugwYjCsNBf1.jpeg
Minhyuk Sung
user
Minhyuk
ORIGEN: Zero-Shot 3D Orientation Grounding in Text-to-Image Generation
We introduce ORIGEN, the first zero-shot method for 3D orientation grounding
in text-to-image generation across multiple objects and diverse categories.
While previous work on spatial grounding in image generation has mainly focused
on 2D positioning, it lacks control over 3D orientation. To address this, we
propose a reward-guided sampling approach using a pretrained discriminative
model for 3D orientation estimation and a one-step text-to-image generative
flow model. While gradient-ascent-based optimization is a natural choice for
reward-based guidance, it struggles to maintain image realism. Instead, we
adopt a sampling-based approach using Langevin dynamics, which extends gradient
ascent by simply injecting random noise--requiring just a single additional
line of code. Additionally, we introduce adaptive time rescaling based on the
reward function to accelerate convergence. Our experiments show that ORIGEN
outperforms both training-based and test-time guidance methods across
quantitative metrics and user studies.
A zero-shot method for 3D orientation grounding in text-to-image generation uses a reward-guided sampling approach with Langevin dynamics and adaptive time rescaling.
zero-shot method 3D orientation grounding text-to-image generation reward-guided sampling pretrained discriminative model one-step generative flow model Langevin dynamics adaptive time rescaling