From Editor to Dense Geometry Estimator
Leveraging visual priors from pre-trained text-to-image (T2I) generative
models has shown success in dense prediction. However, dense prediction is
inherently an image-to-image task, suggesting that image editing models, rather
than T2I generative models, may be a more suitable foundation for fine-tuning.
  Motivated by this, we conduct a systematic analysis of the fine-tuning
behaviors of both editors and generators for dense geometry estimation. Our
findings show that editing models possess inherent structural priors, which
enable them to converge more stably by ``refining" their innate features, and
ultimately achieve higher performance than their generative counterparts.
  Based on these findings, we introduce FE2E, a framework that
pioneeringly adapts an advanced editing model based on Diffusion Transformer
(DiT) architecture for dense geometry prediction. Specifically, to tailor the
editor for this deterministic task, we reformulate the editor's original flow
matching loss into the ``consistent velocity" training objective. And we use
logarithmic quantization to resolve the precision conflict between the editor's
native BFloat16 format and the high precision demand of our tasks.
Additionally, we leverage the DiT's global attention for a cost-free joint
estimation of depth and normals in a single forward pass, enabling their
supervisory signals to mutually enhance each other.
  Without scaling up the training data, FE2E achieves impressive performance
improvements in zero-shot monocular depth and normal estimation across multiple
datasets. Notably, it achieves over 35\% performance gains on the ETH3D dataset
and outperforms the DepthAnything series, which is trained on 100times data.
The project page can be accessed https://amap-ml.github.io/FE2E/{here}.
68ba64a6736018af705e8ea9
JiYuan Wang
claimed_verified
2025-09-05T08:01:34.119Z
63cf4ceb7332dafae2b9884c
/avatars/652255476e1898c175c28f7a1d67cc06.svg
JiYuan Wang
user
exander
68ba64a6736018af705e8eaa
Chunyu Lin
admin_assigned
2025-09-05T14:30:01.810Z
67aea8aabdb9354439534e49
/avatars/79d245ab62aae59279ccd366d7f1de5e.svg
Chris Lin
user
chunyulin
68ba64a6736018af705e8eab
Lei Sun
68ba64a6736018af705e8eac
Rongying Liu
admin_assigned
2025-09-05T14:30:07.625Z
68288895682a9738b93b1d2b
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/w_nnDk938berOQvlZsByM.jpeg
Rongying Liu
user
Roinnnn11
68ba64a6736018af705e8ead
Lang Nie
68ba64a6736018af705e8eae
Mingxing Li
admin_assigned
2025-09-05T14:30:32.021Z
6513a0f14f1682e4407758a9
/avatars/b2a6886114492944cfa235363817565f.svg
Mingxing Li
user
MingxingLi
68ba64a6736018af705e8eaf
Kang Liao
admin_assigned
2025-09-05T14:30:13.569Z
65bc98383b879593a5a2f5e5
https://cdn-avatars.huggingface.co/v1/production/uploads/65bc98383b879593a5a2f5e5/p2ZtoTFN6tW-QkcPJf7YT.jpeg
Kang Liao
user
KangLiao
68ba64a6736018af705e8eb0
Xiangxiang Chu
68ba64a6736018af705e8eb1
Yao Zhao
From Editor to Dense Geometry Estimator
Leveraging visual priors from pre-trained text-to-image (T2I) generative
models has shown success in dense prediction. However, dense prediction is
inherently an image-to-image task, suggesting that image editing models, rather
than T2I generative models, may be a more suitable foundation for fine-tuning.
  Motivated by this, we conduct a systematic analysis of the fine-tuning
behaviors of both editors and generators for dense geometry estimation. Our
findings show that editing models possess inherent structural priors, which
enable them to converge more stably by ``refining" their innate features, and
ultimately achieve higher performance than their generative counterparts.
  Based on these findings, we introduce FE2E, a framework that
pioneeringly adapts an advanced editing model based on Diffusion Transformer
(DiT) architecture for dense geometry prediction. Specifically, to tailor the
editor for this deterministic task, we reformulate the editor's original flow
matching loss into the ``consistent velocity" training objective. And we use
logarithmic quantization to resolve the precision conflict between the editor's
native BFloat16 format and the high precision demand of our tasks.
Additionally, we leverage the DiT's global attention for a cost-free joint
estimation of depth and normals in a single forward pass, enabling their
supervisory signals to mutually enhance each other.
  Without scaling up the training data, FE2E achieves impressive performance
improvements in zero-shot monocular depth and normal estimation across multiple
datasets. Notably, it achieves over 35\% performance gains on the ETH3D dataset
and outperforms the DepthAnything series, which is trained on 100times data.
The project page can be accessed https://amap-ml.github.io/FE2E/{here}.
FE2E, a framework using a Diffusion Transformer for dense geometry prediction, outperforms generative models in zero-shot monocular depth and normal estimation with improved performance and efficiency.
text-to-image dense prediction image editing models generative models fine-tuning dense geometry estimation structural priors flow matching loss consistent velocity logarithmic quantization BFloat16 global attention ETH3D DepthAnything