From Ideal to Real: Unified and Data-Efficient Dense Prediction for
  Real-World Scenarios
Dense prediction tasks hold significant importance of computer vision, aiming
to learn pixel-wise annotated label for an input image. Despite advances in
this field, existing methods primarily focus on idealized conditions, with
limited generalization to real-world scenarios and facing the challenging
scarcity of real-world data. To systematically study this problem, we first
introduce DenseWorld, a benchmark spanning a broad set of 25 dense prediction
tasks that correspond to urgent real-world applications, featuring unified
evaluation across tasks. Then, we propose DenseDiT, which maximally exploits
generative models' visual priors to perform diverse real-world dense prediction
tasks through a unified strategy. DenseDiT combines a parameter-reuse mechanism
and two lightweight branches that adaptively integrate multi-scale context,
working with less than 0.1% additional parameters. Evaluations on DenseWorld
reveal significant performance drops in existing general and specialized
baselines, highlighting their limited real-world generalization. In contrast,
DenseDiT achieves superior results using less than 0.01% training data of
baselines, underscoring its practical value for real-world deployment. Our
data, and checkpoints and codes are available at
https://xcltql666.github.io/DenseDiTProj
686218679e7509383d29abb3
Changliang Xia
686218679e7509383d29abb4
Chengyou Jia
claimed_verified
2025-07-01T10:14:28.128Z
6602548a68d519ed324b47c5
/avatars/5ab411f87440cc2a98c7a1c6a3ed5548.svg
ChengyouJia
user
ChengyouJia
686218679e7509383d29abb5
Zhuohang Dang
686218679e7509383d29abb6
Minnan Luo
From Ideal to Real: Unified and Data-Efficient Dense Prediction for
  Real-World Scenarios
Dense prediction tasks hold significant importance of computer vision, aiming
to learn pixel-wise annotated label for an input image. Despite advances in
this field, existing methods primarily focus on idealized conditions, with
limited generalization to real-world scenarios and facing the challenging
scarcity of real-world data. To systematically study this problem, we first
introduce DenseWorld, a benchmark spanning a broad set of 25 dense prediction
tasks that correspond to urgent real-world applications, featuring unified
evaluation across tasks. Then, we propose DenseDiT, which maximally exploits
generative models' visual priors to perform diverse real-world dense prediction
tasks through a unified strategy. DenseDiT combines a parameter-reuse mechanism
and two lightweight branches that adaptively integrate multi-scale context,
working with less than 0.1% additional parameters. Evaluations on DenseWorld
reveal significant performance drops in existing general and specialized
baselines, highlighting their limited real-world generalization. In contrast,
DenseDiT achieves superior results using less than 0.01% training data of
baselines, underscoring its practical value for real-world deployment. Our
data, and checkpoints and codes are available at
https://xcltql666.github.io/DenseDiTProj
DenseDiT, a generative model-based approach, achieves superior performance in real-world dense prediction tasks using minimal training data compared to existing methods.
dense prediction generative models visual priors parameter-reuse mechanism lightweight branches multi-scale context DenseWorld DenseDiT