UniWorld: High-Resolution Semantic Encoders for Unified Visual
  Understanding and Generation
Although existing unified models deliver strong performance on
vision-language understanding and text-to-image generation, their models are
limited in exploring image perception and manipulation tasks, which are
urgently desired by users for wide applications. Recently, OpenAI released
their powerful GPT-4o-Image model for comprehensive image perception and
manipulation, achieving expressive capability and attracting community
interests. By observing the performance of GPT-4o-Image in our carefully
constructed experiments, we infer that GPT-4o-Image leverages features
extracted by semantic encoders instead of VAE, while VAEs are considered
essential components in many image manipulation models. Motivated by such
inspiring observations, we present a unified generative framework named
UniWorld based on semantic features provided by powerful visual-language models
and contrastive semantic encoders. As a result, we build a strong unified model
using only 1% amount of BAGEL's data, which consistently outperforms BAGEL on
image editing benchmarks. UniWorld also maintains competitive image
understanding and generation capabilities, achieving strong performance across
multiple image perception tasks. We fully open-source our models, including
model weights, training and evaluation scripts, and datasets.
683fae55c6b71c5994ccd4fe
Bin Lin
claimed_verified
2025-06-04T08:56:49.923Z
6367a8175bb06007ea099b8f
https://cdn-avatars.huggingface.co/v1/production/uploads/6367a8175bb06007ea099b8f/IjG7HyWyWRlVt_XwRbxRW.jpeg
linbin
user
LanguageBind
683fae55c6b71c5994ccd4ff
Zongjian Li
claimed_verified
2025-06-04T15:03:37.427Z
646df3c04ad7f907279f14c3
https://cdn-avatars.huggingface.co/v1/production/uploads/646df3c04ad7f907279f14c3/WZjSDtAmezmjbczLtCP2B.jpeg
Zongjian Li
user
chestnutlzj
683fae55c6b71c5994ccd500
Xinhua Cheng
683fae55c6b71c5994ccd501
Yuwei Niu
683fae55c6b71c5994ccd502
Yang Ye
683fae55c6b71c5994ccd503
Xianyi He
683fae55c6b71c5994ccd504
Shenghai Yuan
claimed_verified
2025-06-04T08:56:52.748Z
63468720dd6d90d82ccf3450
https://cdn-avatars.huggingface.co/v1/production/uploads/63468720dd6d90d82ccf3450/tVBFlmZNz8FRMkOrDaDID.jpeg
YSH
user
BestWishYsh
683fae55c6b71c5994ccd505
Wangbo Yu
683fae55c6b71c5994ccd506
Shaodong Wang
683fae55c6b71c5994ccd507
Yunyang Ge
admin_assigned
2025-06-05T07:41:16.521Z
646de6402fd5a8eb8c518aa6
https://cdn-avatars.huggingface.co/v1/production/uploads/646de6402fd5a8eb8c518aa6/HYWb8-fT1kTm-ROBr1-0X.jpeg
yunyangge
user
yunyangge
683fae55c6b71c5994ccd508
Yatian Pang
683fae55c6b71c5994ccd509
Li Yuan
UniWorld: High-Resolution Semantic Encoders for Unified Visual
  Understanding and Generation
Although existing unified models deliver strong performance on
vision-language understanding and text-to-image generation, their models are
limited in exploring image perception and manipulation tasks, which are
urgently desired by users for wide applications. Recently, OpenAI released
their powerful GPT-4o-Image model for comprehensive image perception and
manipulation, achieving expressive capability and attracting community
interests. By observing the performance of GPT-4o-Image in our carefully
constructed experiments, we infer that GPT-4o-Image leverages features
extracted by semantic encoders instead of VAE, while VAEs are considered
essential components in many image manipulation models. Motivated by such
inspiring observations, we present a unified generative framework named
UniWorld based on semantic features provided by powerful visual-language models
and contrastive semantic encoders. As a result, we build a strong unified model
using only 1% amount of BAGEL's data, which consistently outperforms BAGEL on
image editing benchmarks. UniWorld also maintains competitive image
understanding and generation capabilities, achieving strong performance across
multiple image perception tasks. We fully open-source our models, including
model weights, training and evaluation scripts, and datasets.
A unified generative framework called UniWorld uses semantic features from visual-language models for image perception and manipulation, outperforming BAGEL with reduced data.
GPT-4o-Image semantic encoders VAE UniWorld visual-language models contrastive semantic encoders image editing benchmarks image perception tasks