JanusFlow: Harmonizing Autoregression and Rectified Flow for Unified
  Multimodal Understanding and Generation
We present JanusFlow, a powerful framework that unifies image understanding
and generation in a single model. JanusFlow introduces a minimalist
architecture that integrates autoregressive language models with rectified
flow, a state-of-the-art method in generative modeling. Our key finding
demonstrates that rectified flow can be straightforwardly trained within the
large language model framework, eliminating the need for complex architectural
modifications. To further improve the performance of our unified model, we
adopt two key strategies: (i) decoupling the understanding and generation
encoders, and (ii) aligning their representations during unified training.
Extensive experiments show that JanusFlow achieves comparable or superior
performance to specialized models in their respective domains, while
significantly outperforming existing unified approaches across standard
benchmarks. This work represents a step toward more efficient and versatile
vision-language models.
673423249a3e8b8171a34bbe
Yiyang Ma
admin_assigned
2024-11-15T14:33:33.530Z
667282bf992bdf3ecb3bdc86
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/aXQRm8RUJXNJYZaBQzlDu.jpeg
Ma_Yiyang
user
CNMaxwell
673423249a3e8b8171a34bbf
Xingchao Liu
admin_assigned
2024-11-15T14:33:41.130Z
646b0bbdec9a61e871799339
https://cdn-avatars.huggingface.co/v1/production/uploads/646b0bbdec9a61e871799339/Xippv-ajkHkrGGAA7caLn.jpeg
Xingchao Liu
user
XCLiu
673423249a3e8b8171a34bc0
Xiaokang Chen
admin_assigned
2024-11-15T14:33:58.750Z
6635e701420baf7bc3f93561
/avatars/cdbb3085fee73ac520888977e2c575ea.svg
Xiaokang Chen
user
CharlesCXK
673423249a3e8b8171a34bc1
Wen Liu
claimed_verified
2024-12-30T19:37:41.758Z
63198e8802fb322037332f2d
/avatars/d3f9c206e387df35beb0ed0ef1cdf865.svg
Wen Liu
user
doubility123
673423249a3e8b8171a34bc2
Chengyue Wu
admin_assigned
2024-11-15T14:34:16.414Z
617526c9de8feb54b0ce45ad
/avatars/7faf8c6f71fc318a0113d780d376c381.svg
Wu Chengyue
user
WuChengyue
673423249a3e8b8171a34bc3
Zhiyu Wu
admin_assigned
2024-11-15T14:34:40.906Z
647a3a7de321bdf96ec45c28
/avatars/7c30461f15eff3a49e6395d5c49f53ff.svg
Zhiyu Wu
user
juliuswu
673423249a3e8b8171a34bc4
Zizheng Pan
admin_assigned
2024-11-15T14:34:46.332Z
629d55b53a3221bb2117422c
/avatars/e39f25cd7190d282478cdac20a7700ee.svg
Zizheng Pan
user
zizhpan
673423249a3e8b8171a34bc5
Zhenda Xie
admin_assigned
2024-11-15T14:34:53.870Z
6539f6ea26df26ecd1393c37
https://cdn-avatars.huggingface.co/v1/production/uploads/6539f6ea26df26ecd1393c37/9VJusLLAiLhUxfBAFggpF.jpeg
Zhenda Xie
user
zdaxie
673423249a3e8b8171a34bc6
Haowei Zhang
673423249a3e8b8171a34bc7
Xingkai yu
673423249a3e8b8171a34bc8
Liang Zhao
673423249a3e8b8171a34bc9
Yisong Wang
673423249a3e8b8171a34bca
Jiaying Liu
admin_assigned
2024-11-15T14:35:28.944Z
647ff859095af0bf116e6039
/avatars/63e201d20e792383483787cff0e38edb.svg
jiaying
user
jiayingliu
673423249a3e8b8171a34bcb
Chong Ruan
admin_assigned
2024-11-15T14:35:34.180Z
6398203609f12714ed1935c2
https://cdn-avatars.huggingface.co/v1/production/uploads/6398203609f12714ed1935c2/uXgl0LgKnFYjq1Wz39-a6.jpeg
Chong Ruan
user
Chester111
JanusFlow: Harmonizing Autoregression and Rectified Flow for Unified
  Multimodal Understanding and Generation
We present JanusFlow, a powerful framework that unifies image understanding
and generation in a single model. JanusFlow introduces a minimalist
architecture that integrates autoregressive language models with rectified
flow, a state-of-the-art method in generative modeling. Our key finding
demonstrates that rectified flow can be straightforwardly trained within the
large language model framework, eliminating the need for complex architectural
modifications. To further improve the performance of our unified model, we
adopt two key strategies: (i) decoupling the understanding and generation
encoders, and (ii) aligning their representations during unified training.
Extensive experiments show that JanusFlow achieves comparable or superior
performance to specialized models in their respective domains, while
significantly outperforming existing unified approaches across standard
benchmarks. This work represents a step toward more efficient and versatile
vision-language models.
JanusFlow unifies image understanding and generation using rectified flow within a language model framework, demonstrating superior performance across benchmarks.
JanusFlow autoregressive language models rectified flow generative modeling understanding encoders generation encoders unified training vision-language models