EMOv2: Pushing 5M Vision Model Frontier
This work focuses on developing parameter-efficient and lightweight models
for dense predictions while trading off parameters, FLOPs, and performance. Our
goal is to set up the new frontier of the 5M magnitude lightweight model on
various downstream tasks. Inverted Residual Block (IRB) serves as the
infrastructure for lightweight CNNs, but no counterparts have been recognized
by attention-based design. Our work rethinks the lightweight infrastructure of
efficient IRB and practical components in Transformer from a unified
perspective, extending CNN-based IRB to attention-based models and abstracting
a one-residual Meta Mobile Block (MMBlock) for lightweight model design.
Following neat but effective design criterion, we deduce a modern Improved
Inverted Residual Mobile Block (i2RMB) and improve a hierarchical Efficient
MOdel (EMOv2) with no elaborate complex structures. Considering the
imperceptible latency for mobile users when downloading models under 4G/5G
bandwidth and ensuring model performance, we investigate the performance upper
limit of lightweight models with a magnitude of 5M. Extensive experiments on
various vision recognition, dense prediction, and image generation tasks
demonstrate the superiority of our EMOv2 over state-of-the-art methods, e.g.,
EMOv2-1M/2M/5M achieve 72.3, 75.8, and 79.4 Top-1 that surpass equal-order
CNN-/Attention-based models significantly. At the same time, EMOv2-5M equipped
RetinaNet achieves 41.5 mAP for object detection tasks that surpasses the
previous EMO-5M by +2.6. When employing the more robust training recipe, our
EMOv2-5M eventually achieves 82.9 Top-1 accuracy, which elevates the
performance of 5M magnitude models to a new level. Code is available at
https://github.com/zhangzjn/EMOv2.
67592785adabd5ddff254756
Jiangning Zhang
67592785adabd5ddff254757
Teng Hu
67592785adabd5ddff254758
Haoyang He
admin_assigned
2024-12-11T10:38:08.473Z
65b9f891efa633aa9a9a8315
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/onhK-rjtHmaqSmHpSS11S.jpeg
haoyang He
user
hhy724
67592785adabd5ddff254759
Zhucun Xue
67592785adabd5ddff25475a
Yabiao Wang
67592785adabd5ddff25475b
Chengjie Wang
admin_assigned
2024-12-18T13:47:37.742Z
65729087379284b904a6d81d
/avatars/0e7968b1d9221d523a54e3ac787e449d.svg
Chengjie Wang
user
chengjie-wang
67592785adabd5ddff25475c
Yong Liu
67592785adabd5ddff25475d
Xiangtai Li
claimed_verified
2024-12-11T09:19:46.630Z
63958b4414513eaf9029ebf1
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/U1g5H071pWRswGAG9UTpo.png
Xiangtai Li
user
LXT
67592785adabd5ddff25475e
Dacheng Tao
EMOv2: Pushing 5M Vision Model Frontier
This work focuses on developing parameter-efficient and lightweight models
for dense predictions while trading off parameters, FLOPs, and performance. Our
goal is to set up the new frontier of the 5M magnitude lightweight model on
various downstream tasks. Inverted Residual Block (IRB) serves as the
infrastructure for lightweight CNNs, but no counterparts have been recognized
by attention-based design. Our work rethinks the lightweight infrastructure of
efficient IRB and practical components in Transformer from a unified
perspective, extending CNN-based IRB to attention-based models and abstracting
a one-residual Meta Mobile Block (MMBlock) for lightweight model design.
Following neat but effective design criterion, we deduce a modern Improved
Inverted Residual Mobile Block (i2RMB) and improve a hierarchical Efficient
MOdel (EMOv2) with no elaborate complex structures. Considering the
imperceptible latency for mobile users when downloading models under 4G/5G
bandwidth and ensuring model performance, we investigate the performance upper
limit of lightweight models with a magnitude of 5M. Extensive experiments on
various vision recognition, dense prediction, and image generation tasks
demonstrate the superiority of our EMOv2 over state-of-the-art methods, e.g.,
EMOv2-1M/2M/5M achieve 72.3, 75.8, and 79.4 Top-1 that surpass equal-order
CNN-/Attention-based models significantly. At the same time, EMOv2-5M equipped
RetinaNet achieves 41.5 mAP for object detection tasks that surpasses the
previous EMO-5M by +2.6. When employing the more robust training recipe, our
EMOv2-5M eventually achieves 82.9 Top-1 accuracy, which elevates the
performance of 5M magnitude models to a new level. Code is available at
https://github.com/zhangzjn/EMOv2.
A new lightweight model infrastructure, EMOv2, achieves high performance in dense predictions and vision recognition tasks using a parameter-efficient design that combines inverted residual blocks with attention mechanisms.
Inverted Residual Block Meta Mobile Block Improved Inverted Residual Mobile Block Efficient Model RetinaNet object detection Top-1 accuracy mAP