Atlas: Multi-Scale Attention Improves Long Context Image Modeling
Efficiently modeling massive images is a long-standing challenge in machine
learning. To this end, we introduce Multi-Scale Attention (MSA). MSA relies on
two key ideas, (i) multi-scale representations (ii) bi-directional cross-scale
communication. MSA creates O(log N) scales to represent the image across
progressively coarser features and leverages cross-attention to propagate
information across scales. We then introduce Atlas, a novel neural network
architecture based on MSA. We demonstrate that Atlas significantly improves the
compute-performance tradeoff of long-context image modeling in a
high-resolution variant of ImageNet 100. At 1024px resolution, Atlas-B achieves
91.04% accuracy, comparable to ConvNext-B (91.92%) while being 4.3x faster.
Atlas is 2.95x faster and 7.38% better than FasterViT, 2.25x faster and 4.96%
better than LongViT. In comparisons against MambaVision-S, we find Atlas-S
achieves 5%, 16% and 32% higher accuracy at 1024px, 2048px and 4096px
respectively, while obtaining similar runtimes. Code for reproducing our
experiments and pretrained models is available at
https://github.com/yalalab/atlas.
67daf5dff34884a28b17bb53
Kumar Krishna Agrawal
extracted_pending
2025-03-19T16:50:44.848Z
6254623af74165c7d2df0c18
/avatars/4ddd1aa858723dd9348fc51966f700c3.svg
Kumar Krishna Agrawal
user
kumarkrishna
67daf5dff34884a28b17bb54
Long Lian
claimed_verified
2025-03-20T10:45:56.150Z
63797c273f575acc2f6893c0
/avatars/32d7a6a8881c8c4d80a097b732ed24b6.svg
Long(Tony) Lian
user
longlian
67daf5dff34884a28b17bb55
Longchao Liu
67daf5dff34884a28b17bb56
Natalia Harguindeguy
extracted_confirmed
2025-03-20T04:19:22.595Z
6333a9195a032dcd095dda13
https://cdn-avatars.huggingface.co/v1/production/uploads/1664329996201-noauth.jpeg
Adam Yala
user
yala
67daf5dff34884a28b17bb57
Boyi Li
67daf5dff34884a28b17bb58
Alexander Bick
67daf5dff34884a28b17bb59
Maggie Chung
67daf5dff34884a28b17bb5a
Trevor Darrell
67daf5dff34884a28b17bb5b
Adam Yala
Atlas: Multi-Scale Attention Improves Long Context Image Modeling
Efficiently modeling massive images is a long-standing challenge in machine
learning. To this end, we introduce Multi-Scale Attention (MSA). MSA relies on
two key ideas, (i) multi-scale representations (ii) bi-directional cross-scale
communication. MSA creates O(log N) scales to represent the image across
progressively coarser features and leverages cross-attention to propagate
information across scales. We then introduce Atlas, a novel neural network
architecture based on MSA. We demonstrate that Atlas significantly improves the
compute-performance tradeoff of long-context image modeling in a
high-resolution variant of ImageNet 100. At 1024px resolution, Atlas-B achieves
91.04% accuracy, comparable to ConvNext-B (91.92%) while being 4.3x faster.
Atlas is 2.95x faster and 7.38% better than FasterViT, 2.25x faster and 4.96%
better than LongViT. In comparisons against MambaVision-S, we find Atlas-S
achieves 5%, 16% and 32% higher accuracy at 1024px, 2048px and 4096px
respectively, while obtaining similar runtimes. Code for reproducing our
experiments and pretrained models is available at
https://github.com/yalalab/atlas.
Atlas, a multi-scale attention-based neural network, improves computational efficiency without sacrificing accuracy in high-resolution image modeling.
Multi-Scale Attention MSA multi-scale representations bi-directional cross-scale communication cross-attention Atlas neural network ImageNet 100 ConvNext-B FasterViT LongViT MambaVision-S