Step-3 is Large yet Affordable: Model-system Co-design for
  Cost-effective Decoding
Large language models (LLMs) face low hardware efficiency during decoding,
especially for long-context reasoning tasks. This paper introduces Step-3, a
321B-parameter VLM with hardware-aware model-system co-design optimized for
minimizing decoding costs. Step-3 innovates in two key dimensions: (1) A novel
Multi-Matrix Factorization Attention (MFA) mechanism that significantly reduces
both KV cache size and computation while maintaining high attention
expressiveness, and (2) Attention-FFN Disaggregation (AFD), a distributed
inference system that decouples attention and Feed-Forward Network (FFN) layers
into specialized subsystems. This co-design achieves unprecedented cost
efficiency: Step-3 significantly reduces theoretical decoding costs compared
with models like DeepSeek-V3 and Qwen3 MoE 235B, with the gains widening at
longer context. Step-3 achieves low cost while activating 38B parameters per
token (more than DeepSeek-V3 and Qwen3 MoE 235B), demonstrating that
hardware-aligned attention arithmetic intensity, MoE sparsity, and AFD are
critical to cost-effectiveness. We perform a head-to-head comparison with
DeepSeek-V3 in its favorable scenarios. Our implementation on Hopper GPUs
achieves a decoding throughput of up to 4,039 tokens per second per GPU under
50ms TPOT SLA (4K context, FP8, no MTP). It is higher than DeepSeek-V3's 2,324
in the same setup and sets a new Pareto frontier for LLM decoding.
688b59b8292830d55fa6894a
StepFun
688b59b8292830d55fa6894c
Bin Wang
688b59b8292830d55fa6894d
Bojun Wang
688b59b8292830d55fa6894e
Changyi Wan
688b59b8292830d55fa6894f
Guanzhe Huang
688b59b8292830d55fa68950
Hanpeng Hu
688b59b8292830d55fa68951
Haonan Jia
688b59b8292830d55fa68952
Hao Nie
688b59b8292830d55fa68953
Mingliang Li
688b59b8292830d55fa68954
Nuo Chen
688b59b8292830d55fa68955
Siyu Chen
688b59b8292830d55fa68956
Song Yuan
688b59b8292830d55fa68957
Wuxun Xie
688b59b8292830d55fa68958
Xiaoniu Song
688b59b8292830d55fa68959
Xing Chen
688b59b8292830d55fa6895a
Xingping Yang
688b59b8292830d55fa6895b
Xuelin Zhang
688b59b8292830d55fa6895c
Yanbo Yu
688b59b8292830d55fa6895d
Yaoyu Wang
688b59b8292830d55fa6895e
Yibo Zhu
688b59b8292830d55fa6895f
Yimin Jiang
688b59b8292830d55fa68960
Yu Zhou
688b59b8292830d55fa68961
Yuanwei Lu
688b59b8292830d55fa68962
Houyi Li
688b59b8292830d55fa68963
Jingcheng Hu
688b59b8292830d55fa68964
Ka Man Lo
688b59b8292830d55fa68965
Ailin Huang
688b59b8292830d55fa68966
Binxing Jiao
688b59b8292830d55fa68967
Bo Li
688b59b8292830d55fa68968
Boyu Chen
688b59b8292830d55fa68969
Changxin Miao
688b59b8292830d55fa6896a
Chang Lou
688b59b8292830d55fa6896b
Chen Hu
claimed_verified
2025-09-01T07:55:05.573Z
648183bfea00b120773198ba
/avatars/b69d04bba11e393d1a59e98653d5cfa6.svg
huchen
user
ccchen1006
688b59b8292830d55fa6896c
Chen Xu
688b59b8292830d55fa6896d
Chenfeng Yu
688b59b8292830d55fa6896e
Chengyuan Yao
688b59b8292830d55fa6896f
Daokuan Lv
688b59b8292830d55fa68970
Dapeng Shi
688b59b8292830d55fa68971
Deshan Sun
688b59b8292830d55fa68972
Ding Huang
688b59b8292830d55fa68973
Dingyuan Hu
688b59b8292830d55fa68974
Dongqing Pang
688b59b8292830d55fa68975
Enle Liu
688b59b8292830d55fa68976
Fajie Zhang
688b59b8292830d55fa68977
Fanqi Wan
688b59b8292830d55fa68978
Gulin Yan
688b59b8292830d55fa68979
Han Zhang
688b59b8292830d55fa6897a
Han Zhou
688b59b8292830d55fa6897b
Hanghao Wu
688b59b8292830d55fa6897c
Hangyu Guo
688b59b8292830d55fa6897d
Hanqi Chen
688b59b8292830d55fa6897e
Hanshan Zhang
claimed_verified
2025-08-06T19:32:02.861Z
64b7874b9f5987572ca28461
/avatars/d24ee0a6329ff93936aa7829481e2046.svg
hanshanzhang
user
brain-zhang
688b59b8292830d55fa6897f
Hao Wu
688b59b8292830d55fa68980
Haocheng Zhang
688b59b8292830d55fa68981
Haolong Yan
688b59b8292830d55fa68982
Haoran Lv
688b59b8292830d55fa68983
Haoran Wei
688b59b8292830d55fa68984
Hebin Zhou
688b59b8292830d55fa68985
Heng Wang
688b59b8292830d55fa68986
Heng Wang
688b59b8292830d55fa68987
Hongxin Li
688b59b8292830d55fa68988
Hongyu Zhou
688b59b8292830d55fa68989
Hongyuan Wang
688b59b8292830d55fa6898a
Huiyong Guo
688b59b8292830d55fa6898b
Jia Wang
688b59b8292830d55fa6898c
Jiahao Gong
688b59b8292830d55fa6898d
Jialing Xie
688b59b8292830d55fa6898e
Jian Zhou
688b59b8292830d55fa6898f
Jianjian Sun
688b59b8292830d55fa68990
Jiaoren Wu
688b59b8292830d55fa68991
Jiaran Zhang
688b59b8292830d55fa68992
Jiayu Liu
688b59b8292830d55fa68993
Jie Cheng
688b59b8292830d55fa68994
Jie Luo
688b59b8292830d55fa68995
Jie Yan
688b59b8292830d55fa68996
Jie Yang
688b59b8292830d55fa68997
Jieyi Hou
688b59b8292830d55fa68998
Jinguang Zhang
688b59b8292830d55fa68999
Jinlan Cao
688b59b8292830d55fa6899a
Jisheng Yin
688b59b8292830d55fa6899b
Junfeng Liu
688b59b8292830d55fa6899c
Junhao Huang
688b59b8292830d55fa6899d
Junzhe Lin
688b59b8292830d55fa6899e
Kaijun Tan
688b59b8292830d55fa6899f
Kaixiang Li
688b59b8292830d55fa689a0
Kang An
688b59b8292830d55fa689a1
Kangheng Lin
688b59b8292830d55fa689a2
Kenkun Liu
688b59b8292830d55fa689a3
Lei Yang
688b59b8292830d55fa689a4
Liang Zhao
688b59b8292830d55fa689a5
Liangyu Chen
688b59b8292830d55fa689a6
Lieyu Shi
688b59b8292830d55fa689a7
Liguo Tan
688b59b8292830d55fa689a8
Lin Lin
688b59b8292830d55fa689a9
Lin Zhang
688b59b8292830d55fa689aa
Lina Chen
688b59b8292830d55fa689ab
Liwen Huang
688b59b8292830d55fa689ac
Liying Shi
688b59b8292830d55fa689ad
Longlong Gu
688b59b8292830d55fa689ae
Mei Chen
688b59b8292830d55fa689af
Mengqiang Ren
688b59b8292830d55fa689b0
Ming Li
688b59b8292830d55fa689b1
Mingzhe Chen
688b59b8292830d55fa689b2
Na Wang
688b59b8292830d55fa689b3
Nan Wu
688b59b8292830d55fa689b4
Qi Han
688b59b8292830d55fa689b5
Qian Zhao
688b59b8292830d55fa689b6
Qiang Zhang
688b59b8292830d55fa689b7
Qianni Liu
688b59b8292830d55fa689b8
Qiaohui Chen
688b59b8292830d55fa689b9
Qiling Wu
688b59b8292830d55fa689ba
Qinglin He
688b59b8292830d55fa689bb
Qinyuan Tan
688b59b8292830d55fa689bc
Qiufeng Wang
688b59b8292830d55fa689bd
Qiuping Wu
688b59b8292830d55fa689be
Qiuyan Liang
688b59b8292830d55fa689bf
Quan Sun
688b59b8292830d55fa689c0
Rui Li
688b59b8292830d55fa689c1
Ruihang Miao
688b59b8292830d55fa689c2
Ruosi Wan
688b59b8292830d55fa689c3
Ruyan Guo
688b59b8292830d55fa689c4
Shangwu Zhong
688b59b8292830d55fa689c5
Shaoliang Pang
688b59b8292830d55fa689c6
Shengjie Fan
688b59b8292830d55fa689c7
Shijie Shang
688b59b8292830d55fa689c8
Shilei Jiang
688b59b8292830d55fa689c9
Shiliang Yang
688b59b8292830d55fa689ca
Shiming Hao
688b59b8292830d55fa689cb
Shuli Gao
688b59b8292830d55fa689cc
Siming Huang
688b59b8292830d55fa689cd
Siqi Liu
688b59b8292830d55fa689ce
Tiancheng Cao
688b59b8292830d55fa689cf
Tianhao Cheng
688b59b8292830d55fa689d0
Tianhao Peng
688b59b8292830d55fa689d1
Wang You
688b59b8292830d55fa689d2
Wei Ji
688b59b8292830d55fa689d3
Wen Sun
688b59b8292830d55fa689d4
Wenjin Deng
688b59b8292830d55fa689d5
Wenqing He
688b59b8292830d55fa689d6
Wenzhen Zheng
688b59b8292830d55fa689d7
Xi Chen
688b59b8292830d55fa689d8
Xiangwen Kong
688b59b8292830d55fa689d9
Xianzhen Luo
688b59b8292830d55fa689da
Xiaobo Yang
688b59b8292830d55fa689db
Xiaojia Liu
688b59b8292830d55fa689dc
Xiaoxiao Ren
688b59b8292830d55fa689dd
Xin Han
688b59b8292830d55fa689de
Xin Li
688b59b8292830d55fa689df
Xin Wu
688b59b8292830d55fa689e0
Xu Zhao
688b59b8292830d55fa689e1
Yanan Wei
688b59b8292830d55fa689e2
Yang Li
688b59b8292830d55fa689e3
Yangguang Li
688b59b8292830d55fa689e4
Yangshijie Xu
688b59b8292830d55fa689e5
Yanming Xu
688b59b8292830d55fa689e6
Yaqiang Shi
688b59b8292830d55fa689e7
Yeqing Shen
688b59b8292830d55fa689e8
Yi Yang
688b59b8292830d55fa689e9
Yifei Yang
688b59b8292830d55fa689ea
Yifeng Gong
688b59b8292830d55fa689eb
Yihan Chen
688b59b8292830d55fa689ec
Yijing Yang
688b59b8292830d55fa689ed
Yinmin Zhang
688b59b8292830d55fa689ee
Yizhuang Zhou
688b59b8292830d55fa689ef
Yuanhao Ding
688b59b8292830d55fa689f0
Yuantao Fan
688b59b8292830d55fa689f1
Yuanzhen Yang
688b59b8292830d55fa689f2
Yuchu Luo
688b59b8292830d55fa689f3
Yue Peng
688b59b8292830d55fa689f4
Yufan Lu
688b59b8292830d55fa689f5
Yuhang Deng
688b59b8292830d55fa689f6
Yuhe Yin
688b59b8292830d55fa689f7
Yujie Liu
688b59b8292830d55fa689f8
Yukun Chen
688b59b8292830d55fa689f9
Yuling Zhao
688b59b8292830d55fa689fa
Yun Mou
688b59b8292830d55fa689fb
Yunlong Li
688b59b8292830d55fa689fc
Yunzhou Ju
688b59b8292830d55fa689fd
Yusheng Li
688b59b8292830d55fa689fe
Yuxiang Yang
688b59b8292830d55fa689ff
Yuxiang Zhang
688b59b8292830d55fa68a00
Yuyang Chen
688b59b8292830d55fa68a01
Zejia Weng
688b59b8292830d55fa68a02
Zhe Xie
688b59b8292830d55fa68a03
Zheng Ge
688b59b8292830d55fa68a04
Zheng Gong
688b59b8292830d55fa68a05
Zhenyi Lu
688b59b8292830d55fa68a06
Zhewei Huang
claimed_verified
2025-07-31T21:00:24.735Z
63ae768220176b2d216ac807
/avatars/d658b98f469bd12f5bc3bb9c12811780.svg
Zhewei Huang
user
hzwer
688b59b8292830d55fa68a07
Zhichao Chang
688b59b8292830d55fa68a08
Zhiguo Huang
688b59b8292830d55fa68a09
Zhirui Wang
688b59b8292830d55fa68a0a
Zidong Yang
688b59b8292830d55fa68a0b
Zili Wang
688b59b8292830d55fa68a0c
Ziqi Wang
688b59b8292830d55fa68a0d
Zixin Zhang
688b59b8292830d55fa68a0e
Binxing Jiao
688b59b8292830d55fa68a0f
Daxin Jiang
688b59b8292830d55fa68a10
Heung-Yeung Shum
688b59b8292830d55fa68a11
Xiangyu Zhang
Step-3 is Large yet Affordable: Model-system Co-design for
  Cost-effective Decoding
Large language models (LLMs) face low hardware efficiency during decoding,
especially for long-context reasoning tasks. This paper introduces Step-3, a
321B-parameter VLM with hardware-aware model-system co-design optimized for
minimizing decoding costs. Step-3 innovates in two key dimensions: (1) A novel
Multi-Matrix Factorization Attention (MFA) mechanism that significantly reduces
both KV cache size and computation while maintaining high attention
expressiveness, and (2) Attention-FFN Disaggregation (AFD), a distributed
inference system that decouples attention and Feed-Forward Network (FFN) layers
into specialized subsystems. This co-design achieves unprecedented cost
efficiency: Step-3 significantly reduces theoretical decoding costs compared
with models like DeepSeek-V3 and Qwen3 MoE 235B, with the gains widening at
longer context. Step-3 achieves low cost while activating 38B parameters per
token (more than DeepSeek-V3 and Qwen3 MoE 235B), demonstrating that
hardware-aligned attention arithmetic intensity, MoE sparsity, and AFD are
critical to cost-effectiveness. We perform a head-to-head comparison with
DeepSeek-V3 in its favorable scenarios. Our implementation on Hopper GPUs
achieves a decoding throughput of up to 4,039 tokens per second per GPU under
50ms TPOT SLA (4K context, FP8, no MTP). It is higher than DeepSeek-V3's 2,324
in the same setup and sets a new Pareto frontier for LLM decoding.
Step-3, a 321B-parameter VLM, optimizes decoding costs through Multi-Matrix Factorization Attention and Attention-FFN Disaggregation, achieving high efficiency and throughput compared to other models.
Multi-Matrix Factorization Attention Attention-FFN Disaggregation VLM KV cache decoding costs DeepSeek-V3 Qwen3 MoE hardware-aware model-system co-design attention arithmetic intensity MoE sparsity decoding throughput TPOT SLA Hopper GPUs