Technologies on Effectiveness and Efficiency: A Survey of State Spaces
  Models
State Space Models (SSMs) have emerged as a promising alternative to the
popular transformer-based models and have been increasingly gaining attention.
Compared to transformers, SSMs excel at tasks with sequential data or longer
contexts, demonstrating comparable performances with significant efficiency
gains. In this survey, we provide a coherent and systematic overview for SSMs,
including their theoretical motivations, mathematical formulations, comparison
with existing model classes, and various applications. We divide the SSM series
into three main sections, providing a detailed introduction to the original
SSM, the structured SSM represented by S4, and the selective SSM typified by
Mamba. We put an emphasis on technicality, and highlight the various key
techniques introduced to address the effectiveness and efficiency of SSMs. We
hope this manuscript serves as an introduction for researchers to explore the
theoretical foundations of SSMs.
67d788b6ba098a0651e1e235
Xingtai Lv
claimed_verified
2025-03-17T08:44:34.410Z
663f07d029be04778ba97871
/avatars/fb7c9d4a2c537d918a3267e7cbc03f04.svg
Xingtai Lv
user
XingtaiHF
67d788b6ba098a0651e1e236
Youbang Sun
admin_assigned
2025-03-17T08:50:17.568Z
679ce8c048ebd7903d76a832
/avatars/5f3fecaacfee6e2d5a72dd19fe87055a.svg
Youbang Sun
user
Youbang
67d788b6ba098a0651e1e237
Kaiyan Zhang
claimed_verified
2025-03-17T08:44:26.057Z
60bc94cd85a3ab33829b6211
/avatars/b57d36c7577fbbb42ea5b963eef4144a.svg
Kaiyan Zhang
user
iseesaw
67d788b6ba098a0651e1e238
Shang Qu
claimed_verified
2025-03-17T14:42:23.487Z
65597738deee83130a1301d5
/avatars/9bcc40aebe4db079927675d95c00463c.svg
Shang (Lindsay) Qu
user
lindsay-qu
67d788b6ba098a0651e1e239
Xuekai Zhu
admin_assigned
2025-03-17T08:50:38.118Z
647ffddeb82adfa7cc1a10d9
/avatars/26aa168d6b2068298ebb16584aa52b6c.svg
zhu
user
xuekai
67d788b6ba098a0651e1e23a
Yuchen Fan
admin_assigned
2025-03-17T08:50:54.445Z
672c2d7816766a76a747b7b5
/avatars/12c7b26d2b81721ccac3a5c71e32a1a1.svg
Yuchen Fan
user
yuchenFan
67d788b6ba098a0651e1e23b
Yi Wu
67d788b6ba098a0651e1e23c
Ermo Hua
claimed_verified
2025-03-17T08:44:30.639Z
6445fa2ffc22e309d78bef3e
https://cdn-avatars.huggingface.co/v1/production/uploads/6445fa2ffc22e309d78bef3e/FQaINLd0PjgY9EnK_APRk.jpeg
Messi Hua
user
Messi-Hua
67d788b6ba098a0651e1e23d
Xinwei Long
admin_assigned
2025-03-17T08:51:02.068Z
667e577139b49eba118d569f
/avatars/1a26dd96b4b352b8968561750ecae9a7.svg
Xinwei Long
user
xinwei666
67d788b6ba098a0651e1e23e
Ning Ding
admin_assigned
2025-03-17T08:51:08.621Z
677b80e31ad30ab2c798e776
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/X8IFnIK3TDHOGKZCzLTe8.jpeg
Ning Ding
user
BradPitt2025
67d788b6ba098a0651e1e23f
Bowen Zhou
admin_assigned
2025-03-17T08:51:15.825Z
669f614b59adf5b56e05bce3
/avatars/ffd4189efbceb0e63a03db273065a44b.svg
BowenZhou
user
bowenZhou
Technologies on Effectiveness and Efficiency: A Survey of State Spaces
  Models
State Space Models (SSMs) have emerged as a promising alternative to the
popular transformer-based models and have been increasingly gaining attention.
Compared to transformers, SSMs excel at tasks with sequential data or longer
contexts, demonstrating comparable performances with significant efficiency
gains. In this survey, we provide a coherent and systematic overview for SSMs,
including their theoretical motivations, mathematical formulations, comparison
with existing model classes, and various applications. We divide the SSM series
into three main sections, providing a detailed introduction to the original
SSM, the structured SSM represented by S4, and the selective SSM typified by
Mamba. We put an emphasis on technicality, and highlight the various key
techniques introduced to address the effectiveness and efficiency of SSMs. We
hope this manuscript serves as an introduction for researchers to explore the
theoretical foundations of SSMs.
This survey provides a comprehensive overview of State Space Models (SSMs), their mathematical foundations, and their applications, emphasizing their efficiency and effectiveness compared to transformer-based models.
State Space Models SSMs transformers sequential data longer contexts theoretical motivations mathematical formulations structured SSM S4 selective SSM Mamba