Small Language Models are the Future of Agentic AI
Large language models (LLMs) are often praised for exhibiting near-human
performance on a wide range of tasks and valued for their ability to hold a
general conversation. The rise of agentic AI systems is, however, ushering in a
mass of applications in which language models perform a small number of
specialized tasks repetitively and with little variation.
  Here we lay out the position that small language models (SLMs) are
sufficiently powerful, inherently more suitable, and necessarily more
economical for many invocations in agentic systems, and are therefore the
future of agentic AI. Our argumentation is grounded in the current level of
capabilities exhibited by SLMs, the common architectures of agentic systems,
and the economy of LM deployment. We further argue that in situations where
general-purpose conversational abilities are essential, heterogeneous agentic
systems (i.e., agents invoking multiple different models) are the natural
choice. We discuss the potential barriers for the adoption of SLMs in agentic
systems and outline a general LLM-to-SLM agent conversion algorithm.
  Our position, formulated as a value statement, highlights the significance of
the operational and economic impact even a partial shift from LLMs to SLMs is
to have on the AI agent industry. We aim to stimulate the discussion on the
effective use of AI resources and hope to advance the efforts to lower the
costs of AI of the present day. Calling for both contributions to and critique
of our position, we commit to publishing all such correspondence at
https://research.nvidia.com/labs/lpr/slm-agents.
6841c780655161e7f1207108
Peter Belcak
claimed_verified
2025-07-07T11:25:57.865Z
63e8cccddd2c4effdd6283cf
/avatars/b289d4dda0aafd60af3f14e19837b69c.svg
Peter Belcak
user
pbelcak
6841c780655161e7f1207109
Greg Heinrich
claimed_verified
2025-07-18T07:50:55.318Z
64c23e06203c836866ed0bdd
https://cdn-avatars.huggingface.co/v1/production/uploads/64c23e06203c836866ed0bdd/rPUrP8PqL1dhWNO6L3XwI.png
Greg Heinrich
user
gheinrich
6841c780655161e7f120710a
Shizhe Diao
6841c780655161e7f120710b
Yonggan Fu
6841c780655161e7f120710c
Xin Dong
6841c780655161e7f120710d
Saurav Muralidharan
6841c780655161e7f120710e
Yingyan Celine Lin
6841c780655161e7f120710f
Pavlo Molchanov
Small Language Models are the Future of Agentic AI
Large language models (LLMs) are often praised for exhibiting near-human
performance on a wide range of tasks and valued for their ability to hold a
general conversation. The rise of agentic AI systems is, however, ushering in a
mass of applications in which language models perform a small number of
specialized tasks repetitively and with little variation.
  Here we lay out the position that small language models (SLMs) are
sufficiently powerful, inherently more suitable, and necessarily more
economical for many invocations in agentic systems, and are therefore the
future of agentic AI. Our argumentation is grounded in the current level of
capabilities exhibited by SLMs, the common architectures of agentic systems,
and the economy of LM deployment. We further argue that in situations where
general-purpose conversational abilities are essential, heterogeneous agentic
systems (i.e., agents invoking multiple different models) are the natural
choice. We discuss the potential barriers for the adoption of SLMs in agentic
systems and outline a general LLM-to-SLM agent conversion algorithm.
  Our position, formulated as a value statement, highlights the significance of
the operational and economic impact even a partial shift from LLMs to SLMs is
to have on the AI agent industry. We aim to stimulate the discussion on the
effective use of AI resources and hope to advance the efforts to lower the
costs of AI of the present day. Calling for both contributions to and critique
of our position, we commit to publishing all such correspondence at
https://research.nvidia.com/labs/lpr/slm-agents.
Small language models are argued to be more suitable, powerful, and economical for specialized, repetitive tasks in agentic systems, with heterogeneous systems using multiple models where general conversational abilities are needed.
large language models (LLMs) small language models (SLMs) agentic AI systems heterogeneous agentic systems LLM-to-SLM agent conversion