Provable Benefits of In-Tool Learning for Large Language Models
Tool-augmented language models, equipped with retrieval, memory, or external
APIs, are reshaping AI, yet their theoretical advantages remain underexplored.
In this paper, we address this question by demonstrating the benefits of
in-tool learning (external retrieval) over in-weight learning (memorization)
for factual recall. We show that the number of facts a model can memorize
solely in its weights is fundamentally limited by its parameter count. In
contrast, we prove that tool-use enables unbounded factual recall via a simple
and efficient circuit construction. These results are validated in controlled
experiments, where tool-using models consistently outperform memorizing ones.
We further show that for pretrained large language models, teaching tool-use
and general rules is more effective than finetuning facts into memory. Our work
provides both a theoretical and empirical foundation, establishing why
tool-augmented workflows are not just practical, but provably more scalable.
68b12c01b19c5400014849db
Sam Houliston
68b12c01b19c5400014849dc
Ambroise Odonnat
68b12c01b19c5400014849dd
Charles Arnal
68b12c01b19c5400014849de
Vivien Cabannes
Provable Benefits of In-Tool Learning for Large Language Models
Tool-augmented language models, equipped with retrieval, memory, or external
APIs, are reshaping AI, yet their theoretical advantages remain underexplored.
In this paper, we address this question by demonstrating the benefits of
in-tool learning (external retrieval) over in-weight learning (memorization)
for factual recall. We show that the number of facts a model can memorize
solely in its weights is fundamentally limited by its parameter count. In
contrast, we prove that tool-use enables unbounded factual recall via a simple
and efficient circuit construction. These results are validated in controlled
experiments, where tool-using models consistently outperform memorizing ones.
We further show that for pretrained large language models, teaching tool-use
and general rules is more effective than finetuning facts into memory. Our work
provides both a theoretical and empirical foundation, establishing why
tool-augmented workflows are not just practical, but provably more scalable.
Tool-augmented language models achieve better factual recall and scalability through external retrieval rather than memorization in their weights.
in-tool learning in-weight learning factual recall parameter count tool-use unbounded factual recall circuit construction pretrained large language models teaching tool-use general rules finetuning