General Reasoning Requires Learning to Reason from the Get-go
Large Language Models (LLMs) have demonstrated impressive real-world utility,
exemplifying artificial useful intelligence (AUI). However, their ability to
reason adaptively and robustly -- the hallmarks of artificial general
intelligence (AGI) -- remains fragile. While LLMs seemingly succeed in
commonsense reasoning, programming, and mathematics, they struggle to
generalize algorithmic understanding across novel contexts. Our experiments
with algorithmic tasks in esoteric programming languages reveal that LLM's
reasoning overfits to the training data and is limited in its transferability.
We hypothesize that the core issue underlying such limited transferability is
the coupling of reasoning and knowledge in LLMs.
  To transition from AUI to AGI, we propose disentangling knowledge and
reasoning through three key directions: (1) pretaining to reason using RL from
scratch as an alternative to the widely used next-token prediction pretraining,
(2) using a curriculum of synthetic tasks to ease the learning of a
reasoning prior for RL that can then be transferred to natural
language tasks, and (3) learning more generalizable reasoning functions using a
small context window to reduce exploiting spurious correlations between tokens.
Such a reasoning system coupled with a trained retrieval system and a large
external memory bank as a knowledge store can overcome several limitations of
existing architectures at learning to reason in novel scenarios.
67c66a6321d722b4247e5959
Seungwook Han
admin_assigned
2025-03-04T16:08:58.266Z
6520d6db2a16045c092b3b36
/avatars/dab34f141a1aef39d00c789ff85e729f.svg
Seungwook Han
user
hanseungwook
67c66a6321d722b4247e595a
Jyothish Pari
67c66a6321d722b4247e595b
Samuel J. Gershman
extracted_confirmed
2025-03-04T13:57:29.748Z
6520d6db2a16045c092b3b36
/avatars/dab34f141a1aef39d00c789ff85e729f.svg
Seungwook Han
user
hanseungwook
67c66a6321d722b4247e595c
Pulkit Agrawal
General Reasoning Requires Learning to Reason from the Get-go
Large Language Models (LLMs) have demonstrated impressive real-world utility,
exemplifying artificial useful intelligence (AUI). However, their ability to
reason adaptively and robustly -- the hallmarks of artificial general
intelligence (AGI) -- remains fragile. While LLMs seemingly succeed in
commonsense reasoning, programming, and mathematics, they struggle to
generalize algorithmic understanding across novel contexts. Our experiments
with algorithmic tasks in esoteric programming languages reveal that LLM's
reasoning overfits to the training data and is limited in its transferability.
We hypothesize that the core issue underlying such limited transferability is
the coupling of reasoning and knowledge in LLMs.
  To transition from AUI to AGI, we propose disentangling knowledge and
reasoning through three key directions: (1) pretaining to reason using RL from
scratch as an alternative to the widely used next-token prediction pretraining,
(2) using a curriculum of synthetic tasks to ease the learning of a
reasoning prior for RL that can then be transferred to natural
language tasks, and (3) learning more generalizable reasoning functions using a
small context window to reduce exploiting spurious correlations between tokens.
Such a reasoning system coupled with a trained retrieval system and a large
external memory bank as a knowledge store can overcome several limitations of
existing architectures at learning to reason in novel scenarios.
Proposes disentangling knowledge and reasoning in Large Language Models to improve generalization and adaptability across novel contexts through reinforcement learning, synthetic task curriculums, and generalizable reasoning functions.
reinforcement learning curriculum learning reasoning prior synthetic tasks generalizable reasoning functions retrieval system external memory bank