From Word Models to World Models: Translating from Natural Language to
  the Probabilistic Language of Thought
How does language inform our downstream thinking? In particular, how do
humans make meaning from language -- and how can we leverage a theory of
linguistic meaning to build machines that think in more human-like ways? In
this paper, we propose rational meaning construction, a computational
framework for language-informed thinking that combines neural models of
language with probabilistic models for rational inference. We frame linguistic
meaning as a context-sensitive mapping from natural language into a
probabilistic language of thought (PLoT) -- a general-purpose symbolic
substrate for probabilistic, generative world modeling. Our architecture
integrates two powerful computational tools that have not previously come
together: we model thinking with probabilistic programs, an expressive
representation for flexible commonsense reasoning; and we model meaning
construction with large language models (LLMs), which support
broad-coverage translation from natural language utterances to code expressions
in a probabilistic programming language. We illustrate our framework in action
through examples covering four core domains from cognitive science:
probabilistic reasoning, logical and relational reasoning, visual and physical
reasoning, and social reasoning about agents and their plans. In each, we show
that LLMs can generate context-sensitive translations that capture
pragmatically-appropriate linguistic meanings, while Bayesian inference with
the generated programs supports coherent and robust commonsense reasoning. We
extend our framework to integrate cognitively-motivated symbolic modules to
provide a unified commonsense thinking interface from language. Finally, we
explore how language can drive the construction of world models themselves.
6494ee749a0573a99591e839
Lionel Wong
6494ee749a0573a99591e83a
Gabriel Grand
admin_assigned
2023-06-23T08:45:54.188Z
60b90fdd48cc96f40ef4fda6
/avatars/a5bbf261fd2f4db784b83b52be014bc5.svg
Gabe Grand
user
gabegrand
6494ee749a0573a99591e83b
Alexander K. Lew
6494ee749a0573a99591e83c
Noah D. Goodman
claimed_verified
2024-11-15T09:56:00.189Z
67321274c1f20c742bcf7a8d
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/ltcQhre6eDRVzn6Vbbyhu.png
Noah D. Goodman
user
ngoodman
6494ee749a0573a99591e83d
Vikash K. Mansinghka
6494ee749a0573a99591e83e
Jacob Andreas
6494ee749a0573a99591e83f
Joshua B. Tenenbaum
From Word Models to World Models: Translating from Natural Language to
  the Probabilistic Language of Thought
How does language inform our downstream thinking? In particular, how do
humans make meaning from language -- and how can we leverage a theory of
linguistic meaning to build machines that think in more human-like ways? In
this paper, we propose rational meaning construction, a computational
framework for language-informed thinking that combines neural models of
language with probabilistic models for rational inference. We frame linguistic
meaning as a context-sensitive mapping from natural language into a
probabilistic language of thought (PLoT) -- a general-purpose symbolic
substrate for probabilistic, generative world modeling. Our architecture
integrates two powerful computational tools that have not previously come
together: we model thinking with probabilistic programs, an expressive
representation for flexible commonsense reasoning; and we model meaning
construction with large language models (LLMs), which support
broad-coverage translation from natural language utterances to code expressions
in a probabilistic programming language. We illustrate our framework in action
through examples covering four core domains from cognitive science:
probabilistic reasoning, logical and relational reasoning, visual and physical
reasoning, and social reasoning about agents and their plans. In each, we show
that LLMs can generate context-sensitive translations that capture
pragmatically-appropriate linguistic meanings, while Bayesian inference with
the generated programs supports coherent and robust commonsense reasoning. We
extend our framework to integrate cognitively-motivated symbolic modules to
provide a unified commonsense thinking interface from language. Finally, we
explore how language can drive the construction of world models themselves.
A framework combining large language models and probabilistic programs translates language into a symbolic language of thought for rational, commonsense reasoning across multiple cognitive domains.
rational meaning construction probabilistic language of thought (PLoT) probabilistic programs large language models (LLMs) probabilistic reasoning logical and relational reasoning visual and physical reasoning social reasoning Bayesian inference symbolic modules world models