NewtonBench: Benchmarking Generalizable Scientific Law Discovery in LLM
  Agents
Large language models are emerging as powerful tools for scientific law
discovery, a foundational challenge in AI-driven science. However, existing
benchmarks for this task suffer from a fundamental methodological trilemma,
forcing a trade-off between scientific relevance, scalability, and resistance
to memorization. Furthermore, they oversimplify discovery as static function
fitting, failing to capture the authentic scientific process of uncovering
embedded laws through the interactive exploration of complex model systems. To
address these critical gaps, we introduce NewtonBench, a benchmark comprising
324 scientific law discovery tasks across 12 physics domains. Our design
mitigates the evaluation trilemma by using metaphysical shifts - systematic
alterations of canonical laws - to generate a vast suite of problems that are
scalable, scientifically relevant, and memorization-resistant. Moreover, we
elevate the evaluation from static function fitting to interactive model
discovery, requiring agents to experimentally probe simulated complex systems
to uncover hidden principles. Our extensive experiment reveals a clear but
fragile capability for discovery in frontier LLMs: this ability degrades
precipitously with increasing system complexity and exhibits extreme
sensitivity to observational noise. Notably, we uncover a paradoxical effect of
tool assistance: providing a code interpreter can hinder more capable models by
inducing a premature shift from exploration to exploitation, causing them to
satisfice on suboptimal solutions. These results demonstrate that robust,
generalizable discovery in complex, interactive environments remains the core
challenge. By providing a scalable, robust, and scientifically authentic
testbed, NewtonBench offers a crucial tool for measuring true progress and
guiding the development of next-generation AI agents capable of genuine
scientific discovery.
68e7b69644180c42eca8afe9
Tianshi Zheng
claimed_verified
2025-10-11T13:48:26.316Z
626fbdcca8dcec0066f4ae12
/avatars/1e4e174c425943b7dde29c140d322478.svg
Tianshi ZHENG
user
StoneTZHENG
68e7b69644180c42eca8afea
Kelvin Kiu-Wai Tam
68e7b69644180c42eca8afeb
Newt Hue-Nam K. Nguyen
claimed_verified
2025-10-11T14:04:53.298Z
68e88dabd0fb027e12184a8f
/avatars/4bcb3282da5213c3c1c23ac3742f2b32.svg
Newt Nguyen
user
newtdes
68e7b69644180c42eca8afec
Baixuan Xu
68e7b69644180c42eca8afed
Zhaowei Wang
68e7b69644180c42eca8afee
Jiayang Cheng
68e7b69644180c42eca8afef
Hong Ting Tsang
68e7b69644180c42eca8aff0
Weiqi Wang
68e7b69644180c42eca8aff1
Jiaxin Bai
68e7b69644180c42eca8aff2
Tianqing Fang
claimed_verified
2025-10-10T09:13:21.705Z
641129818573c51c0458b793
/avatars/d4bc67c160a07146cf41c614678aa36b.svg
Tianqing Fang
user
tqfang229
68e7b69644180c42eca8aff3
Yangqiu Song
68e7b69644180c42eca8aff4
Ginny Y. Wong
68e7b69644180c42eca8aff5
Simon See
NewtonBench: Benchmarking Generalizable Scientific Law Discovery in LLM
  Agents
Large language models are emerging as powerful tools for scientific law
discovery, a foundational challenge in AI-driven science. However, existing
benchmarks for this task suffer from a fundamental methodological trilemma,
forcing a trade-off between scientific relevance, scalability, and resistance
to memorization. Furthermore, they oversimplify discovery as static function
fitting, failing to capture the authentic scientific process of uncovering
embedded laws through the interactive exploration of complex model systems. To
address these critical gaps, we introduce NewtonBench, a benchmark comprising
324 scientific law discovery tasks across 12 physics domains. Our design
mitigates the evaluation trilemma by using metaphysical shifts - systematic
alterations of canonical laws - to generate a vast suite of problems that are
scalable, scientifically relevant, and memorization-resistant. Moreover, we
elevate the evaluation from static function fitting to interactive model
discovery, requiring agents to experimentally probe simulated complex systems
to uncover hidden principles. Our extensive experiment reveals a clear but
fragile capability for discovery in frontier LLMs: this ability degrades
precipitously with increasing system complexity and exhibits extreme
sensitivity to observational noise. Notably, we uncover a paradoxical effect of
tool assistance: providing a code interpreter can hinder more capable models by
inducing a premature shift from exploration to exploitation, causing them to
satisfice on suboptimal solutions. These results demonstrate that robust,
generalizable discovery in complex, interactive environments remains the core
challenge. By providing a scalable, robust, and scientifically authentic
testbed, NewtonBench offers a crucial tool for measuring true progress and
guiding the development of next-generation AI agents capable of genuine
scientific discovery.
NewtonBench is a benchmark for scientific law discovery that addresses scalability, scientific relevance, and memorization resistance by using metaphysical shifts and interactive model discovery.
large language models scientific law discovery NewtonBench metaphysical shifts interactive model discovery observational noise code interpreter exploration exploitation