LLM-SRBench: A New Benchmark for Scientific Equation Discovery with
  Large Language Models
Scientific equation discovery is a fundamental task in the history of
scientific progress, enabling the derivation of laws governing natural
phenomena. Recently, Large Language Models (LLMs) have gained interest for this
task due to their potential to leverage embedded scientific knowledge for
hypothesis generation. However, evaluating the true discovery capabilities of
these methods remains challenging, as existing benchmarks often rely on common
equations that are susceptible to memorization by LLMs, leading to inflated
performance metrics that do not reflect discovery. In this paper, we introduce
LLM-SRBench, a comprehensive benchmark with 239 challenging problems across
four scientific domains specifically designed to evaluate LLM-based scientific
equation discovery methods while preventing trivial memorization. Our benchmark
comprises two main categories: LSR-Transform, which transforms common physical
models into less common mathematical representations to test reasoning beyond
memorized forms, and LSR-Synth, which introduces synthetic, discovery-driven
problems requiring data-driven reasoning. Through extensive evaluation of
several state-of-the-art methods, using both open and closed LLMs, we find that
the best-performing system so far achieves only 31.5% symbolic accuracy. These
findings highlight the challenges of scientific equation discovery, positioning
LLM-SRBench as a valuable resource for future research.
67fddc276a3c533dc1d3bcbe
Parshin Shojaee
extracted_confirmed
2025-06-10T02:31:30.711Z
6520621836008ecc88699622
/avatars/b08c00af00f1736a4f4938443e575b0e.svg
Parshin Shojaee
user
parshinsh
67fddc276a3c533dc1d3bcbf
Ngoc-Hieu Nguyen
67fddc276a3c533dc1d3bcc0
Kazem Meidani
claimed_verified
2025-04-15T07:53:43.431Z
67b4cdee376cfc783f9ec8cf
/avatars/d9a9f320cd01dc5addfca14feefefef4.svg
Meidani
user
mkmeidani
67fddc276a3c533dc1d3bcc1
Amir Barati Farimani
67fddc276a3c533dc1d3bcc2
Khoa D Doan
67fddc276a3c533dc1d3bcc3
Chandan K Reddy
LLM-SRBench: A New Benchmark for Scientific Equation Discovery with
  Large Language Models
Scientific equation discovery is a fundamental task in the history of
scientific progress, enabling the derivation of laws governing natural
phenomena. Recently, Large Language Models (LLMs) have gained interest for this
task due to their potential to leverage embedded scientific knowledge for
hypothesis generation. However, evaluating the true discovery capabilities of
these methods remains challenging, as existing benchmarks often rely on common
equations that are susceptible to memorization by LLMs, leading to inflated
performance metrics that do not reflect discovery. In this paper, we introduce
LLM-SRBench, a comprehensive benchmark with 239 challenging problems across
four scientific domains specifically designed to evaluate LLM-based scientific
equation discovery methods while preventing trivial memorization. Our benchmark
comprises two main categories: LSR-Transform, which transforms common physical
models into less common mathematical representations to test reasoning beyond
memorized forms, and LSR-Synth, which introduces synthetic, discovery-driven
problems requiring data-driven reasoning. Through extensive evaluation of
several state-of-the-art methods, using both open and closed LLMs, we find that
the best-performing system so far achieves only 31.5% symbolic accuracy. These
findings highlight the challenges of scientific equation discovery, positioning
LLM-SRBench as a valuable resource for future research.
LLM-SRBench is a comprehensive benchmark designed to evaluate LLMs in scientific equation discovery by preventing memorization and using novel problem representations, indicating significant challenges in this domain.
Large Language Models LLM-SRBench scientific equation discovery LSR-Transform LSR-Synth symbolic accuracy