Toward Reliable Biomedical Hypothesis Generation: Evaluating
  Truthfulness and Hallucination in Large Language Models
Large language models (LLMs) have shown significant potential in scientific
disciplines such as biomedicine, particularly in hypothesis generation, where
they can analyze vast literature, identify patterns, and suggest research
directions. However, a key challenge lies in evaluating the truthfulness of
generated hypotheses, as verifying their accuracy often requires substantial
time and resources. Additionally, the hallucination problem in LLMs can lead to
the generation of hypotheses that appear plausible but are ultimately
incorrect, undermining their reliability. To facilitate the systematic study of
these challenges, we introduce TruthHypo, a benchmark for assessing the
capabilities of LLMs in generating truthful biomedical hypotheses, and KnowHD,
a knowledge-based hallucination detector to evaluate how well hypotheses are
grounded in existing knowledge. Our results show that LLMs struggle to generate
truthful hypotheses. By analyzing hallucinations in reasoning steps, we
demonstrate that the groundedness scores provided by KnowHD serve as an
effective metric for filtering truthful hypotheses from the diverse outputs of
LLMs. Human evaluations further validate the utility of KnowHD in identifying
truthful hypotheses and accelerating scientific discovery. Our data and source
code are available at https://github.com/Teddy-XiongGZ/TruthHypo.
683a4c9070c36271751e3fbc
Guangzhi Xiong
683a4c9070c36271751e3fbd
Eric Xie
683a4c9070c36271751e3fbe
Corey Williams
683a4c9070c36271751e3fbf
Myles Kim
683a4c9070c36271751e3fc0
Amir Hassan Shariatmadari
683a4c9070c36271751e3fc1
Sikun Guo
683a4c9070c36271751e3fc2
Stefan Bekiranov
683a4c9070c36271751e3fc3
Aidong Zhang
Toward Reliable Biomedical Hypothesis Generation: Evaluating
  Truthfulness and Hallucination in Large Language Models
Large language models (LLMs) have shown significant potential in scientific
disciplines such as biomedicine, particularly in hypothesis generation, where
they can analyze vast literature, identify patterns, and suggest research
directions. However, a key challenge lies in evaluating the truthfulness of
generated hypotheses, as verifying their accuracy often requires substantial
time and resources. Additionally, the hallucination problem in LLMs can lead to
the generation of hypotheses that appear plausible but are ultimately
incorrect, undermining their reliability. To facilitate the systematic study of
these challenges, we introduce TruthHypo, a benchmark for assessing the
capabilities of LLMs in generating truthful biomedical hypotheses, and KnowHD,
a knowledge-based hallucination detector to evaluate how well hypotheses are
grounded in existing knowledge. Our results show that LLMs struggle to generate
truthful hypotheses. By analyzing hallucinations in reasoning steps, we
demonstrate that the groundedness scores provided by KnowHD serve as an
effective metric for filtering truthful hypotheses from the diverse outputs of
LLMs. Human evaluations further validate the utility of KnowHD in identifying
truthful hypotheses and accelerating scientific discovery. Our data and source
code are available at https://github.com/Teddy-XiongGZ/TruthHypo.
TruthHypo and KnowHD assess and filter truthful biomedical hypotheses generated by large language models, addressing the challenge of hallucination and truthfulness in scientific research.
large language models LLMs hypothesis generation truthfulness hallucination biomedical hypotheses benchmark knowledge-based hallucination detector groundedness scores human evaluations