CBT-Bench: Evaluating Large Language Models on Assisting Cognitive
  Behavior Therapy
There is a significant gap between patient needs and available mental health
support today. In this paper, we aim to thoroughly examine the potential of
using Large Language Models (LLMs) to assist professional psychotherapy. To
this end, we propose a new benchmark, CBT-BENCH, for the systematic evaluation
of cognitive behavioral therapy (CBT) assistance. We include three levels of
tasks in CBT-BENCH: I: Basic CBT knowledge acquisition, with the task of
multiple-choice questions; II: Cognitive model understanding, with the tasks of
cognitive distortion classification, primary core belief classification, and
fine-grained core belief classification; III: Therapeutic response generation,
with the task of generating responses to patient speech in CBT therapy
sessions. These tasks encompass key aspects of CBT that could potentially be
enhanced through AI assistance, while also outlining a hierarchy of capability
requirements, ranging from basic knowledge recitation to engaging in real
therapeutic conversations. We evaluated representative LLMs on our benchmark.
Experimental results indicate that while LLMs perform well in reciting CBT
knowledge, they fall short in complex real-world scenarios requiring deep
analysis of patients' cognitive structures and generating effective responses,
suggesting potential future work.
6712cece5fd735b90e62e60e
Mian Zhang
claimed_verified
2024-10-21T07:48:04.871Z
650857fef3060ea840ffbbfe
/avatars/f9c87a75c9e5cf829b7d60102a3b7096.svg
Mian Zhang
user
billmianz
6712cece5fd735b90e62e60f
Xianjun Yang
admin_assigned
2024-10-22T09:40:13.798Z
604137e5c15e823a685f2af2
https://cdn-avatars.huggingface.co/v1/production/uploads/604137e5c15e823a685f2af2/rrxZUyrbLpKqCmG6xtP8v.jpeg
Xianjun Yang
user
Xianjun
6712cece5fd735b90e62e610
Xinlu Zhang
admin_assigned
2024-10-22T09:40:32.686Z
650b78532ec41a31f31163e9
/avatars/522549da5ee5703b41396ce0b6811e89.svg
xinlu zhang
user
xz97
6712cece5fd735b90e62e611
Travis Labrum
6712cece5fd735b90e62e612
Jamie C. Chiu
6712cece5fd735b90e62e613
Shaun M. Eack
6712cece5fd735b90e62e614
Fei Fang
6712cece5fd735b90e62e615
William Yang Wang
admin_assigned
2024-10-22T09:41:04.582Z
6476607efb22e3b77f3f1452
/avatars/22f40c81e6b6e09bdb66a5afd978f98b.svg
William Yang Wang
user
wangwilliamyang
6712cece5fd735b90e62e616
Zhiyu Zoey Chen
CBT-Bench: Evaluating Large Language Models on Assisting Cognitive
  Behavior Therapy
There is a significant gap between patient needs and available mental health
support today. In this paper, we aim to thoroughly examine the potential of
using Large Language Models (LLMs) to assist professional psychotherapy. To
this end, we propose a new benchmark, CBT-BENCH, for the systematic evaluation
of cognitive behavioral therapy (CBT) assistance. We include three levels of
tasks in CBT-BENCH: I: Basic CBT knowledge acquisition, with the task of
multiple-choice questions; II: Cognitive model understanding, with the tasks of
cognitive distortion classification, primary core belief classification, and
fine-grained core belief classification; III: Therapeutic response generation,
with the task of generating responses to patient speech in CBT therapy
sessions. These tasks encompass key aspects of CBT that could potentially be
enhanced through AI assistance, while also outlining a hierarchy of capability
requirements, ranging from basic knowledge recitation to engaging in real
therapeutic conversations. We evaluated representative LLMs on our benchmark.
Experimental results indicate that while LLMs perform well in reciting CBT
knowledge, they fall short in complex real-world scenarios requiring deep
analysis of patients' cognitive structures and generating effective responses,
suggesting potential future work.
A new benchmark, CBT-BENCH, evaluates LLM performance in cognitive behavioral therapy tasks, highlighting strengths in knowledge recitation but limitations in complex therapeutic response generation.
Large Language Models (LLMs) cognitive behavioral therapy (CBT) CBT-BENCH multiple-choice questions cognitive distortion classification primary core belief classification fine-grained core belief classification therapeutic response generation cognitive structures