Preference Learning Unlocks LLMs' Psycho-Counseling Skills
Applying large language models (LLMs) to assist in psycho-counseling is an
emerging and meaningful approach, driven by the significant gap between patient
needs and the availability of mental health support. However, current LLMs
struggle to consistently provide effective responses to client speeches,
largely due to the lack of supervision from high-quality real psycho-counseling
data, whose content is typically inaccessible due to client privacy concerns.
Furthermore, the quality of therapists' responses in available sessions can
vary significantly based on their professional training and experience.
Assessing the quality of therapists' responses remains an open challenge. In
this work, we address these challenges by first proposing a set of professional
and comprehensive principles to evaluate therapists' responses to client
speeches. Using these principles, we create a preference dataset,
PsychoCounsel-Preference, which contains 36k high-quality preference comparison
pairs. This dataset aligns with the preferences of professional
psychotherapists, providing a robust foundation for evaluating and improving
LLMs in psycho-counseling. Experiments on reward modeling and preference
learning demonstrate that PsychoCounsel-Preference is an excellent resource for
LLMs to acquire essential skills for responding to clients in a counseling
session. Our best-aligned model, PsychoCounsel-Llama3-8B, achieves an
impressive win rate of 87% against GPT-4o. We release PsychoCounsel-Preference,
PsychoCounsel-Llama3-8B and the reward model PsychoCounsel Llama3-8B-Reward to
facilitate the research of psycho-counseling with LLMs at:
https://hf.co/Psychotherapy-LLM.
67c36b35e12b50f698e7db1d
Mian Zhang
claimed_verified
2025-03-04T08:51:31.238Z
650857fef3060ea840ffbbfe
/avatars/f9c87a75c9e5cf829b7d60102a3b7096.svg
Mian Zhang
user
billmianz
67c36b35e12b50f698e7db1e
Shaun M. Eack
67c36b35e12b50f698e7db1f
Zhiyu Zoey Chen
Preference Learning Unlocks LLMs' Psycho-Counseling Skills
Applying large language models (LLMs) to assist in psycho-counseling is an
emerging and meaningful approach, driven by the significant gap between patient
needs and the availability of mental health support. However, current LLMs
struggle to consistently provide effective responses to client speeches,
largely due to the lack of supervision from high-quality real psycho-counseling
data, whose content is typically inaccessible due to client privacy concerns.
Furthermore, the quality of therapists' responses in available sessions can
vary significantly based on their professional training and experience.
Assessing the quality of therapists' responses remains an open challenge. In
this work, we address these challenges by first proposing a set of professional
and comprehensive principles to evaluate therapists' responses to client
speeches. Using these principles, we create a preference dataset,
PsychoCounsel-Preference, which contains 36k high-quality preference comparison
pairs. This dataset aligns with the preferences of professional
psychotherapists, providing a robust foundation for evaluating and improving
LLMs in psycho-counseling. Experiments on reward modeling and preference
learning demonstrate that PsychoCounsel-Preference is an excellent resource for
LLMs to acquire essential skills for responding to clients in a counseling
session. Our best-aligned model, PsychoCounsel-Llama3-8B, achieves an
impressive win rate of 87% against GPT-4o. We release PsychoCounsel-Preference,
PsychoCounsel-Llama3-8B and the reward model PsychoCounsel Llama3-8B-Reward to
facilitate the research of psycho-counseling with LLMs at:
https://hf.co/Psychotherapy-LLM.
A preference dataset, PsychoCounsel-Preference, is created to evaluate and improve large language models for psycho-counseling, demonstrating significant advancements over existing models.
large language models psycho-counseling preference dataset preference comparison pairs professional psychotherapists reward modeling preference learning PsychoCounsel-Llama3-8B PsychoCounsel Llama3-8B-Reward