RLVER: Reinforcement Learning with Verifiable Emotion Rewards for
  Empathetic Agents
Large language models (LLMs) excel at logical and algorithmic reasoning, yet
their emotional intelligence (EQ) still lags far behind their cognitive
prowess. While reinforcement learning from verifiable rewards (RLVR) has
advanced in other domains, its application to dialogue-especially for emotional
intelligence-remains underexplored. In this work, we introduce RLVER, the first
end-to-end reinforcement learning framework that leverages verifiable emotion
rewards from simulated users to cultivate higher-order empathetic abilities in
LLMs. Within this framework, self-consistent affective simulated users engage
in dialogue rollouts and produce deterministic emotion scores during
conversations, serving as reward signals to guide the LLM's learning.
Fine-tuning publicly available Qwen2.5-7B-Instruct model with PPO boosts its
Sentient-Benchmark score from 13.3 to 79.2 while largely preserving
mathematical and coding competence. Extensive experiments reveal that: (i)
RLVER consistently improves multiple dialogue capabilities; (ii) Thinking and
non-thinking models show distinct trends--thinking models excel in empathy and
insight, while non-thinking models favor action; (iii) GRPO often yields stable
gains, while PPO can push certain capabilities to a higher ceiling; (iv) More
challenging environments are not always better-moderate ones can yield stronger
outcomes. Our results show that RLVER is a practical route toward emotionally
intelligent and broadly capable language agents.
686c8b1c364e2ad167eb53b4
Peisong Wang
claimed_verified
2025-07-09T08:51:33.965Z
64be4408c05a0df0d2b6012e
/avatars/09d8427505a418090391dc5a3f8bfef2.svg
PSWang
user
CedarWang
686c8b1c364e2ad167eb53b5
Ruotian Ma
claimed_verified
2025-07-09T08:51:32.096Z
648294b2eb4befee378951c1
/avatars/da5d8bf9d8662cc2ffa2c0de49bd66a3.svg
Ruotian Ma
user
vvibt
686c8b1c364e2ad167eb53b6
Bang Zhang
686c8b1c364e2ad167eb53b7
Xingyu Chen
686c8b1c364e2ad167eb53b8
Zhiwei He
686c8b1c364e2ad167eb53b9
Kang Luo
686c8b1c364e2ad167eb53ba
Qingsong Lv
686c8b1c364e2ad167eb53bb
Qingxuan Jiang
686c8b1c364e2ad167eb53bc
Zheng Xie
686c8b1c364e2ad167eb53bd
Shanyi Wang
686c8b1c364e2ad167eb53be
Yuan Li
686c8b1c364e2ad167eb53bf
Fanghua Ye
686c8b1c364e2ad167eb53c0
Jian Li
686c8b1c364e2ad167eb53c1
Yifan Yang
686c8b1c364e2ad167eb53c2
Zhaopeng Tu
686c8b1c364e2ad167eb53c3
Xiaolong Li
RLVER: Reinforcement Learning with Verifiable Emotion Rewards for
  Empathetic Agents
Large language models (LLMs) excel at logical and algorithmic reasoning, yet
their emotional intelligence (EQ) still lags far behind their cognitive
prowess. While reinforcement learning from verifiable rewards (RLVR) has
advanced in other domains, its application to dialogue-especially for emotional
intelligence-remains underexplored. In this work, we introduce RLVER, the first
end-to-end reinforcement learning framework that leverages verifiable emotion
rewards from simulated users to cultivate higher-order empathetic abilities in
LLMs. Within this framework, self-consistent affective simulated users engage
in dialogue rollouts and produce deterministic emotion scores during
conversations, serving as reward signals to guide the LLM's learning.
Fine-tuning publicly available Qwen2.5-7B-Instruct model with PPO boosts its
Sentient-Benchmark score from 13.3 to 79.2 while largely preserving
mathematical and coding competence. Extensive experiments reveal that: (i)
RLVER consistently improves multiple dialogue capabilities; (ii) Thinking and
non-thinking models show distinct trends--thinking models excel in empathy and
insight, while non-thinking models favor action; (iii) GRPO often yields stable
gains, while PPO can push certain capabilities to a higher ceiling; (iv) More
challenging environments are not always better-moderate ones can yield stronger
outcomes. Our results show that RLVER is a practical route toward emotionally
intelligent and broadly capable language agents.
RLVER, an end-to-end reinforcement learning framework using verifiable emotion rewards, enhances emotional intelligence in LLMs without sacrificing cognitive abilities.
reinforcement learning from verifiable rewards RLVER self-consistent affective simulated users dialogue rollouts emotion scores reward signals fine-tuning Qwen2.5-7B-Instruct PPO Sentient-Benchmark dialogue capabilities thinking models non-thinking models GRPO emotionally intelligent language agents