IHEval: Evaluating Language Models on Following the Instruction
  Hierarchy
The instruction hierarchy, which establishes a priority order from system
messages to user messages, conversation history, and tool outputs, is essential
for ensuring consistent and safe behavior in language models (LMs). Despite its
importance, this topic receives limited attention, and there is a lack of
comprehensive benchmarks for evaluating models' ability to follow the
instruction hierarchy. We bridge this gap by introducing IHEval, a novel
benchmark comprising 3,538 examples across nine tasks, covering cases where
instructions in different priorities either align or conflict. Our evaluation
of popular LMs highlights their struggle to recognize instruction priorities.
All evaluated models experience a sharp performance decline when facing
conflicting instructions, compared to their original instruction-following
performance. Moreover, the most competitive open-source model only achieves 48%
accuracy in resolving such conflicts. Our results underscore the need for
targeted optimization in the future development of LMs.
67b4cf1994ec5e365fb7995d
Zhihan Zhang
extracted_confirmed
2025-02-18T18:19:31.455Z
63bf9695da08ed054400205e
/avatars/5223b212431337c6186db96ac70b7ae7.svg
Zhihan Zhang
user
zhihz0535
67b4cf1994ec5e365fb7995e
Shiyang Li
67b4cf1994ec5e365fb7995f
Zixuan Zhang
67b4cf1994ec5e365fb79960
Xin Liu
67b4cf1994ec5e365fb79961
Haoming Jiang
67b4cf1994ec5e365fb79962
Xianfeng Tang
admin_assigned
2025-02-19T15:25:56.312Z
6465f6467ff8fcbef7d22513
/avatars/07992835c235fbb07016a0ea4f1d61cb.svg
Xianfeng Tang
user
xianft
67b4cf1994ec5e365fb79963
Yifan Gao
67b4cf1994ec5e365fb79964
Zheng Li
67b4cf1994ec5e365fb79965
Haodong Wang
67b4cf1994ec5e365fb79966
Zhaoxuan Tan
admin_assigned
2025-02-19T15:26:12.966Z
638eaca4a26f3d5af37be8b3
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/FjnAVJ0sP_H74QEj8PR_k.png
Zhaoxuan_Tan
user
Zhaoxuan
67b4cf1994ec5e365fb79967
Yichuan Li
67b4cf1994ec5e365fb79968
Qingyu Yin
admin_assigned
2025-02-19T15:26:54.938Z
6453cb22908e259483c0a061
https://cdn-avatars.huggingface.co/v1/production/uploads/6453cb22908e259483c0a061/hMgdwZUsUbgquGalzPGzV.jpeg
Qingyu_Yin
user
MikaStars39
67b4cf1994ec5e365fb79969
Bing Yin
67b4cf1994ec5e365fb7996a
Meng Jiang
IHEval: Evaluating Language Models on Following the Instruction
  Hierarchy
The instruction hierarchy, which establishes a priority order from system
messages to user messages, conversation history, and tool outputs, is essential
for ensuring consistent and safe behavior in language models (LMs). Despite its
importance, this topic receives limited attention, and there is a lack of
comprehensive benchmarks for evaluating models' ability to follow the
instruction hierarchy. We bridge this gap by introducing IHEval, a novel
benchmark comprising 3,538 examples across nine tasks, covering cases where
instructions in different priorities either align or conflict. Our evaluation
of popular LMs highlights their struggle to recognize instruction priorities.
All evaluated models experience a sharp performance decline when facing
conflicting instructions, compared to their original instruction-following
performance. Moreover, the most competitive open-source model only achieves 48%
accuracy in resolving such conflicts. Our results underscore the need for
targeted optimization in the future development of LMs.
A new benchmark, IHEval, assesses language models' adherence to instruction hierarchy, revealing their significant struggles with conflicting instructions.
instruction hierarchy language models LMs IHEval benchmark instruction-following conflicting instructions accuracy