Inverse IFEval: Can LLMs Unlearn Stubborn Training Conventions to Follow
  Real Instructions?
Large Language Models (LLMs) achieve strong performance on diverse tasks but
often exhibit cognitive inertia, struggling to follow instructions that
conflict with the standardized patterns learned during supervised fine-tuning
(SFT). To evaluate this limitation, we propose Inverse IFEval, a benchmark that
measures models Counter-intuitive Abilitytheir capacity to override
training-induced biases and comply with adversarial instructions. Inverse
IFEval introduces eight types of such challenges, including Question
Correction, Intentional Textual Flaws, Code without Comments, and
Counterfactual Answering. Using a human-in-the-loop pipeline, we construct a
dataset of 1012 high-quality Chinese and English questions across 23 domains,
evaluated under an optimized LLM-as-a-Judge framework. Experiments on existing
leading LLMs demonstrate the necessity of our proposed Inverse IFEval
benchmark. Our findings emphasize that future alignment efforts should not only
pursue fluency and factual correctness but also account for adaptability under
unconventional contexts. We hope that Inverse IFEval serves as both a
diagnostic tool and a foundation for developing methods that mitigate cognitive
inertia, reduce overfitting to narrow patterns, and ultimately enhance the
instruction-following reliability of LLMs in diverse and unpredictable
real-world scenarios.
68ba6843736018af705e8eb4
Qinyan Zhang
admin_assigned
2025-09-05T14:20:01.213Z
68ba81e3d389ea993bf35893
/avatars/67a790f8cd8423de63196adda1ec12a3.svg
Qinyan Zhang
user
zqyzqyzqy
68ba6843736018af705e8eb5
Xinping Lei
admin_assigned
2025-09-05T14:20:17.772Z
646319712538819c72a1b699
/avatars/19ace41095877f5a485d9100b393ca13.svg
lei xin ping
user
leixp
68ba6843736018af705e8eb6
Ruijie Miao
admin_assigned
2025-09-05T14:20:24.037Z
66cd5ebadbfa8dff219acb70
/avatars/fd4225dca0d4c3ba7d86304bd1d826b3.svg
Ruijie Miao
user
Baton6257
68ba6843736018af705e8eb7
Yu Fu
68ba6843736018af705e8eb8
Haojie Fan
68ba6843736018af705e8eb9
Le Chang
claimed_verified
2025-09-09T13:49:50.053Z
68b7ccb7637461a7e5882271
/avatars/aaecd06634fca5875efe36c5ea0bf6d0.svg
chanleg
user
chanleg
68ba6843736018af705e8eba
Jiafan Hou
68ba6843736018af705e8ebb
Dingling Zhang
68ba6843736018af705e8ebc
Zhongfei Hou
68ba6843736018af705e8ebd
Ziqiang Yang
admin_assigned
2025-09-05T14:20:51.949Z
68a00f7b5140d6d8305ea065
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/ShETxmglZOsaHN7q9Gzfv.png
杨子强
user
ziqiangyang
68ba6843736018af705e8ebe
Changxin Pu
68ba6843736018af705e8ebf
Fei Hu
68ba6843736018af705e8ec0
Jingkai Liu
68ba6843736018af705e8ec1
Mengyun Liu
admin_assigned
2025-09-05T14:21:17.429Z
665929223498a9ef525d7b1c
/avatars/445f5298228d54ed5cfc27b6a92a339e.svg
Mengyun Liu
user
liumengyun
68ba6843736018af705e8ec2
Yang Liu
68ba6843736018af705e8ec3
Xiang Gao
claimed_verified
2025-09-22T02:37:14.696Z
671068b8243baf4c58fc29c2
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/leMkq-kw1icPo2bCLivGv.png
Xiang Gao
user
coffiney
68ba6843736018af705e8ec4
Jiaheng Liu
admin_assigned
2025-09-05T14:21:09.876Z
65377c30e48353201e6fdda0
/avatars/a8f803b6f2e598eaee9c52c0d2ddfc16.svg
Jiaheng Liu
user
CheeryLJH
68ba6843736018af705e8ec5
Tong Yang
68ba6843736018af705e8ec6
Zaiyuan Wang
68ba6843736018af705e8ec7
Ge Zhang
claimed_verified
2025-09-05T08:01:32.173Z
638efcf4c67af472d316d424
/avatars/97a57859d7d87a3a8f1bb41d32a72bc2.svg
Ge Zhang
user
zhangysk
68ba6843736018af705e8ec8
Wenhao Huang
admin_assigned
2025-09-05T14:19:54.443Z
686c983f8c812deb40e83270
/avatars/7d1e4a38929b7ac79004463fd0060e9b.svg
Wenhao Huang
user
WenhaoHuang
Inverse IFEval: Can LLMs Unlearn Stubborn Training Conventions to Follow
  Real Instructions?
Large Language Models (LLMs) achieve strong performance on diverse tasks but
often exhibit cognitive inertia, struggling to follow instructions that
conflict with the standardized patterns learned during supervised fine-tuning
(SFT). To evaluate this limitation, we propose Inverse IFEval, a benchmark that
measures models Counter-intuitive Abilitytheir capacity to override
training-induced biases and comply with adversarial instructions. Inverse
IFEval introduces eight types of such challenges, including Question
Correction, Intentional Textual Flaws, Code without Comments, and
Counterfactual Answering. Using a human-in-the-loop pipeline, we construct a
dataset of 1012 high-quality Chinese and English questions across 23 domains,
evaluated under an optimized LLM-as-a-Judge framework. Experiments on existing
leading LLMs demonstrate the necessity of our proposed Inverse IFEval
benchmark. Our findings emphasize that future alignment efforts should not only
pursue fluency and factual correctness but also account for adaptability under
unconventional contexts. We hope that Inverse IFEval serves as both a
diagnostic tool and a foundation for developing methods that mitigate cognitive
inertia, reduce overfitting to narrow patterns, and ultimately enhance the
instruction-following reliability of LLMs in diverse and unpredictable
real-world scenarios.
Inverse IFEval evaluates Large Language Models' ability to override training biases and follow unconventional instructions, highlighting the need for adaptability in diverse contexts.
Large Language Models cognitive inertia supervised fine-tuning Inverse IFEval Counter-intuitive Ability Question Correction Intentional Textual Flaws Code without Comments Counterfactual Answering LLM-as-a-Judge