Reasoning over Boundaries: Enhancing Specification Alignment via
  Test-time Delibration
Large language models (LLMs) are increasingly applied in diverse real-world
scenarios, each governed by bespoke behavioral and safety specifications (spec)
custom-tailored by users or organizations. These spec, categorized into
safety-spec and behavioral-spec, vary across scenarios and evolve with changing
preferences and requirements. We formalize this challenge as specification
alignment, focusing on LLMs' ability to follow dynamic, scenario-specific spec
from both behavioral and safety perspectives. To address this challenge, we
propose Align3, a lightweight method that employs Test-Time Deliberation (TTD)
with hierarchical reflection and revision to reason over the specification
boundaries. We further present SpecBench, a unified benchmark for measuring
specification alignment, covering 5 scenarios, 103 spec, and 1,500 prompts.
Experiments on 15 reasoning and 18 instruct models with several TTD methods,
including Self-Refine, TPO, and MoreThink, yield three key findings: (i)
test-time deliberation enhances specification alignment; (ii) Align3 advances
the safety-helpfulness trade-off frontier with minimal overhead; (iii)
SpecBench effectively reveals alignment gaps. These results highlight the
potential of test-time deliberation as an effective strategy for reasoning over
the real-world specification boundaries.
68ccc6a13df9ac65e93dc66a
Haoran Zhang
claimed_verified
2025-09-19T06:48:23.170Z
689ec537196ab997b13dc977
/avatars/28e0ba07ca26056833e1197a6aa0d14f.svg
Haoran Zhang
user
zzzhr97
68ccc6a13df9ac65e93dc66b
Yafu Li
admin_assigned
2025-09-19T13:12:52.499Z
63f3502a520c14618925825a
/avatars/e986a2a6625e7be6890616a417f908d2.svg
Yafu Li
user
yaful
68ccc6a13df9ac65e93dc66c
Xuyang Hu
admin_assigned
2025-09-19T13:13:07.959Z
6498fde776d49ee00f79cbfe
/avatars/4c284a71080150e6cb3b9632dfccef60.svg
Xuyang Hu
user
huxy912
68ccc6a13df9ac65e93dc66d
Dongrui Liu
admin_assigned
2025-09-19T13:13:15.250Z
657fe7a8504da7f6f30a2832
/avatars/65987e3cba449b5d250616510ee11f33.svg
Dongrui Liu
user
Max9803
68ccc6a13df9ac65e93dc66e
Zhilin Wang
68ccc6a13df9ac65e93dc66f
Bo Li
68ccc6a13df9ac65e93dc670
Yu Cheng
Reasoning over Boundaries: Enhancing Specification Alignment via
  Test-time Delibration
Large language models (LLMs) are increasingly applied in diverse real-world
scenarios, each governed by bespoke behavioral and safety specifications (spec)
custom-tailored by users or organizations. These spec, categorized into
safety-spec and behavioral-spec, vary across scenarios and evolve with changing
preferences and requirements. We formalize this challenge as specification
alignment, focusing on LLMs' ability to follow dynamic, scenario-specific spec
from both behavioral and safety perspectives. To address this challenge, we
propose Align3, a lightweight method that employs Test-Time Deliberation (TTD)
with hierarchical reflection and revision to reason over the specification
boundaries. We further present SpecBench, a unified benchmark for measuring
specification alignment, covering 5 scenarios, 103 spec, and 1,500 prompts.
Experiments on 15 reasoning and 18 instruct models with several TTD methods,
including Self-Refine, TPO, and MoreThink, yield three key findings: (i)
test-time deliberation enhances specification alignment; (ii) Align3 advances
the safety-helpfulness trade-off frontier with minimal overhead; (iii)
SpecBench effectively reveals alignment gaps. These results highlight the
potential of test-time deliberation as an effective strategy for reasoning over
the real-world specification boundaries.
Align3, a lightweight method using Test-Time Deliberation, enhances specification alignment in large language models across diverse scenarios with minimal overhead.
Large language models specification alignment Test-Time Deliberation hierarchical reflection revision SpecBench safety-helpfulness trade-off Self-Refine TPO MoreThink