Building a Foundational Guardrail for General Agentic Systems via
  Synthetic Data
While LLM agents can plan multi-step tasks, intervening at the planning
stage-before any action is executed-is often the safest way to prevent harm,
since certain risks can lead to severe consequences once carried out. However,
existing guardrails mostly operate post-execution, which is difficult to scale
and leaves little room for controllable supervision at the plan level. To
address this challenge, we highlight three critical gaps in current research:
data gap, model gap, and evaluation gap. To close the data gap, we introduce
AuraGen, a controllable engine that (i) synthesizes benign trajectories, (ii)
injects category-labeled risks with calibrated difficulty, and (iii) filters
outputs via an automated reward model, producing large and reliable corpora for
pre-execution safety. To close the guardian model gap, we propose a
foundational guardrail Safiron, combining a cross-planner adapter with a
compact guardian model. The adapter unifies different input formats, while
Safiron flags risky cases, assigns risk types, and generates rationales;
trained in two stages with a broadly explored data recipe, Safiron achieves
robust transfer across settings. To close the evaluation gap, we release
Pre-Exec Bench, a realistic benchmark covering diverse tools and branching
trajectories, which measures detection, fine-grained categorization,
explanation, and cross-planner generalization in human-verified scenarios.
Extensive experiments demonstrate consistent gains of the proposed guardrail
over strong baselines on Pre-Exec Bench, and ablations further distill
actionable practices, providing a practical template for safer agentic systems.
68edcbffde1fee572713a927
Yue Huang
68edcbffde1fee572713a928
Hang Hua
claimed_verified
2025-10-14T07:29:51.627Z
639f8277beb95d698de007dd
/avatars/57f223ccd9d3cb03166ccf0e41361c58.svg
HangHua
user
hhua2
68edcbffde1fee572713a929
Yujun Zhou
68edcbffde1fee572713a92a
Pengcheng Jing
68edcbffde1fee572713a92b
Manish Nagireddy
68edcbffde1fee572713a92c
Inkit Padhi
68edcbffde1fee572713a92d
Greta Dolcetti
68edcbffde1fee572713a92e
Zhangchen Xu
68edcbffde1fee572713a92f
Subhajit Chaudhury
68edcbffde1fee572713a930
Ambrish Rawat
68edcbffde1fee572713a931
Liubov Nedoshivina
68edcbffde1fee572713a932
Pin-Yu Chen
68edcbffde1fee572713a933
Prasanna Sattigeri
68edcbffde1fee572713a934
Xiangliang Zhang
Building a Foundational Guardrail for General Agentic Systems via
  Synthetic Data
While LLM agents can plan multi-step tasks, intervening at the planning
stage-before any action is executed-is often the safest way to prevent harm,
since certain risks can lead to severe consequences once carried out. However,
existing guardrails mostly operate post-execution, which is difficult to scale
and leaves little room for controllable supervision at the plan level. To
address this challenge, we highlight three critical gaps in current research:
data gap, model gap, and evaluation gap. To close the data gap, we introduce
AuraGen, a controllable engine that (i) synthesizes benign trajectories, (ii)
injects category-labeled risks with calibrated difficulty, and (iii) filters
outputs via an automated reward model, producing large and reliable corpora for
pre-execution safety. To close the guardian model gap, we propose a
foundational guardrail Safiron, combining a cross-planner adapter with a
compact guardian model. The adapter unifies different input formats, while
Safiron flags risky cases, assigns risk types, and generates rationales;
trained in two stages with a broadly explored data recipe, Safiron achieves
robust transfer across settings. To close the evaluation gap, we release
Pre-Exec Bench, a realistic benchmark covering diverse tools and branching
trajectories, which measures detection, fine-grained categorization,
explanation, and cross-planner generalization in human-verified scenarios.
Extensive experiments demonstrate consistent gains of the proposed guardrail
over strong baselines on Pre-Exec Bench, and ablations further distill
actionable practices, providing a practical template for safer agentic systems.
AuraGen and Safiron address pre-execution safety gaps in LLM agents by synthesizing benign trajectories, injecting risks, and using a cross-planner adapter for robust risk detection and explanation.
AuraGen Safiron cross-planner adapter compact guardian model automated reward model Pre-Exec Bench risk detection risk categorization explanation cross-planner generalization