How Can Input Reformulation Improve Tool Usage Accuracy in a Complex
  Dynamic Environment? A Study on τ-bench
Recent advances in reasoning and planning capabilities of large language
models (LLMs) have enabled their potential as autonomous agents capable of tool
use in dynamic environments. However, in multi-turn conversational environments
like tau-bench, these agents often struggle with consistent reasoning,
adherence to domain-specific policies, and extracting correct information over
a long horizon of tool-calls and conversation. To capture and mitigate these
failures, we conduct a comprehensive manual analysis of the common errors
occurring in the conversation trajectories. We then experiment with
reformulations of inputs to the tool-calling agent for improvement in agent
decision making. Finally, we propose the Input-Reformulation Multi-Agent (IRMA)
framework, which automatically reformulates user queries augmented with
relevant domain rules and tool suggestions for the tool-calling agent to focus
on. The results show that IRMA significantly outperforms ReAct, Function
Calling, and Self-Reflection by 16.1%, 12.7%, and 19.1%, respectively, in
overall pass^5 scores. These findings highlight the superior reliability and
consistency of IRMA compared to other methods in dynamic environments.
68b6779026d3ff3f69d5fd0c
Venkatesh Mishra
68b6779026d3ff3f69d5fd0d
Amir Saeidi
claimed_verified
2025-09-03T08:29:22.149Z
640f6299ef5c6dcac8b1df52
/avatars/022f21183abc8a8b5ce1b198d3ba96dc.svg
Amir
user
sahsaeedi
68b6779026d3ff3f69d5fd0e
Satyam Raj
68b6779026d3ff3f69d5fd0f
Mutsumi Nakamura
68b6779026d3ff3f69d5fd10
Jayanth Srinivasa
68b6779026d3ff3f69d5fd11
Gaowen Liu
68b6779026d3ff3f69d5fd12
Ali Payani
68b6779026d3ff3f69d5fd13
Chitta Baral
How Can Input Reformulation Improve Tool Usage Accuracy in a Complex
  Dynamic Environment? A Study on τ-bench
Recent advances in reasoning and planning capabilities of large language
models (LLMs) have enabled their potential as autonomous agents capable of tool
use in dynamic environments. However, in multi-turn conversational environments
like tau-bench, these agents often struggle with consistent reasoning,
adherence to domain-specific policies, and extracting correct information over
a long horizon of tool-calls and conversation. To capture and mitigate these
failures, we conduct a comprehensive manual analysis of the common errors
occurring in the conversation trajectories. We then experiment with
reformulations of inputs to the tool-calling agent for improvement in agent
decision making. Finally, we propose the Input-Reformulation Multi-Agent (IRMA)
framework, which automatically reformulates user queries augmented with
relevant domain rules and tool suggestions for the tool-calling agent to focus
on. The results show that IRMA significantly outperforms ReAct, Function
Calling, and Self-Reflection by 16.1%, 12.7%, and 19.1%, respectively, in
overall pass^5 scores. These findings highlight the superior reliability and
consistency of IRMA compared to other methods in dynamic environments.
The IRMA framework improves the reliability and consistency of large language models in dynamic environments by reformulating user queries with domain rules and tool suggestions.
large language models LLMs autonomous agents tool use multi-turn conversational environments $\tau$-bench consistent reasoning domain-specific policies conversation trajectories tool-calling agent decision making Input-Reformulation Multi-Agent IRMA ReAct Function Calling Self-Reflection pass^5 scores