Husky: A Unified, Open-Source Language Agent for Multi-Step Reasoning
Language agents perform complex tasks by using tools to execute each step
precisely. However, most existing agents are based on proprietary models or
designed to target specific tasks, such as mathematics or multi-hop question
answering. We introduce Husky, a holistic, open-source language agent that
learns to reason over a unified action space to address a diverse set of
complex tasks involving numerical, tabular, and knowledge-based reasoning.
Husky iterates between two stages: 1) generating the next action to take
towards solving a given task and 2) executing the action using expert models
and updating the current solution state. We identify a thorough ontology of
actions for addressing complex tasks and curate high-quality data to train
expert models for executing these actions. Our experiments show that Husky
outperforms prior language agents across 14 evaluation datasets. Moreover, we
introduce HuskyQA, a new evaluation set which stress tests language agents for
mixed-tool reasoning, with a focus on retrieving missing knowledge and
performing numerical reasoning. Despite using 7B models, Husky matches or even
exceeds frontier LMs such as GPT-4 on these tasks, showcasing the efficacy of
our holistic approach in addressing complex reasoning problems. Our code and
models are available at https://github.com/agent-husky/Husky-v1.
6667cfedcdac3c8f11acf0a0
Joongwon Kim
admin_assigned
2024-06-11T07:53:14.744Z
64b81bc20d8834cb3225bbc0
https://cdn-avatars.huggingface.co/v1/production/uploads/64b81bc20d8834cb3225bbc0/F2ciNosMyIlC2ZLlVTpw2.jpeg
Joongwon Kim
user
jwonkim
6667cfedcdac3c8f11acf0a1
Bhargavi Paranjape
admin_assigned
2024-06-11T07:55:49.678Z
63efd0944a788ed1dd84fd57
/avatars/db1e0fee54151fcf795b4826d625a332.svg
Bhargavi Paranjape
user
bhargaviparanjape
6667cfedcdac3c8f11acf0a2
Tushar Khot
admin_assigned
2024-06-11T07:55:55.579Z
62cdf2dbaac2c91c95581830
https://cdn-avatars.huggingface.co/v1/production/uploads/62cdf2dbaac2c91c95581830/N3NuO7j53qkauJOf3kkRS.jpeg
Tushar Khot
user
tusharkhot
6667cfedcdac3c8f11acf0a3
Hannaneh Hajishirzi
Husky: A Unified, Open-Source Language Agent for Multi-Step Reasoning
Language agents perform complex tasks by using tools to execute each step
precisely. However, most existing agents are based on proprietary models or
designed to target specific tasks, such as mathematics or multi-hop question
answering. We introduce Husky, a holistic, open-source language agent that
learns to reason over a unified action space to address a diverse set of
complex tasks involving numerical, tabular, and knowledge-based reasoning.
Husky iterates between two stages: 1) generating the next action to take
towards solving a given task and 2) executing the action using expert models
and updating the current solution state. We identify a thorough ontology of
actions for addressing complex tasks and curate high-quality data to train
expert models for executing these actions. Our experiments show that Husky
outperforms prior language agents across 14 evaluation datasets. Moreover, we
introduce HuskyQA, a new evaluation set which stress tests language agents for
mixed-tool reasoning, with a focus on retrieving missing knowledge and
performing numerical reasoning. Despite using 7B models, Husky matches or even
exceeds frontier LMs such as GPT-4 on these tasks, showcasing the efficacy of
our holistic approach in addressing complex reasoning problems. Our code and
models are available at https://github.com/agent-husky/Husky-v1.
Husky, an open-source language agent, excels in complex reasoning tasks across multiple domains by iteratively generating and executing actions, outperforming existing models and matching GPT-4 on a comprehensive evaluation set.
language agents action space numerical reasoning tabular reasoning knowledge-based reasoning expert models ontology of actions high-quality data HuskyQA evaluation datasets holistic approach