Octo-planner: On-device Language Model for Planner-Action Agents
AI agents have become increasingly significant in various domains, enabling
autonomous decision-making and problem-solving. To function effectively, these
agents require a planning process that determines the best course of action and
then executes the planned actions. In this paper, we present an efficient
on-device Planner-Action framework that separates planning and action execution
into two distinct components: a planner agent based on Phi-3 Mini, a 3.8
billion parameter LLM optimized for edge devices, and an action agent using the
Octopus model for function execution. The planner agent first responds to user
queries by decomposing tasks into a sequence of sub-steps, which are then
executed by the action agent. To optimize performance on resource-constrained
devices, we employ model fine-tuning instead of in-context learning, reducing
computational costs and energy consumption while improving response times. Our
approach involves using GPT-4 to generate diverse planning queries and
responses based on available functions, with subsequent validations to ensure
data quality. We fine-tune the Phi-3 Mini model on this curated dataset,
achieving a 97\% success rate in our in-domain test environment. To address
multi-domain planning challenges, we developed a multi-LoRA training method
that merges weights from LoRAs trained on distinct function subsets. This
approach enables flexible handling of complex, multi-domain queries while
maintaining computational efficiency on resource-constrained devices. To
support further research, we have open-sourced our model weights at
https://huggingface.co/NexaAIDev/octopus-planning. For the demo, please
refer to https://www.nexa4ai.com/octo-planner.
667cbb5102ffd4993e2611f0
Wei Chen
claimed_verified
2024-06-28T08:03:10.308Z
65c082c626c639b0009a3c82
https://cdn-avatars.huggingface.co/v1/production/uploads/65c082c626c639b0009a3c82/w_Q_5iLOM-nh_drd9Tu4X.jpeg
Alex Chen
user
alexchen4ai
667cbb5102ffd4993e2611f1
Zhiyuan Li
claimed_verified
2024-06-27T07:31:33.153Z
644de55076c0ab1880b9cb4d
https://cdn-avatars.huggingface.co/v1/production/uploads/644de55076c0ab1880b9cb4d/zKm7baOIzt_a4py8mYkck.jpeg
Zack Li
user
zackli4ai
667cbb5102ffd4993e2611f2
Zhen Guo
claimed_verified
2024-06-27T07:31:30.780Z
638e4e66629b4d0a62ce1bf3
https://cdn-avatars.huggingface.co/v1/production/uploads/638e4e66629b4d0a62ce1bf3/s7uQ2qmee2CaXfjZDOovZ.jpeg
Zhen Guo
user
zguo0525
667cbb5102ffd4993e2611f3
Yikang Shen
admin_assigned
2024-06-27T19:08:17.201Z
6463995939359568c63d740c
/avatars/11caacefc68c1259dbc8764cc4340481.svg
Yikang Shen
user
YikangS
Octo-planner: On-device Language Model for Planner-Action Agents
AI agents have become increasingly significant in various domains, enabling
autonomous decision-making and problem-solving. To function effectively, these
agents require a planning process that determines the best course of action and
then executes the planned actions. In this paper, we present an efficient
on-device Planner-Action framework that separates planning and action execution
into two distinct components: a planner agent based on Phi-3 Mini, a 3.8
billion parameter LLM optimized for edge devices, and an action agent using the
Octopus model for function execution. The planner agent first responds to user
queries by decomposing tasks into a sequence of sub-steps, which are then
executed by the action agent. To optimize performance on resource-constrained
devices, we employ model fine-tuning instead of in-context learning, reducing
computational costs and energy consumption while improving response times. Our
approach involves using GPT-4 to generate diverse planning queries and
responses based on available functions, with subsequent validations to ensure
data quality. We fine-tune the Phi-3 Mini model on this curated dataset,
achieving a 97\% success rate in our in-domain test environment. To address
multi-domain planning challenges, we developed a multi-LoRA training method
that merges weights from LoRAs trained on distinct function subsets. This
approach enables flexible handling of complex, multi-domain queries while
maintaining computational efficiency on resource-constrained devices. To
support further research, we have open-sourced our model weights at
https://huggingface.co/NexaAIDev/octopus-planning. For the demo, please
refer to https://www.nexa4ai.com/octo-planner.
An on-device Planner-Action framework, utilizing Phi-3 Mini for planning and Octopus for execution, employs model fine-tuning and multi-LoRA training to enhance efficiency and multi-domain handling on resource-constrained devices.
LLM Phi-3 Mini Octopus model model fine-tuning GPT-4 in-context learning LoRA multi-LoRA training on-device framework computational efficiency