Octopus v4: Graph of language models
Language models have been effective in a wide range of applications, yet the
most sophisticated models are often proprietary. For example, GPT-4 by OpenAI
and various models by Anthropic are expensive and consume substantial energy.
In contrast, the open-source community has produced competitive models, like
Llama3. Furthermore, niche-specific smaller language models, such as those
tailored for legal, medical or financial tasks, have outperformed their
proprietary counterparts. This paper introduces a novel approach that employs
functional tokens to integrate multiple open-source models,
each optimized for particular tasks. Our newly developed Octopus v4 model
leverages functional tokens to intelligently direct user queries to
the most appropriate vertical model and reformat the query to achieve the best
performance. Octopus v4, an evolution of the Octopus v1, v2, and v3 models,
excels in selection and parameter understanding and reformatting. Additionally,
we explore the use of graph as a versatile data structure that effectively
coordinates multiple open-source models by harnessing the capabilities of the
Octopus model and functional tokens. Use our open-sourced GitHub
(https://www.nexa4ai.com/) to try Octopus v4 models
(https://huggingface.co/NexaAIDev/Octopus-v4), and contrite to a larger
graph of language models. By activating models less than 10B parameters, we
achieved SOTA MMLU score of 74.8 among the same level models.
663197a6a2354b0f50ae1605
Wei Chen
admin_assigned
2024-05-02T07:51:10.068Z
65c082c626c639b0009a3c82
https://cdn-avatars.huggingface.co/v1/production/uploads/65c082c626c639b0009a3c82/w_Q_5iLOM-nh_drd9Tu4X.jpeg
Alex Chen
user
alexchen4ai
663197a6a2354b0f50ae1606
Zhiyuan Li
claimed_verified
2024-05-02T08:01:51.908Z
644de55076c0ab1880b9cb4d
https://cdn-avatars.huggingface.co/v1/production/uploads/644de55076c0ab1880b9cb4d/zKm7baOIzt_a4py8mYkck.jpeg
Zack Li
user
zackli4ai
Octopus v4: Graph of language models
Language models have been effective in a wide range of applications, yet the
most sophisticated models are often proprietary. For example, GPT-4 by OpenAI
and various models by Anthropic are expensive and consume substantial energy.
In contrast, the open-source community has produced competitive models, like
Llama3. Furthermore, niche-specific smaller language models, such as those
tailored for legal, medical or financial tasks, have outperformed their
proprietary counterparts. This paper introduces a novel approach that employs
functional tokens to integrate multiple open-source models,
each optimized for particular tasks. Our newly developed Octopus v4 model
leverages functional tokens to intelligently direct user queries to
the most appropriate vertical model and reformat the query to achieve the best
performance. Octopus v4, an evolution of the Octopus v1, v2, and v3 models,
excels in selection and parameter understanding and reformatting. Additionally,
we explore the use of graph as a versatile data structure that effectively
coordinates multiple open-source models by harnessing the capabilities of the
Octopus model and functional tokens. Use our open-sourced GitHub
(https://www.nexa4ai.com/) to try Octopus v4 models
(https://huggingface.co/NexaAIDev/Octopus-v4), and contrite to a larger
graph of language models. By activating models less than 10B parameters, we
achieved SOTA MMLU score of 74.8 among the same level models.
The Octopus v4 model uses functional tokens to integrate and direct queries to task-specific open-source language models, achieving SOTA performance with models under 10B parameters.
functional tokens Octopus v4 Llama3 parameter-efficient fine-tuning graph data structure MMLU score