Fully Open Source Moxin-7B Technical Report
Recently, Large Language Models (LLMs) have undergone a significant
transformation, marked by a rapid rise in both their popularity and
capabilities. Leading this evolution are proprietary LLMs like GPT-4 and
GPT-o1, which have captured widespread attention in the AI community due to
their remarkable performance and versatility. Simultaneously, open-source LLMs,
such as LLaMA and Mistral, have made great contributions to the ever-increasing
popularity of LLMs due to the ease to customize and deploy the models across
diverse applications. Although open-source LLMs present unprecedented
opportunities for innovation and research, the commercialization of LLMs has
raised concerns about transparency, reproducibility, and safety. Many
open-source LLMs fail to meet fundamental transparency requirements by
withholding essential components like training code and data, and some use
restrictive licenses whilst claiming to be "open-source," which may hinder
further innovations on LLMs. To mitigate this issue, we introduce Moxin 7B, a
fully open-source LLM developed in accordance with the Model Openness Framework
(MOF), a ranked classification system that evaluates AI models based on model
completeness and openness, adhering to principles of open science, open source,
open data, and open access. Our model achieves the highest MOF classification
level of "open science" through the comprehensive release of pre-training code
and configurations, training and fine-tuning datasets, and intermediate and
final checkpoints. Experiments show that our model achieves superior
performance in zero-shot evaluation compared with popular 7B models and
performs competitively in few-shot evaluation.
6758ff6ea90b1c382daf3526
Pu Zhao
6758ff6ea90b1c382daf3527
Xuan Shen
claimed_verified
2025-03-25T08:22:14.683Z
65f7197c66afd39dd7f58522
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/zx-6RlvxTATmIO3dCIYez.png
Shawn Shen
user
shawnricecake
6758ff6ea90b1c382daf3528
Zhenglun Kong
claimed_verified
2025-03-20T10:54:54.577Z
5f2c36551ebc8c6ede2f0e53
/avatars/e3ddbd15f50b86958377b5fc2460a57e.svg
Tony Kong
user
TonyK
6758ff6ea90b1c382daf3529
Yixin Shen
admin_assigned
2024-12-11T13:54:16.438Z
65e02dcd121aa0d0b4a95963
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/6C5bq1weQxI9s80u3Z8v1.jpeg
Yixin
user
yixinshen1
6758ff6ea90b1c382daf352a
Sung-En Chang
6758ff6ea90b1c382daf352b
Timothy Rupprecht
6758ff6ea90b1c382daf352c
Lei Lu
6758ff6ea90b1c382daf352d
Enfu Nan
admin_assigned
2024-12-11T13:54:31.486Z
65382338ef7d2bb8c4b373bc
/avatars/a469924cf3b983c6e420ee4dccce0290.svg
ENFU NAN
user
nanenfu
6758ff6ea90b1c382daf352e
Changdi Yang
admin_assigned
2024-12-11T13:54:43.185Z
654321438caf77017835e5c0
/avatars/999e63e33e17058baa2ce53d90f49d63.svg
yangchangdi
user
yangchangdi
6758ff6ea90b1c382daf352f
Yumei He
admin_assigned
2024-12-11T13:54:51.471Z
66c9ec0edf893af1ae3329b3
/avatars/0a09c5ad9e56bee64eb098d893b1c969.svg
Yumei He
user
Joy2024HuggingFace
6758ff6ea90b1c382daf3530
Xingchen Xu
6758ff6ea90b1c382daf3531
Yu Huang
admin_assigned
2024-12-11T13:55:07.226Z
6463d44a9dd8b530a85fe6c9
/avatars/24364533d8e3ccec20c0043c98967f35.svg
Yu Huang
user
yuhuang
6758ff6ea90b1c382daf3532
Wei Wang
6758ff6ea90b1c382daf3533
Yue Chen
6758ff6ea90b1c382daf3534
Yong He
admin_assigned
2024-12-11T13:53:21.591Z
674f84d2b2c5d446d8fcbe72
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/Vpq1zbM_8hZX9YENKnC-Q.png
Yong He
user
heyong4725
6758ff6ea90b1c382daf3535
Yanzhi Wang
Fully Open Source Moxin-7B Technical Report
Recently, Large Language Models (LLMs) have undergone a significant
transformation, marked by a rapid rise in both their popularity and
capabilities. Leading this evolution are proprietary LLMs like GPT-4 and
GPT-o1, which have captured widespread attention in the AI community due to
their remarkable performance and versatility. Simultaneously, open-source LLMs,
such as LLaMA and Mistral, have made great contributions to the ever-increasing
popularity of LLMs due to the ease to customize and deploy the models across
diverse applications. Although open-source LLMs present unprecedented
opportunities for innovation and research, the commercialization of LLMs has
raised concerns about transparency, reproducibility, and safety. Many
open-source LLMs fail to meet fundamental transparency requirements by
withholding essential components like training code and data, and some use
restrictive licenses whilst claiming to be "open-source," which may hinder
further innovations on LLMs. To mitigate this issue, we introduce Moxin 7B, a
fully open-source LLM developed in accordance with the Model Openness Framework
(MOF), a ranked classification system that evaluates AI models based on model
completeness and openness, adhering to principles of open science, open source,
open data, and open access. Our model achieves the highest MOF classification
level of "open science" through the comprehensive release of pre-training code
and configurations, training and fine-tuning datasets, and intermediate and
final checkpoints. Experiments show that our model achieves superior
performance in zero-shot evaluation compared with popular 7B models and
performs competitively in few-shot evaluation.
Moxin 7B is a fully open-source LLM that achieves high performance in zero-shot and few-shot evaluations while adhering to the Model Openness Framework principles.
Large Language Models LLMs GPT-4 GPT-o1 LLaMA Mistral Model Openness Framework MOF open science open source open data open access zero-shot evaluation few-shot evaluation