F2LLM Technical Report: Matching SOTA Embedding Performance with 6
  Million Open-Source Data
We introduce F2LLM - Foundation to Feature Large Language Models, a suite of
state-of-the-art embedding models in three sizes: 0.6B, 1.7B, and 4B. Unlike
previous top-ranking embedding models that require massive contrastive
pretraining, sophisticated training pipelines, and costly synthetic training
data, F2LLM is directly finetuned from foundation models on 6 million
query-document-negative tuples curated from open-source, non-synthetic
datasets, striking a strong balance between training cost, model size, and
embedding performance. On the MTEB English leaderboard, F2LLM-4B ranks 2nd
among models with approximately 4B parameters and 7th overall, while F2LLM-1.7B
ranks 1st among models in the 1B-2B size range. To facilitate future research
in the field, we release the models, training dataset, and code, positioning
F2LLM as a strong, reproducible, and budget-friendly baseline for future works.
68df2abedf49fb0df1e03bdb
Ziyin Zhang
claimed_verified
2025-10-03T12:33:05.611Z
6430bdd8cd31d174a9f900fb
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/Y9SPnRfpKSbYc7MhNdP-H.jpeg
Ziyin Zhang
user
Geralt-Targaryen
68df2abedf49fb0df1e03bdc
Zihan Liao
68df2abedf49fb0df1e03bdd
Hang Yu
claimed_verified
2025-10-10T09:39:21.522Z
6837243353739ee0d588d04c
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/CYU8xxEHxxGkwNVCpua6X.png
Hang Yu
user
fhlyhv
68df2abedf49fb0df1e03bde
Peng Di
68df2abedf49fb0df1e03bdf
Rui Wang
F2LLM Technical Report: Matching SOTA Embedding Performance with 6
  Million Open-Source Data
We introduce F2LLM - Foundation to Feature Large Language Models, a suite of
state-of-the-art embedding models in three sizes: 0.6B, 1.7B, and 4B. Unlike
previous top-ranking embedding models that require massive contrastive
pretraining, sophisticated training pipelines, and costly synthetic training
data, F2LLM is directly finetuned from foundation models on 6 million
query-document-negative tuples curated from open-source, non-synthetic
datasets, striking a strong balance between training cost, model size, and
embedding performance. On the MTEB English leaderboard, F2LLM-4B ranks 2nd
among models with approximately 4B parameters and 7th overall, while F2LLM-1.7B
ranks 1st among models in the 1B-2B size range. To facilitate future research
in the field, we release the models, training dataset, and code, positioning
F2LLM as a strong, reproducible, and budget-friendly baseline for future works.
F2LLM, a suite of large language models, achieves high embedding performance with efficient fine-tuning from foundation models using open-source datasets.
embedding models foundation models contrastive pretraining training pipelines synthetic training data query-document-negative tuples MTEB English leaderboard parameter-efficient fine-tuning