Benchmarking Recommendation, Classification, and Tracing Based on
  Hugging Face Knowledge Graph
The rapid growth of open source machine learning (ML) resources, such as
models and datasets, has accelerated IR research. However, existing platforms
like Hugging Face do not explicitly utilize structured representations,
limiting advanced queries and analyses such as tracing model evolution and
recommending relevant datasets. To fill the gap, we construct HuggingKG, the
first large-scale knowledge graph built from the Hugging Face community for ML
resource management. With 2.6 million nodes and 6.2 million edges, HuggingKG
captures domain-specific relations and rich textual attributes. It enables us
to further present HuggingBench, a multi-task benchmark with three novel test
collections for IR tasks including resource recommendation, classification, and
tracing. Our experiments reveal unique characteristics of HuggingKG and the
derived tasks. Both resources are publicly available, expected to advance
research in open source resource sharing and management.
6833f24ed5c438959f7decf9
Qiaosheng Chen
claimed_verified
2025-05-28T09:01:00.198Z
619ef3f253061ce00477b09e
https://cdn-avatars.huggingface.co/v1/production/uploads/619ef3f253061ce00477b09e/FknZhgQhV2_3aTqIKVsTo.jpeg
Qiaosheng Chen
user
cqsss
6833f24ed5c438959f7decfa
Kaijia Huang
6833f24ed5c438959f7decfb
Xiao Zhou
6833f24ed5c438959f7decfc
Weiqing Luo
6833f24ed5c438959f7decfd
Yuanning Cui
6833f24ed5c438959f7decfe
Gong Cheng
Benchmarking Recommendation, Classification, and Tracing Based on
  Hugging Face Knowledge Graph
The rapid growth of open source machine learning (ML) resources, such as
models and datasets, has accelerated IR research. However, existing platforms
like Hugging Face do not explicitly utilize structured representations,
limiting advanced queries and analyses such as tracing model evolution and
recommending relevant datasets. To fill the gap, we construct HuggingKG, the
first large-scale knowledge graph built from the Hugging Face community for ML
resource management. With 2.6 million nodes and 6.2 million edges, HuggingKG
captures domain-specific relations and rich textual attributes. It enables us
to further present HuggingBench, a multi-task benchmark with three novel test
collections for IR tasks including resource recommendation, classification, and
tracing. Our experiments reveal unique characteristics of HuggingKG and the
derived tasks. Both resources are publicly available, expected to advance
research in open source resource sharing and management.
HuggingKG, a large-scale knowledge graph, enhances open source ML resource management by enabling advanced queries and analyses via HuggingBench.
knowledge graph resource recommendation classification tracing multi-task benchmark