CritiQ: Mining Data Quality Criteria from Human Preferences
Language model heavily depends on high-quality data for optimal performance.
Existing approaches rely on manually designed heuristics, the perplexity of
existing models, training classifiers, or careful prompt engineering, which
require significant expert experience and human annotation effort while
introduce biases. We introduce CritiQ, a novel data selection method that
automatically mines criteria from human preferences for data quality with only
sim30 human-annotated pairs and performs efficient data selection. The main
component, CritiQ Flow, employs a manager agent to evolve quality criteria and
worker agents to make pairwise judgments. We build a knowledge base that
extracts quality criteria from previous work to boost CritiQ Flow. Compared to
perplexity- and classifier- based methods, verbal criteria are more
interpretable and possess reusable value. After deriving the criteria, we train
the CritiQ Scorer to give quality scores and perform efficient data selection.
We demonstrate the effectiveness of our method in the code, math, and logic
domains, achieving high accuracy on human-annotated test sets. To validate the
quality of the selected data, we continually train Llama 3.1 models and observe
improved performance on downstream tasks compared to uniform sampling. Ablation
studies validate the benefits of the knowledge base and the reflection process.
We analyze how criteria evolve and the effectiveness of majority voting.
67bffaca3f838c1e33e074e7
Honglin Guo
claimed_verified
2025-02-27T09:13:52.094Z
638ef0b0c67af472d31674a6
/avatars/02df97d15a0f46b47f9162221733b121.svg
Honglin Guo
user
KYLN24
67bffaca3f838c1e33e074e8
Kai Lv
67bffaca3f838c1e33e074e9
Qipeng Guo
67bffaca3f838c1e33e074ea
Tianyi Liang
claimed_verified
2025-05-06T08:36:33.906Z
62c14609ac1b639c2d87192c
https://cdn-avatars.huggingface.co/v1/production/uploads/1656833489364-noauth.png
liangtianyi
user
tianyilt
67bffaca3f838c1e33e074eb
Zhiheng Xi
67bffaca3f838c1e33e074ec
Demin Song
67bffaca3f838c1e33e074ed
Qiuyinzhe Zhang
67bffaca3f838c1e33e074ee
Yu Sun
67bffaca3f838c1e33e074ef
Kai Chen
67bffaca3f838c1e33e074f0
Xipeng Qiu
67bffaca3f838c1e33e074f1
Tao Gui
CritiQ: Mining Data Quality Criteria from Human Preferences
Language model heavily depends on high-quality data for optimal performance.
Existing approaches rely on manually designed heuristics, the perplexity of
existing models, training classifiers, or careful prompt engineering, which
require significant expert experience and human annotation effort while
introduce biases. We introduce CritiQ, a novel data selection method that
automatically mines criteria from human preferences for data quality with only
sim30 human-annotated pairs and performs efficient data selection. The main
component, CritiQ Flow, employs a manager agent to evolve quality criteria and
worker agents to make pairwise judgments. We build a knowledge base that
extracts quality criteria from previous work to boost CritiQ Flow. Compared to
perplexity- and classifier- based methods, verbal criteria are more
interpretable and possess reusable value. After deriving the criteria, we train
the CritiQ Scorer to give quality scores and perform efficient data selection.
We demonstrate the effectiveness of our method in the code, math, and logic
domains, achieving high accuracy on human-annotated test sets. To validate the
quality of the selected data, we continually train Llama 3.1 models and observe
improved performance on downstream tasks compared to uniform sampling. Ablation
studies validate the benefits of the knowledge base and the reflection process.
We analyze how criteria evolve and the effectiveness of majority voting.
CritiQ automates data selection using human preferences, improving language model performance through efficient and interpretable criteria.
CritiQ CritiQ Flow manager agent worker agents quality criteria knowledge base CritiQ Scorer data selection Llama 3.1 models downstream tasks majority voting