Toxicity of the Commons: Curating Open-Source Pre-Training Data
Open-source large language models are becoming increasingly available and
popular among researchers and practitioners. While significant progress has
been made on open-weight models, open training data is a practice yet to be
adopted by the leading open-weight models creators. At the same time, there
researchers are working to make language models safer. We propose a data
curation pipeline to reduce harmful outputs by models trained on public domain
data. There are unique challenges to working with public domain data, as these
sources differ from web text in both form and content. Many sources are
historical documents and are the result of Optical Character Recognition (OCR).
Consequently, current state-of-the-art approaches to toxicity filtering are
often infeasible or inappropriate for open data models. In this paper, we
introduce a new fully open-source pipeline for open-data toxicity filtering.
Our contributions are threefold. We create a custom training dataset,
ToxicCommons, which is composed of texts which have been classified across five
different dimensions (racial/origin-based, gender/sex-based, religious,
ability-based discrimination, and violence). We use this dataset to train a
custom classifier, Celadon, that can be used to detect toxic content in open
data more efficiently at a larger scale. Finally, we describe the balanced
approach to content filtration that optimizes safety filtering with respect to
the filtered data available for training.
6722e83247de1a565f91c768
Catherine Arnett
claimed_verified
2024-10-31T14:06:04.601Z
6384d424ac61472e5e92d403
https://cdn-avatars.huggingface.co/v1/production/uploads/6384d424ac61472e5e92d403/RfEyLD3FmEobLcpNE9pNu.png
Catherine Arnett
user
catherinearnett
6722e83247de1a565f91c769
Eliot Jones
extracted_confirmed
2024-10-31T02:18:38.592Z
66214936126ebc0deb501a87
/avatars/e038e84f58c8577425df327a06b801d0.svg
Eliot Jones
user
eliotj
6722e83247de1a565f91c76a
Ivan P. Yamshchikov
extracted_confirmed
2024-10-31T15:30:07.401Z
6723a215f60c084a7dd016b8
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/5z5Ap4AFQhCkMJz3oOXCU.png
Prof. Ivan Yamshchikov
user
ivan-the-bearable
6722e83247de1a565f91c76b
Pierre-Carl Langlais
Toxicity of the Commons: Curating Open-Source Pre-Training Data
Open-source large language models are becoming increasingly available and
popular among researchers and practitioners. While significant progress has
been made on open-weight models, open training data is a practice yet to be
adopted by the leading open-weight models creators. At the same time, there
researchers are working to make language models safer. We propose a data
curation pipeline to reduce harmful outputs by models trained on public domain
data. There are unique challenges to working with public domain data, as these
sources differ from web text in both form and content. Many sources are
historical documents and are the result of Optical Character Recognition (OCR).
Consequently, current state-of-the-art approaches to toxicity filtering are
often infeasible or inappropriate for open data models. In this paper, we
introduce a new fully open-source pipeline for open-data toxicity filtering.
Our contributions are threefold. We create a custom training dataset,
ToxicCommons, which is composed of texts which have been classified across five
different dimensions (racial/origin-based, gender/sex-based, religious,
ability-based discrimination, and violence). We use this dataset to train a
custom classifier, Celadon, that can be used to detect toxic content in open
data more efficiently at a larger scale. Finally, we describe the balanced
approach to content filtration that optimizes safety filtering with respect to
the filtered data available for training.
A new fully open-source pipeline for toxicity filtering is introduced to detect and reduce harmful outputs in language models trained on public domain data.
OCR toxicity filtering custom training dataset ToxicCommons custom classifier Celadon