LLM Teacher-Student Framework for Text Classification With No Manually
  Annotated Data: A Case Study in IPTC News Topic Classification
With the ever-increasing number of news stories available online, classifying
them by topic, regardless of the language they are written in, has become
crucial for enhancing readers' access to relevant content. To address this
challenge, we propose a teacher-student framework based on large language
models (LLMs) for developing multilingual news classification models of
reasonable size with no need for manual data annotation. The framework employs
a Generative Pretrained Transformer (GPT) model as the teacher model to develop
an IPTC Media Topic training dataset through automatic annotation of news
articles in Slovenian, Croatian, Greek, and Catalan. The teacher model exhibits
a high zero-shot performance on all four languages. Its agreement with human
annotators is comparable to that between the human annotators themselves. To
mitigate the computational limitations associated with the requirement of
processing millions of texts daily, smaller BERT-like student models are
fine-tuned on the GPT-annotated dataset. These student models achieve high
performance comparable to the teacher model. Furthermore, we explore the impact
of the training data size on the performance of the student models and
investigate their monolingual, multilingual and zero-shot cross-lingual
capabilities. The findings indicate that student models can achieve high
performance with a relatively small number of training instances, and
demonstrate strong zero-shot cross-lingual abilities. Finally, we publish the
best-performing news topic classifier, enabling multilingual classification
with the top-level categories of the IPTC Media Topic schema.
674d5e4ee208467411046583
Taja Kuzman
extracted_confirmed
2024-12-02T07:21:13.856Z
636e15698ba65db4a09384ab
https://cdn-avatars.huggingface.co/v1/production/uploads/636e15698ba65db4a09384ab/jtkbytatqb7oiIIB3oWMF.jpeg
Taja Kuzman Pungeršek
user
TajaKuzmanPungersek
674d5e4ee208467411046584
Nikola Ljubešić
admin_assigned
2024-12-02T09:46:03.442Z
601431aad1f82f3c09cb43ac
https://cdn-avatars.huggingface.co/v1/production/uploads/1632563266956-601431aad1f82f3c09cb43ac.png
Nikola Ljubešić
user
nljubesi
LLM Teacher-Student Framework for Text Classification With No Manually
  Annotated Data: A Case Study in IPTC News Topic Classification
With the ever-increasing number of news stories available online, classifying
them by topic, regardless of the language they are written in, has become
crucial for enhancing readers' access to relevant content. To address this
challenge, we propose a teacher-student framework based on large language
models (LLMs) for developing multilingual news classification models of
reasonable size with no need for manual data annotation. The framework employs
a Generative Pretrained Transformer (GPT) model as the teacher model to develop
an IPTC Media Topic training dataset through automatic annotation of news
articles in Slovenian, Croatian, Greek, and Catalan. The teacher model exhibits
a high zero-shot performance on all four languages. Its agreement with human
annotators is comparable to that between the human annotators themselves. To
mitigate the computational limitations associated with the requirement of
processing millions of texts daily, smaller BERT-like student models are
fine-tuned on the GPT-annotated dataset. These student models achieve high
performance comparable to the teacher model. Furthermore, we explore the impact
of the training data size on the performance of the student models and
investigate their monolingual, multilingual and zero-shot cross-lingual
capabilities. The findings indicate that student models can achieve high
performance with a relatively small number of training instances, and
demonstrate strong zero-shot cross-lingual abilities. Finally, we publish the
best-performing news topic classifier, enabling multilingual classification
with the top-level categories of the IPTC Media Topic schema.
A teacher-student framework using large language models and smaller BERT-like models for automatic, multilingual news classification without manual annotation achieves high performance and zero-shot cross-lingual capabilities.
teacher-student framework large language models LLMs Generative Pretrained Transformer GPT IPTC Media Topic automatic annotation Slovenian Croatian Greek Catalan zero-shot performance BERT-like models fine-tuning monolingual multilingual zero-shot cross-lingual top-level categories