Assessing the Effectiveness of GPT-3 in Detecting False Political Statements: A Case Study on the LIAR Dataset
The detection of political fake statements is crucial for maintaining
information integrity and preventing the spread of misinformation in society.
Historically, state-of-the-art machine learning models employed various methods
for detecting deceptive statements. These methods include the use of metadata
(W. Wang et al., 2018), n-grams analysis (Singh et al., 2021), and linguistic
(Wu et al., 2022) and stylometric (Islam et al., 2020) features. Recent
advancements in large language models, such as GPT-3 (Brown et al., 2020) have
achieved state-of-the-art performance on a wide range of tasks. In this study,
we conducted experiments with GPT-3 on the LIAR dataset (W. Wang et al., 2018)
and achieved higher accuracy than state-of-the-art models without using any
additional meta or linguistic features. Additionally, we experimented with
zero-shot learning using a carefully designed prompt and achieved near
state-of-the-art performance. An advantage of this approach is that the model
provided evidence for its decision, which adds transparency to the model's
decision-making and offers a chance for users to verify the validity of the
evidence provided.