Dark patterns in e-commerce: a dataset and its baseline evaluations
Dark patterns, which are user interface designs in online services, induce
users to take unintended actions. Recently, dark patterns have been raised as
an issue of privacy and fairness. Thus, a wide range of research on detecting
dark patterns is eagerly awaited. In this work, we constructed a dataset for
dark pattern detection and prepared its baseline detection performance with
state-of-the-art machine learning methods. The original dataset was obtained
from Mathur et al.'s study in 2019, which consists of 1,818 dark pattern texts
from shopping sites. Then, we added negative samples, i.e., non-dark pattern
texts, by retrieving texts from the same websites as Mathur et al.'s dataset.
We also applied state-of-the-art machine learning methods to show the automatic
detection accuracy as baselines, including BERT, RoBERTa, ALBERT, and XLNet. As
a result of 5-fold cross-validation, we achieved the highest accuracy of 0.975
with RoBERTa. The dataset and baseline source codes are available at
https://github.com/yamanalab/ec-darkpattern.