TruEyes: Utilizing Microtasks in Mobile Apps for Crowdsourced Labeling of Machine Learning Datasets
The growing use of supervised machine learning in research and industry has
increased the need for labeled datasets. Crowdsourcing has emerged as a popular
method to create data labels. However, working on large batches of tasks leads
to worker fatigue, negatively impacting labeling quality. To address this, we
present TruEyes, a collaborative crowdsourcing system, enabling the
distribution of micro-tasks to mobile app users. TruEyes allows machine
learning practitioners to publish labeling tasks, mobile app developers to
integrate task ads for monetization, and users to label data instead of
watching advertisements. To evaluate the system, we conducted an experiment
with N=296 participants. Our results show that the quality of the labeled data
is comparable to traditional crowdsourcing approaches and most users prefer
task ads over traditional ads. We discuss extensions to the system and address
how mobile advertisement space can be used as a productive resource in the
future.