An accurate detection is not all you need to combat label noise in
  web-noisy datasets
Training a classifier on web-crawled data demands learning algorithms that
are robust to annotation errors and irrelevant examples. This paper builds upon
the recent empirical observation that applying unsupervised contrastive
learning to noisy, web-crawled datasets yields a feature representation under
which the in-distribution (ID) and out-of-distribution (OOD) samples are
linearly separable. We show that direct estimation of the separating hyperplane
can indeed offer an accurate detection of OOD samples, and yet, surprisingly,
this detection does not translate into gains in classification accuracy.
Digging deeper into this phenomenon, we discover that the near-perfect
detection misses a type of clean examples that are valuable for supervised
learning. These examples often represent visually simple images, which are
relatively easy to identify as clean examples using standard loss- or
distance-based methods despite being poorly separated from the OOD distribution
using unsupervised learning. Because we further observe a low correlation with
SOTA metrics, this urges us to propose a hybrid solution that alternates
between noise detection using linear separation and a state-of-the-art (SOTA)
small-loss approach. When combined with the SOTA algorithm PLS, we
substantially improve SOTA results for real-world image classification in the
presence of web noise github.com/PaulAlbert31/LSA
668dda5f963fdbcecc82a151
Paul Albert
claimed_verified
2024-07-10T08:19:49.399Z
65f26ed18404ac0e4cfe7d83
/avatars/9c72744836f86a8e355a45700b10e393.svg
Paul Albert
user
PAlbert31
668dda5f963fdbcecc82a152
Jack Valmadre
admin_assigned
2024-07-11T10:58:56.498Z
643632f2aaef013d1af27e54
/avatars/156a4257f3fce1e40f89b42f1ea3522d.svg
Jack Valmadre
user
jvlmdr
668dda5f963fdbcecc82a153
Eric Arazo
668dda5f963fdbcecc82a154
Tarun Krishna
668dda5f963fdbcecc82a155
Noel E. O'Connor
668dda5f963fdbcecc82a156
Kevin McGuinness
An accurate detection is not all you need to combat label noise in
  web-noisy datasets
Training a classifier on web-crawled data demands learning algorithms that
are robust to annotation errors and irrelevant examples. This paper builds upon
the recent empirical observation that applying unsupervised contrastive
learning to noisy, web-crawled datasets yields a feature representation under
which the in-distribution (ID) and out-of-distribution (OOD) samples are
linearly separable. We show that direct estimation of the separating hyperplane
can indeed offer an accurate detection of OOD samples, and yet, surprisingly,
this detection does not translate into gains in classification accuracy.
Digging deeper into this phenomenon, we discover that the near-perfect
detection misses a type of clean examples that are valuable for supervised
learning. These examples often represent visually simple images, which are
relatively easy to identify as clean examples using standard loss- or
distance-based methods despite being poorly separated from the OOD distribution
using unsupervised learning. Because we further observe a low correlation with
SOTA metrics, this urges us to propose a hybrid solution that alternates
between noise detection using linear separation and a state-of-the-art (SOTA)
small-loss approach. When combined with the SOTA algorithm PLS, we
substantially improve SOTA results for real-world image classification in the
presence of web noise github.com/PaulAlbert31/LSA
Incorporating unsupervised contrastive learning with a hybrid noise detection and SOTA small-loss approach enhances classification accuracy on noisy web-crawled datasets.
unsupervised contrastive learning in-distribution (ID) out-of-distribution (OOD) separating hyperplane state-of-the-art (SOTA) small-loss approach hybrid solution