AutoSciDACT: Automated Scientific Discovery through Contrastive Embedding and Hypothesis Testing
Samuel Bright-Thonney, Christina Reissel, Gaia Grosso, Nathaniel Woodward, Katya Govorkova, Andrzej Novak, Sangeon Park, Eric Moreno, Philip Harris
Novelty detection in large scientific datasets faces two key challenges: the intrinsic noise and high dimensionality of experimental data, and the necessity of making statistically robust statements about any observed outliers. While there is a wealth of literature on anomaly detection via dimensionality reduction, most methods do not produce outputs compatible with claims of scientific discovery. In this work we directly address these challenges, presenting the first step towards a unified pipeline for novelty detection adapted for the rigorous demands of scientific discovery. We introduce AutoSciDACT (Automated Scientific Discovery with Anomalous Contrastive Testing), a general purpose two-stage pipeline for detecting novelty in scientific data. The initial pre-training stage involves constructing a low-dimensional embedding of a dataset via contrastive learning, leveraging the abundance of high-quality simulated data in many scientific domains alongside specialist scientific expertise that can guide principled data augmentation strategies.  Next, the discovery stage takes a sample of well-understood in-distribution points (i.e.\ a null hypothesis or background-only set) and performs a powerful machine learning-driven two-sample test against another dataset with unknown composition (i.e.\ observed data) using the New Physics Learning Machine (NPLM) framework. We perform experiments across a range of astronomical, physical, biological, image, and synthetic datasets, demonstrating extraordinary sensitivity to small fractions of anomalous data across all domains.