PCA++: Robust Contrastive PCA under Large Background and Random Noise
Mingqi Wu, Qiang Sun, Archer Yang
Real-world high-dimensional data often hide low-dimensional signals beneath dominant structured backgrounds—from batch effects in genomics to scanner artifacts in imaging. We introduce PCA++, a contrastive PCA that enforces a hard uniformity constraint—requiring projected features to have identity covariance—to robustly isolate shared signal subspaces under strong background noise. PCA++ admits a closed-form solution via a generalized eigendecomposition with low-rank truncation for stability and enjoys rigorous non-asymptotic and high-dimensional guarantees under a linear contrastive factor model. In the fixed aspect-ratio regime, we derive exact BBP‐style limits for subspace error; in the growing-spike regime, we obtain a simple closed-form bias that vanishes as the signal strengthens. Empirically, PCA++ outperforms standard PCA and alignment-only contrastive PCA on simulations, corrupted-MNIST, and single-cell transcriptomics, reliably recovering condition-invariant structure. More broadly, we clarify uniformity’s role in contrastive learning—showing that explicit feature dispersion defends against structured noise and enhances robustness.