Kernel Learning with Adversarial Features: Numerical Efficiency and Adaptive Regularization
Antonio Ribeiro, David Vävinggren, Dave Zachariah, Thomas Schön, Francis Bach
Adversarial training has emerged as a key technique to enhance model robustness against adversarial input perturbations. Many of the existing methods rely on computationally expensive min-max problems that limit their application in practice. We propose a novel formulation of adversarial training in reproducing kernel Hilbert spaces, shifting from input to feature-space perturbations.  This reformulation enables a convex optimization problem that can be efficiently solved and provides a natural adaptive regularization to the noise level and properties of the underlying function. We establish theoretical conditions under which the relaxed formulation serves as a proper approximation of the original problem, prove convexity, and propose an efficient optimization algorithm based on iterative kernel ridge regression. We also provide generalization bounds that help to understand the properties of the method. Furthermore, we extend the formulation to multiple kernel learning. We evaluate our method on synthetic and real-world datasets.