Manipulating Feature Visualizations with Gradient Slingshots
Dilyara Bareeva, Marina Höhne, Alexander Warnecke, Lukas Pirch, Klaus-Robert Müller, Konrad Rieck, Sebastian Lapuschkin, Kirill Bykov
Deep Neural Networks (DNNs) are capable of learning complex and diverse representations; however, the semantic nature of the concepts they capture remains poorly understood. A widely used technique for interpreting the functional role of concepts within a model is Feature Visualization (FV), which synthesizes an input pattern that maximally activates a given feature. In this paper, we examine the vulnerability of FV to adversarial fine-tuning of the model. We introduce a novel technique, Gradient Slingshots, that enables manipulation of FV without changing the model architecture. By shaping new trajectories in the off-distribution regions of the activation landscape of a feature, we coerce the optimization process to converge in a predefined visualization. Crucially, this manipulation largely preserves the model’s external behavior and internal mechanisms, enabling deceptive explanations while maintaining functional integrity. We validate the effectiveness of our technique across several neural network models, demonstrating its ability to conceal the true functionality of arbitrary neurons by replacing their original explanations with selected target explanations during model auditing.