text:
\section{Algorithmic details}

\subsection{Problem formulation for model repairment}
\label{app:problem_formulation}

This section extends the discussion on the problem formulation of model repairment. Specifically, under the modelling assumptions presented in the main text, the goals for model repairment are the following:
\begin{itemize}
    \item For a set of ``failure cases'' $\mathcal{F} = \{ \bm{z}_f = (\bm{x}, y) \}$ where the model with (Bayesian) predictive inference makes wrong predictions, repair the model to make correct predictions on $\mathcal{F}$ and similar cases.
    \item For a set of ``benchmark cases'' $\mathcal{B} = \{ \bm{z}_b = (\bm{x}, y)\}$, maintain a given level of prediction accuracy after model repairment.
\end{itemize}

There are further considerations for executing and evaluating model repairment in practice.
\paragraph{Generalisation and efficiency of model repairment}

In practice the number of failure cases might be large or even infinite. For example, an image classification model that fails on a test input with Gaussian noise may also fail on all the other inputs with such level of noise. So we need to consider both, \emph{generalisation} and \emph{efficiency} aspects of model repairment. Here, good generalisation means that the failure is fixed, not only for observed failure cases but also for future cases for which the model would make the same type of failure before fixes. On the other hand, efficiency of repairment considers the number of failure examples required for the model repairment method to fix a particular type of failure.

To describe both concepts in more details, in addition to a set of known/observed failure examples $\mathcal{F}$ collected by the users, we need to define the set of \emph{unknown/unobserved} failure examples $\mathcal{F}_U$. If examples in $\mathcal{F}$ and $\mathcal{F}_U$ are similar, then good \emph{generalisation} of a model repairment algorithm means that a model repaired by such method using information from $\mathcal{F}$ should produce correct predictions for instances in $\mathcal{F}_U$. On the other hand, a model repairment method is \emph{efficient} if it only needs a small set $\mathcal{F}$ of collected failure cases to achieve good generalisation of repairment.

\paragraph{Specificity of model repairment}
Furthermore, there may be multiple different scenarios in which the model fails, and therefore the set of failure cases may consist of several groups, i.e.,
\begin{equation}
\mathcal{F} = \sqcup_{m=1}^M \mathcal{F}^{(m)}, \quad \mathcal{F}_U = \sqcup_{m=1}^M \mathcal{F}_U^{(m)},
\end{equation}
where $\mathcal{F}^{(m)}$ denotes the observed failure cases of failure type $m$ and $\mathcal{F}_U^{(m)}$ represents unobserved failure cases of the same type.
Targeting a specific type of mistake and repairing it one at a time may be desirable in practice. One type of mistake might incur more costs than others (e.g., in medical applications, false negative is generally more costly than false positive), so users might have different priorities for different types of errors to be fixed. This is especially the case when there exists a trade-off between fixing different types of errors, again the false negative vs false positive trade-off is a prevalent example.

\paragraph{Repairment by identifying and removing detrimental training data}
There can be many different reasons for a model with Bayesian predictive inference making wrong predictions on $\mathcal{F}$. In this work, we assume that \emph{the main reason is due to the existence of detrimental datapoints in $\mathcal{D}$}. Our hypothesis is that, by removing/correcting these detrimental datapoints and adapting the model with them, the model can be repaired to return correct labels for datapoints in $\mathcal{F}$. Base on the above hypothesis, the model repairment process contains the following steps:
\begin{enumerate}
    \item \textbf{Cause identification:} Identify a set of detrimental data points $\mathcal{C}$ in the training data $\mathcal{D}$ that contributed the most to the failure set $\mathcal{F}$.
    \item \textbf{Treatment: } Given the set of failure causes $\mathcal{C}$, adapt the model to predict correctly on the failure set $\mathcal{F}$, while maintaining performance on other examples which were correctly predicted previously.
\end{enumerate}

\subsection{The ``predictive approach'' for cause identification} \label{sec:supp_derivations}
We use the following function to describe the contribution of a subset $\mathcal{C} \subset \mathcal{D}$ to the model failures on examples in $\mathcal{F}$:
\begin{equation}
    r(\mathcal{C}) =\log \left(\frac{p(\mathcal{F} \,|\, \mathcal{D} \setminus \mathcal{C})}{p(\mathcal{F} \,|\, \mathcal{D})} \right).
\end{equation}
A naive approach would compute $p({\boldsymbol{\theta}} \,|\, \mathcal{D} \setminus \mathcal{C})$ for all subsets of $\mathcal{D}$ with all possible correction methods, which is prohibitively expensive. Instead we present a \textbf{``predictive'' approach} that removes this computational burden. First, notice that we make the i.i.d.~modelling assumption which means that one can write the likelihood term as  follows:
\begin{equation}
    p(\mathcal{D} \,|\, {\boldsymbol{\theta}}) = p(\mathcal{D} \setminus \mathcal{C} \,|\, {\boldsymbol{\theta}}) p(\mathcal{C} \,|\, {\boldsymbol{\theta}}), \quad \forall \; \mathcal{C} \subset \mathcal{D}.
\label{eq:iid_factorisation}
\end{equation}
This allows us to expand the log evidence $\log \, p(\mathcal{F} \,|\, \mathcal{D} \setminus \mathcal{C})$ as
\begin{equation*}
\begin{aligned}
\log p(\mathcal{F} \,|\, \mathcal{D} \setminus \mathcal{C}) &= \log \int p(\mathcal{F} \,|\, {\boldsymbol{\theta}}) p({\boldsymbol{\theta}} \,|\, \mathcal{D} \setminus \mathcal{C}) d{\boldsymbol{\theta}} \\
&= \log \int p(\mathcal{F} \,|\, {\boldsymbol{\theta}}) \textcolor{red}{\frac{p(\mathcal{D} \setminus \mathcal{C} \,|\, {\boldsymbol{\theta}}) p({\boldsymbol{\theta}})}{p(\mathcal{D} \setminus \mathcal{C})}} d{\boldsymbol{\theta}} \quad\quad \text{(Bayes' rule)} \\
&= \log \int p(\mathcal{F} \,|\, {\boldsymbol{\theta}}) \frac{ \textcolor{red}{p(\mathcal{D} \,|\, {\boldsymbol{\theta}})} p({\boldsymbol{\theta}})}{ \textcolor{red}{p(\mathcal{C} \,|\, {\boldsymbol{\theta}}) } p(\mathcal{D} \setminus \mathcal{C})} d{\boldsymbol{\theta}} \quad\quad \text{(by Eq.~(\ref{eq:iid_factorisation}))} \\
&= \log \int \frac{\textcolor{red}{p(\mathcal{D})}}{p(\mathcal{D}\setminus \mathcal{C})}\cdot \frac{p(\mathcal{F}\,|\, {\boldsymbol{\theta}}) }{p(\mathcal{C}\,|\, {\boldsymbol{\theta}})} \cdot \frac{p( \mathcal{D}, {\boldsymbol{\theta}})}{\textcolor{red}{p(\mathcal{D})}} d{\boldsymbol{\theta}} \quad\quad \text{(multiplying $\frac{p(\mathcal{D})}{p(\mathcal{D})}$ and rearranging terms)}\\
&= \log \int \frac{p(\mathcal{F} \,|\, {\boldsymbol{\theta}})}{p(\mathcal{C}\,|\, {\boldsymbol{\theta}})} \textcolor{red}{p({\boldsymbol{\theta}} \,|\, \mathcal{D})} d{\boldsymbol{\theta}} + \log \frac{p(\mathcal{D})}{p(\mathcal{D}\setminus \mathcal{C})}. \quad\quad \text{(Bayes' rule)}
\end{aligned}
\end{equation*}
Then we can rewrite the log density ratio as
\begin{equation}
\begin{aligned}
r(\mathcal{C})&=\log \left(\frac{p(\mathcal{F} \,|\, \mathcal{D} \setminus \mathcal{C})}{p(\mathcal{F} \,|\, \mathcal{D})} \right)
\\ &= \log \int \frac{p(\mathcal{F} \,|\, {\boldsymbol{\theta}})}{p(\mathcal{C}\,|\, {\boldsymbol{\theta}})} p({\boldsymbol{\theta}} \,|\, \mathcal{D}) d{\boldsymbol{\theta}}
- \log \, p(\mathcal{F} \,|\, \mathcal{D})
+ \log p(\mathcal{D}) \textcolor{red}{- \log \int p(\mathcal{D} \setminus \mathcal{C} \,|\, {\boldsymbol{\theta}}) p({\boldsymbol{\theta}}) d{\boldsymbol{\theta}}} \\
& \qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad \text{(by definition of marginal distributions)}\\
\\ &= \log \int \frac{1}{p(\mathcal{C}\,|\, {\boldsymbol{\theta}})} \frac{p(\mathcal{F} \,|\, {\boldsymbol{\theta}}) p({\boldsymbol{\theta}} \,|\, \mathcal{D})}{p(\mathcal{F} \,|\, \mathcal{D})}  d{\boldsymbol{\theta}}
- \log \int \frac{\textcolor{red}{p(\mathcal{D} \,|\, {\boldsymbol{\theta}})}}{\textcolor{red}{p(\mathcal{C} \,|\, {\boldsymbol{\theta}})}} \frac{p({\boldsymbol{\theta}})}{p(\mathcal{D})} d{\boldsymbol{\theta}} \\
& \qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad \text{(by Eq.~(\ref{eq:iid_factorisation}) and rearranging terms)}\\
&= \log \int \frac{ \textcolor{red}{p({\boldsymbol{\theta}} \,|\, \mathcal{D}, \mathcal{F})}}{p(\mathcal{C}\,|\, {\boldsymbol{\theta}})} d{\boldsymbol{\theta}} - \log \int \frac{\textcolor{red}{p({\boldsymbol{\theta}} \,|\, \mathcal{D})}}{p(\mathcal{C}\,|\, {\boldsymbol{\theta}})}  d{\boldsymbol{\theta}} . \quad\quad \text{(Bayes' rule)}
\end{aligned}
\end{equation}
By doing so, instead of computing $p({\boldsymbol{\theta}} \,|\, \mathcal{D} \setminus \mathcal{C})$ for every possible subset $\mathcal{C}$, the ``predictive approach'' only requires computing $p({\boldsymbol{\theta}} \,|\, \mathcal{D}, \mathcal{F})$ once. As shown in the main text, with (approximations of) the two posteriors $p({\boldsymbol{\theta}} \,|\, \mathcal{D})$ and $p({\boldsymbol{\theta}} \,|\, \mathcal{D}, \mathcal{F})$ at hand, the log density ratio $r(\mathcal{C})$ can be efficiently approximated by Monte Carlo and/or further approximations described in the main text that employ Taylor expansions. This approach is ``predictive'' in the sense that the influence of candidate set $\mathcal{C}$ is evaluated by computing ``predictions'' $p(\mathcal{C} \,|\, {\boldsymbol{\theta}})^{-1}$ on them using the two posterior distributions, which is different from existing approaches that compute predictions on $\mathcal{F}$ using approximations to the modified posterior $p({\boldsymbol{\theta}} \,|\, \mathcal{D} \setminus \mathcal{C})$.

\subsection{Objective for EWC-influence}
\label{app:ewc_identification_objective}
Recall in the main text the first-order Taylor series approximation to $r(\mathcal{C})$ is
\begin{equation*}
    \begin{aligned}
    r(\mathcal{C}) \approx \sum_{\boldsymbol{z} \in \mathcal{C}} \ \hat{r}(\boldsymbol{z}), \quad \hat{r}(\boldsymbol{z})= \mathbb{E}_{p({\boldsymbol{\theta}} \,|\, \mathcal{D})}\left[\log p(\boldsymbol{z} \,|\, {\boldsymbol{\theta}}) \right] - \mathbb{E}_{p({\boldsymbol{\theta}} \,|\, \mathcal{D}, \mathcal{F})}\left[\log p(\boldsymbol{z} \,|\, {\boldsymbol{\theta}}) \right].
    \end{aligned}
\end{equation*}
Therefore the ``predictive approach'' for computing $r(\mathcal{C})$ as well as the approximated form require the computation of (approximate) posteriors $p({\boldsymbol{\theta}} \,|\, \mathcal{D})$ and $p({\boldsymbol{\theta}} \,|\, \mathcal{D}, \mathcal{F})$. This can be achieved using continual learning: we assume the model has been trained on $\mathcal{D}$ and an approximation $q({\boldsymbol{\theta}}) \approx p({\boldsymbol{\theta}} \,|\, \mathcal{D})$ has been obtained. Then the current task for continual learning is to adapt the trained model on the failure cases $\mathcal{F}$, which leads to an adapted approximation $q^{\star}_{\mathcal{D},\mathcal{F}}({\boldsymbol{\theta}}) \approx p({\boldsymbol{\theta}} \,|\, \mathcal{D}, \mathcal{F})$.

Elastic Weight Consolidation (EWC) \citep{kirkpatrick2017overcoming} is a continual learning algorithm that can be interpreted as updating the maximum a posteriori (MAP) approximation to the posterior given new tasks. To see this, first in this approach the $q$ posteriors are assumed to be delta measures. In other words, $q({\boldsymbol{\theta}}) = \delta({\boldsymbol{\theta}} - \hat{{\boldsymbol{\theta}}})$ where $\hat{{\boldsymbol{\theta}}}$ is the parameters of the trained model, and $q^{\star}_{\mathcal{D},\mathcal{F}}({\boldsymbol{\theta}}) = \delta({\boldsymbol{\theta}} - \hat{{\boldsymbol{\theta}}}^{\star}_{\mathcal{D},\mathcal{F}})$ where as we shall see $\hat{{\boldsymbol{\theta}}}^{\star}_{\mathcal{D},\mathcal{F}}$ is the parameter obtained by running EWC using $\mathcal{F}$. With these assumptions, the EWC-influence is defined as:
\begin{equation*}
    \begin{aligned}
    r(\mathcal{C}) \approx \sum_{\boldsymbol{z} \in \mathcal{C}} \ \hat{r}(\boldsymbol{z}) \approx \sum_{\boldsymbol{z} \in \mathcal{C}} \ \tilde{r}(\boldsymbol{z}), \quad \tilde{r}(\boldsymbol{z})= \log p(\boldsymbol{z} \,|\, \hat{{\boldsymbol{\theta}}})  - \log p(\boldsymbol{z} \,|\, \hat{{\boldsymbol{\theta}}}^{\star}_{\mathcal{D},\mathcal{F}}).
    \end{aligned}
\end{equation*}

It remains to discuss the optimisation procedure for obtaining $\hat{{\boldsymbol{\theta}}}^{\star}_{\mathcal{D},\mathcal{F}}$. As motivated, we consider MAP approximations to the posterior, which seeks to find the maximum of the log posterior $\log \, p({\boldsymbol{\theta}} \,|\, \mathcal{D}, \mathcal{F})$. Notice that by Bayes' rule:
\begin{equation*}
    p({\boldsymbol{\theta}} \,|\, \mathcal{D}, \mathcal{F}) \propto p(\mathcal{F} \,|\, {\boldsymbol{\theta}}) p({\boldsymbol{\theta}} \,|\, \mathcal{D}) \quad \Rightarrow \quad \log \, p({\boldsymbol{\theta}} \,|\, \mathcal{D}, \mathcal{F}) = \log \, p(\mathcal{F} \,|\, {\boldsymbol{\theta}} ) + \log \, p({\boldsymbol{\theta}} \,|\, \mathcal{D}) + \text{constant}.
\end{equation*}
Computing the first term $\log \, p(\mathcal{F} \,|\, {\boldsymbol{\theta}})$ in the MAP objective is straightforward given the i.i.d.~modelling assumption. For the second term, as $\log \, p({\boldsymbol{\theta}} \,|\, \mathcal{D})$ is intractable, the EWC approach constructs a Laplace approximation to it by assuming the trained model parameter $\hat{{\boldsymbol{\theta}}}$ as a MAP point estimate of $p({\boldsymbol{\theta}} \,|\, \mathcal{D})$. In detail, a Laplace approximation to the posterior is
\begin{equation} \label{eq:taylor_expansion}
\log \, p({\boldsymbol{\theta}}\,|\, \mathcal{D}) \approx \frac{1}{2}({\boldsymbol{\theta}} - \hat{{\boldsymbol{\theta}}})^\top \boldsymbol{H}[\log \, p(\hat{{\boldsymbol{\theta}}} \,|\, \mathcal{D})] ({\boldsymbol{\theta}} - \hat{{\boldsymbol{\theta}}}) + \text{constant}.
\end{equation}
where $\boldsymbol{H}[f({\boldsymbol{\theta}})]$ denotes the Hessian matrix of a twice-differentiable function $f({\boldsymbol{\theta}})$ with respect to parameters ${\boldsymbol{\theta}}$. Further decomposing the Hessian term yields:
\begin{align}
    \boldsymbol{H}[\log \, p(\hat{{\boldsymbol{\theta}}} \,|\, \mathcal{D})] & =  \boldsymbol{H}[\log \, p(\mathcal{D}\,|\, \hat{{\boldsymbol{\theta}}} ) + \log \, p(\hat{{\boldsymbol{\theta}}}) - \log \, p(\mathcal{D})]\\
    & = \underbrace{\boldsymbol{H}[\log p(\mathcal{D}\,|\, \hat{{\boldsymbol{\theta}}} )]}_{\approx -N\hat{\boldsymbol{F}}_{\hat{{\boldsymbol{\theta}}}}} + \underbrace{\boldsymbol{H}[\log p(\hat{{\boldsymbol{\theta}}})]}_{\text{Prior term}}
\end{align}
where $N$ is the total number of samples in $\mathcal{D}$ and $\hat{\boldsymbol{F}}_{\hat{{\boldsymbol{\theta}}}}$ is an empirical estimate of the \emph{Fisher information matrix}. Assuming the zero-mean isotropic Gaussian prior $p({\boldsymbol{\theta}})$ with precision $\lambda\boldsymbol{I}$,  the second term becomes $\boldsymbol{H}[\log p(\hat{{\boldsymbol{\theta}}})] = -\lambda \boldsymbol{I}$. Substituting these results back into Eq.~(\ref{eq:taylor_expansion}) gives us:
\begin{equation}
    \log \, p({\boldsymbol{\theta}}\,|\, \mathcal{D}) \approx - \frac{N}{2}({\boldsymbol{\theta}} - \hat{{\boldsymbol{\theta}}})^\top \hat{\boldsymbol{F}}_{\hat{{\boldsymbol{\theta}}}}  ({\boldsymbol{\theta}} - \hat{{\boldsymbol{\theta}}}) - \frac{\lambda}{2}||{\boldsymbol{\theta}} - \hat{{\boldsymbol{\theta}}}||^2_2 + \text{constant}.
\end{equation}
Combining terms, the optimisation task for EWC adaptation using failure set is:
\begin{equation}
    \hat{{\boldsymbol{\theta}}}^{\star}_{\mathcal{D},\mathcal{F}} = \arg\max_{{\boldsymbol{\theta}}} \ \log \, p(\mathcal{F} \,|\, {\boldsymbol{\theta}}) - \frac{N}{2}({\boldsymbol{\theta}} - \hat{{\boldsymbol{\theta}}})^\top \hat{\boldsymbol{F}}_{\hat{{\boldsymbol{\theta}}}}  ({\boldsymbol{\theta}} - \hat{{\boldsymbol{\theta}}}) - \frac{\lambda}{2}||{\boldsymbol{\theta}} - \hat{{\boldsymbol{\theta}}}||^2_2,
\end{equation}
which is as presented in the main text.
We further note that in the original EWC formulation, the empirical Fisher information is further simplified to contain \textit{diagonal} entries only --- with millions of model parameters as is typically the case with neural networks, as computing the full matrix is expensive.

\subsection{Objective for EWC-deletion}
\label{app:ewc_deletion_objective}
Assuming that the detrimental datapoints $\mathcal{C} \subset \mathcal{D}$ have been identified, the next step for model repairment is \emph{treatment} where we seek to remove the influence of $\mathcal{C}$ to the model. This is done by computing (an approximation to) $p({\boldsymbol{\theta}} \,|\, \mathcal{D} \setminus \mathcal{C})$ which can also be done via continue learning. To see this, we can show by using Bayes' rule and Eq.~(\ref{eq:iid_factorisation}) again,
\begin{equation}
    \log p({\boldsymbol{\theta}} \,|\, \mathcal{D} \setminus \mathcal{C}) = \log \frac{p(\mathcal{D} \setminus \mathcal{C} \,|\, {\boldsymbol{\theta}})p({\boldsymbol{\theta}})}{p(\mathcal{D} \setminus \mathcal{C})} = \log \frac{1}{p(\mathcal{C}\,|\, {\boldsymbol{\theta}})} + \log \underbrace{\frac{p(\mathcal{D}\,|\, {\boldsymbol{\theta}})p({\boldsymbol{\theta}})}{p(\mathcal{D})}}_{= p({\boldsymbol{\theta}} \,|\, \mathcal{D})} + \log \frac{p(\mathcal{D})}{p(\mathcal{D} \setminus \mathcal{C})}.
\end{equation}
This means that finding the MAP estimate of $p({\boldsymbol{\theta}} \,|\, \mathcal{D} \setminus \mathcal{C})$ is equivalent to maximising $-\log p(\mathcal{C} \,|\, {\boldsymbol{\theta}}) + \log p({\boldsymbol{\theta}} \,|\, \mathcal{D})$ w.r.t.~${\boldsymbol{\theta}}$. With further approximation to the posterior $p({\boldsymbol{\theta}} \,|\, \mathcal{D})$ using the Laplace method as discussed in section \ref{app:ewc_identification_objective}, one can write the optimisation task for EWC-deletion as presented in the main text, namely:
\begin{equation}
    \hat{{\boldsymbol{\theta}}}^{\star\star}_{\mathcal{D} \setminus \mathcal{C}} = \arg\max_{{\boldsymbol{\theta}}} \ -\log p(\mathcal{C} \,|\, {\boldsymbol{\theta}}) - \frac{N}{2}({\boldsymbol{\theta}} - \hat{{\boldsymbol{\theta}}})^\top \hat{\boldsymbol{F}}_{\hat{{\boldsymbol{\theta}}}}  ({\boldsymbol{\theta}} - \hat{{\boldsymbol{\theta}}}) - \frac{\lambda}{2}||{\boldsymbol{\theta}} - \hat{{\boldsymbol{\theta}}}||^2_2.
\end{equation}

\section{Additional Results} \label{sec:supp_extra_results}

\subsection{Sample Efficiency of Cause Identification}
Fig.~\ref{fig:sample_efficiency} compares the sample efficiency of different approaches for cause identification on MNIST and CIFAR-10 datasets with annotation noise. More specifically, we want to understand how many failure cases in the query set $\mathcal{F}_q$ we need to see for cause identification to work, i.e., how performance of each approach varies as we reduce
the size of the query set. Here, all detrimental examples i.e., $\tilde{r}(\boldsymbol{z})<0$ are removed from the training data and the corresponding metrics are measured.  EWC-influence approach performs the best in terms of precision, recall, and accuracy on holdout failure set $\mathcal{F}_h$ even as we decrease the size of the query set. All approaches exhibit a stable behavior w.r.t the size of query set. The biggest drop occurs in accuracy on the remaining test set $\mathcal{T}\setminus\mathcal{F}$: EWC-influence presents a stronger degradation in small sample regimes (around 5\% of the original failure set) but without dropping below other approaches.

For the failure accuracy on MNIST, even when the query set is as small as 5\% of the original one ($\approx 20$ examples), all approaches display improvement over the semi-oracle that is trained after removing all instances of annotation errors: this means that all methods, in a sample-efficient manner, not only correct the synthetically added annotation errors, but they also remove other harmful examples that are naturally present to begin with.

\begin{figure}[h!]
\centering
  \includegraphics[width=0.95\linewidth]{figures/labelnoise_sampleefficiency_3.pdf}
  \caption{\footnotesize
  Comparison of sample efficiency for cause identification performance on (top) MNIST and (bottom) CIFAR-10 datasets with class-dependent annotation noise. On the x-axis of each sub-figure, we vary the size of the query failure set $\mathcal{F}_q$ used for cause identification. From left to right, (a) precision, (b) recall, (c) accuracy on holdout failure set $\mathcal{F}_h$, and (d) accuracy on the remaining test set.
  }
  \label{fig:sample_efficiency}
\end{figure}

\clearpage
\subsection{Speed Comparison}
Table~\ref{tab:computation} shows the total computation time of the proposed EWC-influence and other baselines for cause identification on a single Tesla K80 GPU with 12GB of RAM. Note that we rely on the publicly available implementation of SGD-influence\footnote{\url{https://github.com/sato9hara/sgd-influence}}, and that we have implemented our own version of linear influence functions in Pytorch.
Overall, EWC-Influence is either as fast or faster as other baselines, achieving one order of magnitude speed boost compared to SGD-influence in all cases. For CIFAR10 where a considerably larger base model is used, EWC-influence is consistently faster than other approaches, around twice faster than linear influence methods. For MNIST, EWC-influence attains similar computation time as linear influence approaches.

\begin{table}[ht!]
	\caption {Comparison of total computation time for cause identification.}\label{tab:computation}
 	\label{tab:speed_comparison}
    \center
	\begin{tabular}{|l|c|c|c|}
		\hline
		\multicolumn{1}{|c}{\textbf{Experiment}} &  \multicolumn{1}{|c|}{\textbf{Method}} & \multicolumn{1}{c|}{\textbf{Time (s)}} & \multicolumn{1}{c|}{\textbf{Time (s)}} \\
		&     & ( MNIST ) & (CIFAR10)  \\
		\hline
	            & Linear Influence (SA) & 16.4 & 1322.4\\
        Label Noise & Linear Influence (GD) & 11.2 & 996.2 \\
	              & SGD-Influence         & 185.1 & 8301.1 \\
	                      & EWC-Influence (Ours)  & \textbf{9.8 }& \textbf{496.3 }\\
	   \hline
                              & Linear Influence (SA) & 10.8   & 1141.1  \\
              Random          & Linear Influence (GD) &  15.7  & 978.4 \\
	        Input Noise  & SGD-Influence              & 188.2    & 8285.6 \\
	                   & EWC-Influence (\textbf{Ours})& 11.0    & \textbf{527.5}\\
	    \hline
                           &  Linear Influence (SA) & 14.3   & 1312.0  \\
        Adversarial       & Linear Influence (GD)   & \textbf{10.0  } & 984.2 \\
            Poisoning  & SGD-Influence              &  177.29  & 7803.7\\
                              & EWC-Influence (\textbf{Ours})  & 14.5 & \textbf{528.9} \\
        \hline
	\end{tabular}
	\vspace{-2mm}
\end{table}

\subsection{Qualitative Results}

\paragraph{Annotation noise.} In the main text, Fig.~\ref{fig:labelnose_detection}(d) shows examples with annotation noise ranked as most harmful according to EWC-influence. Similarly, Fig.~\ref{fig:label_noise_least_harmful} shows examples ranked as least harmful according to EWC-influence. All of these examples correspond to non-corrupted examples with correct labels.

\begin{figure}[ht!]
\centering
  \includegraphics[width=0.95\linewidth]{figures/labelnoise_least_harmful_examples.pdf}
  \caption{\footnotesize Examples of 16 \textbf{least} harmful examples for MNIST and CIFAR-10 with annotation noise as ranked by EWC-influence. None of the selected examples were corrupted with annotation noise.}
\label{fig:label_noise_least_harmful}
\end{figure}

\clearpage
\paragraph{Random input noise.} Fig.~\ref{fig:input_not_harmful} shows examples that are ranked highest (most harmful) by EWC-influence for datasets contaminated with random input noise. Recall that the target classes of the input noise are 1, 6, 7, 9 for MNIST and plane, bird, cat, dog for CIFAR10, and the rest of the images are free of such noise. First of all, we observe that the most detrimental examples belong to the non-target classes. As shown in the main text, while the input noise itself may not harm the performance of the model by much, the sample size of clean images in the target classes is still smaller as a result of noise injection---such group imbalance can be rectified by sub-sampling the dominant group, for example, by removing those identified detrimental data points. Secondly, many of them appear to be ambiguous instances in non-target classes. It is worth noting that for MNIST, one of the identified examples is interestingly a real instance of 3 that is incorrectly labelled as 5, which is also reported by Northcutt \textit{et al.} \cite{northcutt2021confidentlearning}.

\begin{figure}[h!]
\centering
  \includegraphics[width=0.95\linewidth]{figures/randominputnoise_most_harmful_examples.pdf}
  \caption{\footnotesize Examples of 16 most harmful examples for MNIST and CIFAR-10 with random input noise as ranked by EWC-influence. None of the selected examples were corrupted with random input noise.
  }
  \label{fig:input_not_harmful}
\end{figure}

\subsection{Treating Models by Forgetting Detrimental Past and Learning from Present Mistakes}
While the primary goal of this work is to investigate how many prediction errors can be remedied by identifying harmful training data and removing them, one could alternatively use the labelled failure cases directly to adapt the model. Fig.~\ref{fig:treatment_comparison_with_finetuning} shows our preliminary results where we compare the deletion based methods to an approach that fine-tunes the model directly on the failure query set $\mathcal{F}_{q}$ with an L2-norm based locality constraint $||{\boldsymbol{\theta}} - \hat{{\boldsymbol{\theta}}}||^2_2$ \cite{zhu2020modifyingmemories} and its combination with the best deletion-based approach, that is, fine-tuning on the corrected dataset $\mathcal{D}\setminus \mathcal{C}$. As with other experiments, early stopping is performed based on the loss on a portion of the query set $\mathcal{F}_{q}$. We see that while fine-tuning on $\mathcal{F}_{q}$ (black points) leads to a higher accuracy (on the holdout failure set $\mathcal{F}_{h}$) than fine-tuning on $\mathcal{D}\setminus \mathcal{C}$ (yellow points), the accuracy on the remaining test set $\mathcal{T}\setminus \mathcal{F}$ is worse, even with the weight constraint~---~by 6\% on MNIST and 21\% on CIFAR-10. This issue of over-fitting to the failure cases is also reported in recent works such as \cite{cao2021editing,sinitsin2020editable}. Importantly, by combining the two approaches (brown points), we can attain the best trade-off between the failure and the maintenance accuracy, indicating the complementarity between the proposed data correction methods and such fine-tuning approaches.

\begin{figure}[ht!]
\centering
  \vspace{-3mm}
  \includegraphics[width=0.75\linewidth]{figures/repairment_method_comparison_v2.pdf}
  \vspace{-3mm}
  \caption{\footnotesize Comparison of deletion-based treatment methods and the direct fine-tuning on the failures on (a) MNIST and (b) CIFAR-10 datasets with noisy annotations.}
  \label{fig:treatment_comparison_with_finetuning}
\end{figure}

\newpage
\section{Experimental Details} \label{sec:supp_experimental_details}

\paragraph{Datasets}
 we perform our experiments on the MNIST digit classification task and the CIFAR-10 object recognition task.  The MNIST dataset consists of $60,000$ training and $10,000$ testing examples, all of which are $28\times28$ grayscale images of digits from $0$ to $9$ ($10$ classes). The CIFAR-10 dataset consists of $50,000$ training and $10,000$ testing examples, all of which are $32\times32$ coloured natural images drawn from $10$ classes. Both datasets are preprocessed by subtracting the mean, but no data augmentation is used. For MNIST, to make the task more challenging, we randomly select 3000 examples from the training split and train the base models while the entire test set is used for evaluation.

\paragraph{Architecture Details}
For MNIST, the base classifier was defined as a CNN architecture comprised of $4$ convolution layers, each with $3\times3$ kernels follower by Relu. The number of kernels in respective layers are $\{32, 32, 64, 64\}$. After the first two convolution layers, we perform $2\times2$ max-pooling, and after the last one, we further down-sample the features with Global Average Pooling (GAP) prior to the final fully connected layer. For CIFAR-10, we used a 50-layer ResNet \cite{he2016deep}.

\paragraph{Optimisation}
For all experiments, we employ the same training scheme unless otherwise stated. We optimize parameters using Adam \cite{kingma2014adam} with initial learning rate of $10^{-3}$ and $\beta = [0.9, 0.999]$, with minibatches of size $64$ and train for max $100$ epochs with early stopping with a patience of 5, that is, training is stopped after 5 epochs of
no progress on the validation set (10\% of the training set). For computing both EWC-influence and EWC-deletion, we also employed the same training scheme but applied early stopping based on the performance on a validation split (10\%) of the failure query set $\mathcal{F}_{q}$.

In Fig.~\ref{fig:deletion_comparison} in Sec.~\ref{sec:results_treatment}, we present the performance of Newton update removal and EWC-deletion (ours) with different hyper-parameter settings. For Newton-update deletion \cite{guo2019certified}, we vary the step size $\gamma>0$ of the gradient ascent by scaling the second term in Eq.~\eqref{eq:koh_data_deletion}. For EWC-deletion (our method), we vary the amount of weight regularisation --- the second term in Eq.~\eqref{eq:deletion_ewc} --- by scaling it by $2/\gamma N$ where $N$ denotes the number of training datapoints, and $\gamma>0$. We also set the strength of the prior term to $\lambda=0$. We run both methods for different values of $\gamma$ in the range $[0.01, 0.05]$.
\section{Introduction}
\vspace{-1mm}
Machine learning (ML) models often exhibit unexpected failures once deployed in the "wild". Recent lines of research aim to  alleviate different well-known shortcomings in supervised models, such as vulnerability to annotation errors  \citep{frenay2013classification} and adversarial attacks \cite{xu2020adversarial}, sensitivity to data shifts \citep{koh2021wilds}, and biases to underrepresented subgroups \citep{torralba2011unbiased,chen2018my,sagawa2019distributionally}. However, it is often challenging to anticipate beforehand all plausible failure scenarios, and protect against them pre-emptively. This motivates developing a technique that is able to \textit{repair a model on demand}, as new failure cases arise in practice.

Undesirable behaviours of ML models commonly stem from defects in the training data. However, it is unclear how to detect the causes of such failures automatically, rendering a manual troubleshooting necessary. Furthermore, once the problems are uncovered, one would still need to design fixes, which typically involve further data curation/collection, and model retraining/redesigning from scratch. Executing the above steps demand not only time, but also mature expertise in the relevant ML areas, a scarcity in the present job market.

This work introduces an approach to identifying a set of most detrimental training examples that have caused failure cases observed at test time, and to subsequently repairing the model on these failures by deleting those culprits. At the basis of both cause identification and repairment steps is the approximation of \emph{``counterfactual'' posterior distribution} where some training examples are assumed absent. We formalise this as a Bayesian \emph{continual (un)learning} problem \citep{nguyen2017variational}, where the above counterfactual posterior is estimated by deleting the evidence of selected training data from the current posterior. We note that, while other factors (e.g. model class and optimisation) may play a role, we focus on ``data debugging'' and investigate, to what extent, prediction failures could be remedied by only intervening on data and updating the model accordingly.

\begin{figure}[ht]
\centering
  \includegraphics[width=\linewidth]{figures/overview_3.pdf}
  \vspace{-3mm}
  \caption{\footnotesize (a) Real-world datasets are often fraught with issues such as annotation noise, low-quality inputs, anomalies, and acquisition biases (e.g. demographic imbalances). Such issues may lead to undesirable performance of the trained models in deployment. Our approach aims at repairing such models by (b) identifying detrimental training examples which have caused the target failures, and then (c) erasing efficiently the \textit{memories} of those examples from the models.}
  \label{fig:overview}
\end{figure}

Fig.~\ref{fig:overview} gives an overview of the proposed approach for model repairment, which operates in two steps by 1) identifying causes of failure among training data, and 2) updating the model by erasing the ``memories'' of those harmful examples. Importantly, the proposed framework is agnostic to \emph{why} a particular datapoint hurts model performance, handling both, issues in the input data and/or labels. We only require specification of the set of failed cases for which we wish to improve performance.

\textbf{Our contributions:}
We develop a framework for \textit{repairing} machine learning models by erasing memories of detrimental datapoints. The framework connects both identification and removal of detrimental data under a (Bayesian) continual learning perspective, which brings forth practical benefits. Firstly, the framework subsumes works on influence function \cite{koh2017understanding} and data deletion \cite{guo2019certified} as specific examples, which are developed independently, and our work reveals their close connections and limitations. Secondly, the generality of our formulation allows translating any continual learning method into this model repairment setting, and opens doors to further research. In particular, we extend Elastic Weight Consolidation \citep{kirkpatrick2017overcoming} -- a specific continual learning algorithm -- to cause identification and data removal, and demonstrate improvements over the prior works in a variety of settings where training data are contaminated with annotation and/or input noise.
\vspace{-1mm}

\section{Model Repairment by Data Deletion} \label{sec:method}
Let us consider a prediction model $p(y \,|\, {\boldsymbol{\theta}}, \boldsymbol{x})$ that returns a probability distribution of the output $y$ given an input $\boldsymbol{x}$.
We make the i.i.d.~modelling assumption and denote $p(\bm{\theta} \,|\, \mathcal{D})$ as the posterior distribution over the model parameters $\bm{\theta}$ given training data $\mathcal{D}=\{(\boldsymbol{x}^{(n)}, y^{(n)})\}_{n=1}^N$.
At test time, the posterior predictive distribution is used to infer label $y^\star$ given a new sample $\boldsymbol{x}^\star$:
\begin{equation}\label{eq:predictive_inference}
    p(y^\star|\bm{x}^\star, \mathcal{D}) = \int p(y^\star \,|\, \bm{x}^\star, \bm{\theta}) p(\bm{\theta} \,|\, \mathcal{D}) d\bm{\theta},
\end{equation}
where $p(y^\star |\bm{x}^\star, \bm{\theta})$ is the likelihood term for sample $(x^\star, y^\star)$.
While we use approximate posteriors in practice, we focus on the exact case for now to formalise the problem.

Imagine that, in deployment, this model makes incorrect predictions in certain situations. After collecting a set of examples from such failure mode in the test set\footnote{There may be multiple different ways in which the model fails, and the failure cases may consist of several groups. In practice, one type of mistake might incur more costs than others (e.g. in certain medical applications, \textit{false negative} is more costly than \textit{false positive}). Here we assume that we have identified at least one failure type that we would like to fix.},
$\mathcal{F} = \{(\boldsymbol{x}_{f}^{(n)}, y_{f}^{(n)})\}_{n=1}^{N_f}$, we would like to \textit{repair} the model, such that it improves performance on the failure set $\mathcal{F}$ and similar future cases. We also argue that a successful repairment should also \textit{maintain} a similar level of performance on the rest of test examples.
These two objectives of model repairment are analogous to those of a \textit{medical treatment}, in that both aim to fix a specific problem while leaving the ``healthy'' part as intact as possible. A further discussion of different aspects, including generalisation, efficiency and specificity properties is provided in Appendix \ref{app:problem_formulation}.

In this work, we assume that the main reason for such failures $\mathcal{F}$ is due to the existence of detrimental examples in the training data $\mathcal{D}$, for example, noisy labels, low-quality inputs and/or group imbalances. Our hypothesis is that, by removing these harmful datapoints and adapting the model accordingly, the model can be repaired to return correct predictions for datapoints in $\mathcal{F}$ as well as similar future cases. We acknowledge that there might be other reasons for a model to make wrong predictions on $\mathcal{F}$, such as optimisation and model biases, which are outside the scope of this work. Addressing data-based failures is complementary to accounting for other problems and is of relevance regardless, as datasets typically come with unexpected issues no matter how much we curate them beforehand.

We formulate the process of \textbf{model repairment} as the following two steps (see Fig.~\ref{fig:overview} for an illustration):
\begin{enumerate}

    \item \textbf{Cause identification:} Identify a set of detrimental datapoints $\mathcal{C}$ in the training data $\mathcal{D}$ that contributed the most to the failure set $\mathcal{F}$.

    \item \textbf{Treatment: } Given the set of failure causes $\mathcal{C}$, adapt the model to predict correctly on the failure set $\mathcal{F}$, while maintaining performance on remaining test examples.
\end{enumerate}
Below we describe a formal framework for performing these two steps. In the first phase of cause identification, we need to define a measure of how much a subset of training examples $\mathcal{C} = \{(\boldsymbol{x}_{c}^{(n)}, y_{c}^{(n)})\}_{n=1}^{N_c}\subset \mathcal{D}$ is responsible for the failure cases $\mathcal{F}$. To this end, we propose to check how much the posterior predictive distribution on $\mathcal{F}$ changes as a result of deleting $\mathcal{C}$ from the training data:
\vspace{0mm}
\begin{equation}
r(\mathcal{C}) \coloneqq \log \, p(\mathcal{F} \,|\, \mathcal{D}\setminus \mathcal{C}) - \log \, p(\mathcal{F} \,|\, \mathcal{D}), \label{Eq.rC}
\end{equation}
where $p(\mathcal{F} \,|\, \mathcal{D})$ and $p(\mathcal{F} \,|\, \mathcal{D}\setminus \mathcal{C})$ are the posterior predictive distributions before and after removing a subset of training examples $\mathcal{C}$.
If a given $\mathcal{C}$ leads to a large value of $r(\mathcal{C})$ in Eq.~\ref{Eq.rC}, it would mean that removing $\mathcal{C}$ from $\mathcal{D}$ would have improved the performance of the Bayesian predictive inference on the failure set $\mathcal{F}$. The first \textit{causal identification} step thus entails finding a subset $\mathcal{C}$ with the maximal log-density ratio $r(\mathcal{C})$. In the second \textit{treatment} step, we can directly adopt $p(\mathcal{F} \,|\, \mathcal{D}\setminus \mathcal{C})$ as the updated predictive distribution that needs to be inferred, as it confers the largest improvement over the failure cases. In other words, for both the search of detrimental datapoints and repairment of the model, we need to compute $p(\mathcal{F} \,|\,  \mathcal{D}\setminus \mathcal{C})$.

The central computational question is, therefore, concerned with the \textit{efficient calculation of $p(\mathcal{F} \,|\, \mathcal{D}\setminus \mathcal{C})$ without retraining the model from scratch}. The reasons for avoiding retraining are: (1) in the cause identification step, it is computationally prohibitive as retraining needs to be done for every subset $\mathcal{C} \subset \mathcal{D}$; and (2) in the treatment step, the resulting retrained model may be drastically different from the original model one would like to fix (due to, e.g.~noises in the SGD-based optimisation and parameter re-initialisation), and may lose other desirable properties that one may wish to retain.
In this work, we thus argue for continual learning approaches \cite{parisi2019continual} to approximate $p(\mathcal{F} \,|\, \mathcal{D}\setminus \mathcal{C})$, given their computational efficiency and their natural ability to maintain model behaviour. We now elaborate on the mathematical details of the two steps.

\vspace{-1mm}
\subsection{Step I: Cause Identification}\label{sec:cause_identification}
\vspace{-1mm}

Identifying the set of detrimental examples $\mathcal{C}$
requires solving the following optimisation problem
\begin{equation}
\mathcal{C} = \operatorname{argmax}_{\mathcal{C}' \in \mathbb{P}(\mathcal{D})}  r(\mathcal{C}'),
\end{equation}
where $\mathbb{P}(\mathcal{D})$ denotes the power set of $\mathcal{D}$. Solving this comes with multiple computational challenges.

Firstly, a naive approach would require computing the predictive distribution $p(\mathcal{F} \,|\, \mathcal{D}\setminus \mathcal{C})$ --- and thus, the posterior $p({\boldsymbol{\theta}} \,|\, \mathcal{D} \setminus \mathcal{C})$ --- for every subset $\mathcal{C}$ of $\mathcal{D}$, which is prohibitively expensive. In the following, we present a \textbf{``predictive'' approach} that removes this computational burden.
The key idea is to notice that $p(\mathcal{D} \setminus \mathcal{C} \,|\, {\boldsymbol{\theta}}) = p(\mathcal{D} \,|\, {\boldsymbol{\theta}}) / p(\mathcal{C} \,|\, {\boldsymbol{\theta}})$ for any $\mathcal{C} \subset \mathcal{D}$ due to the i.i.d.~modelling assumption. Inserting this into the Bayes' rule for computing $p({\boldsymbol{\theta}}|\mathcal{D} \setminus \mathcal{C})$ yields:
\begin{equation*}
\begin{aligned}
\log p(\mathcal{F} \,|\, \mathcal{D} \setminus \mathcal{C}) &= \log \int p(\mathcal{F} \,|\, {\boldsymbol{\theta}}) p({\boldsymbol{\theta}} \,|\, \mathcal{D} \setminus \mathcal{C}) d{\boldsymbol{\theta}}
= \log \int \frac{p(\mathcal{F} \,|\, {\boldsymbol{\theta}})}{p(\mathcal{C}\vert {\boldsymbol{\theta}})} \frac{p(\mathcal{D} |{\boldsymbol{\theta}}) p({\boldsymbol{\theta}})}{p(\mathcal{D})} d{\boldsymbol{\theta}} - \log \frac{p(\mathcal{D}\setminus \mathcal{C})}{p(\mathcal{D})}.
\end{aligned}
\end{equation*}
Notice that $p(\mathcal{D}\setminus \mathcal{C}) / p(\mathcal{D}) = \int \frac{1}{p(\mathcal{C} \,|\, {\boldsymbol{\theta}})} \frac{p(\mathcal{D} \,|\, {\boldsymbol{\theta}}) p({\boldsymbol{\theta}})}{p(\mathcal{D})} d{\boldsymbol{\theta}}$ again due to the i.i.d.~assumption.
Therefore, we can compute the log density ratio $r(\mathcal{C}) = \log \,  p(\mathcal{F} \,|\, \mathcal{D}\setminus \mathcal{C}) - \log \, p(\mathcal{F} \,|\, \mathcal{D})$ as follows without computing $p({\boldsymbol{\theta}} \,|\, \mathcal{D} \setminus \mathcal{C})$:
\begin{equation} \label{eq:predictive_log_density_ratio}
r(\mathcal{C})= \log \int \frac{ p({\boldsymbol{\theta}} \,|\, \mathcal{D}, \mathcal{F})}{p(\mathcal{C} \,|\, {\boldsymbol{\theta}})} d{\boldsymbol{\theta}} - \log \int \frac{p({\boldsymbol{\theta}} \,|\, \mathcal{D})}{p(\mathcal{C} \,|\, {\boldsymbol{\theta}})}  d{\boldsymbol{\theta}}.
\end{equation}
In this form, we would only need to compute the posterior $p({\boldsymbol{\theta}}| \mathcal{D}, \mathcal{F})$ once, and can side-step the requirement of computing $p({\boldsymbol{\theta}}|\mathcal{D} \setminus \mathcal{C})$ for inspection of every new $\mathcal{C} \subset \mathcal{D}$, thereby removing one of the key computational bottlenecks. The detailed derivation of Eq.~\eqref{eq:predictive_log_density_ratio} is provided in Sec.~\ref{sec:supp_derivations} in the appendix.

Secondly, we are still left with the combinatorial search for the best subset $\mathcal{C}$, which is also prohibitive when the size of training data $\mathcal{D}$ is large. We address this issue by a first-order Taylor series approximation of $r(\mathcal{C})$ that factorises over the samples in $\mathcal{C}$. Let us re-write the log density ratio as
\begin{align}
    r(\mathcal{C}) = F\left(1,  p\left({\boldsymbol{\theta}} \,|\, \mathcal{D}, \mathcal{F}\right)\right) - F\left(1,  p\left({\boldsymbol{\theta}} \,|\, \mathcal{D}\right)\right),\label{Eq:rCF} \quad
    F(\epsilon, g({\boldsymbol{\theta}})) \coloneqq   \log \int  g({\boldsymbol{\theta}}) e^{-\epsilon \log p(\mathcal{C} \,|\, {\boldsymbol{\theta}})} d{\boldsymbol{\theta}}
\end{align}
Note that $F(0, g({\boldsymbol{\theta}})) = 0$ for any well-defined distribution $g({\boldsymbol{\theta}})$. Assuming $g({\boldsymbol{\theta}})$ is a distribution, we perform a Taylor expansion of $F(\epsilon, g({\boldsymbol{\theta}}))$ around $\epsilon = 0$:
$F(\epsilon, g({\boldsymbol{\theta}})) = - \epsilon \, \mathbb{E}_{ g({\boldsymbol{\theta}})} \left[\log p(\mathcal{C} \,|\, {\boldsymbol{\theta}}) \right] + \mathcal{O}(\epsilon^2)$.
This results in an approximate log density ratio $\hat{r}(\mathcal{C}) \approx r(\mathcal{C})$ by plugging the Taylor expansion into Eq.~(\ref{Eq:rCF}):
\begin{equation}\label{eq:log_density_approximation}
\hat{r}(\mathcal{C}) \coloneqq \mathbb{E}_{p({\boldsymbol{\theta}} \,|\, \mathcal{D})}\left[\log p(\mathcal{C}\,|\, {\boldsymbol{\theta}}) \right] - \mathbb{E}_{p({\boldsymbol{\theta}} \,|\, \mathcal{D}, \mathcal{F})}\left[\log p(\mathcal{C}\,|\, {\boldsymbol{\theta}}) \right].
\end{equation}
Assuming that data are i.i.d., and defining $\boldsymbol{z} = (\boldsymbol{x}, y)$, the above approximation can be expressed as the sum of individual log density ratios, $\hat{r}(\mathcal{C}) =  \sum_{\boldsymbol{z}\in\mathcal{C}}\hat{r}(\boldsymbol{z})$, where each term is given by
\begin{equation*}
    \begin{aligned}
    \hat{r}(\boldsymbol{z})= \mathbb{E}_{p({\boldsymbol{\theta}} \,|\, \mathcal{D})}\left[\log p(\boldsymbol{z} \,|\, {\boldsymbol{\theta}}) \right] - \mathbb{E}_{p({\boldsymbol{\theta}} \,|\, \mathcal{D}, \mathcal{F})}\left[\log p(\boldsymbol{z} \,|\, {\boldsymbol{\theta}}) \right], \quad p(\boldsymbol{z} \,|\, {\boldsymbol{\theta}}) = p(y \,|\, \boldsymbol{x}, {\boldsymbol{\theta}}).
    \end{aligned}
\end{equation*}
With this approximation, in order to find a subset $\mathcal{C}$ that leads to the maximal $\hat{r}(\mathcal{C})$, it suffices to compute $\hat{r}(\boldsymbol{z})$ for every training example $\boldsymbol{z} \in \mathcal{D}$ and find the top $K$ examples with largest $\hat{r}(\boldsymbol{z})$ values, thereby reducing the search space from $\mathcal{O}(|\mathcal{D}|!)$ choices to only $\mathcal{O}(|\mathcal{D}|)$ choices.

In practice, the causes of failure will correspond to examples $\boldsymbol{z}$ with $\hat{r}(\boldsymbol{z}) > 0$. Intuitively, $\hat{r}(\boldsymbol{z})$ measures how much the posterior predictive moments of $\boldsymbol{z}$ changes when the model is further trained on $\mathcal{F}$, i.e., computation of $p({\boldsymbol{\theta}} \,|\, \mathcal{D}, \mathcal{F})$. If the moment difference is positive, i.e., $\hat{r}(\boldsymbol{z}) > 0$, it means that the example $\boldsymbol{z} \in \mathcal{D}$ is a conflicting evidence against the test examples in $\mathcal{F}$; conversely, if $\hat{r}(\boldsymbol{z}) < 0$, then $\boldsymbol{z}$ and $\mathcal{F}$ are aligned. See Algorithm~\ref{alg:cause_identification} for specification of the whole procedure.

\begin{algorithm}[t]
   \caption{\footnotesize Model Repairment}
   \label{alg:cause_identification}
\begin{algorithmic}
    \footnotesize
   \STATE {\bfseries Input:} training data $\mathcal{D}$; failure cases $\mathcal{F}$; approximate posterior $q({\boldsymbol{\theta}}) \approx  p({\boldsymbol{\theta}} \,|\, \mathcal{D})$; likelihood $p(\boldsymbol{z} \,|\, {\boldsymbol{\theta}})$
   \STATE {\bfseries Output:} failure causes $\mathcal{C}$, ``repaired'' posterior $q^{\star\star}_{\mathcal{D} \setminus \mathcal{C}}({\boldsymbol{\theta}})$
  \STATE
   \STATE {\bfseries \footnotesize \textcolor{blue}{\texttt{\# Step I: Cause Identification}}}
   \STATE {\bfseries Update posterior:} Apply a \textit{continual learning} method to obtain $q^{\star}_{\mathcal{D},\mathcal{F}}({\boldsymbol{\theta}}) \approx p({\boldsymbol{\theta}} \vert \mathcal{D}, \mathcal{F})$ by fitting the failure set $\mathcal{F}$

   \STATE {\bfseries Compute influences of training examples on $\mathcal{F}$:} Calculate $\tilde{r}(\boldsymbol{z})\,\, \forall \boldsymbol{z} \in \mathcal{D}$ (Eq.~\eqref{eq:final_influence_score})

   \STATE {\bfseries Find failure causes $\mathcal{C}$:} Return the examples with positive influence, $\mathcal{C} \leftarrow \{\boldsymbol{z} \in \mathcal{D}$: $\tilde{r}(\boldsymbol{z}) > 0\}$
  \STATE
   \STATE {\bfseries \footnotesize \textcolor{red}{\texttt{\# Step II: Treatment}}}
   \STATE {\bfseries Delete information of $\mathcal{C}$}: Apply a \textit{continual (un)learning} method to the original posterior $q({\boldsymbol{\theta}})$, and obtain the posterior on the corrected data $q^{\star\star}_{\mathcal{D} \setminus \mathcal{C}}({\boldsymbol{\theta}}) \approx p({\boldsymbol{\theta}} \vert \mathcal{D}\setminus \mathcal{C})$
\end{algorithmic}
\end{algorithm}
\vspace{-3mm}

Lastly, for non-linear models (e.g.~neural networks), approximate posteriors $q({\boldsymbol{\theta}}) \approx  p({\boldsymbol{\theta}} \,|\, \mathcal{D})$ and $q^{\star}_{\mathcal{D},\mathcal{F}}({\boldsymbol{\theta}}) \approx p({\boldsymbol{\theta}} \,|\, \mathcal{D}, \mathcal{F})$ are needed due to intractability of the exact posteriors. We assume that $q({\boldsymbol{\theta}})$ is available after training, and suffers from the prediction failures $\mathcal{F}$. As recomputing the posterior $q^{\star}_{\mathcal{D},\mathcal{F}}({\boldsymbol{\theta}})$ from scratch can be expensive, we propose to use a \textit{continual learning} technique \cite{parisi2019continual} and obtain this quantity by updating the original posterior $q({\boldsymbol{\theta}})$. Finally, we use the following metric  $\tilde{r}(\boldsymbol{z})$ in practice to calculate the detrimental impact of each training datapoint on the failure set, $\mathcal{F}$:
\begin{equation}\label{eq:final_influence_score}
    \tilde{r}(\boldsymbol{z}):= \mathbb{E}_{q({\boldsymbol{\theta}})}\left[\log \, p(\boldsymbol{z}\vert {\boldsymbol{\theta}}) \right] - \mathbb{E}_{q^{\star}_{\mathcal{D},\mathcal{F}}({\boldsymbol{\theta}}) }\left[\log \, p(\boldsymbol{z}\vert {\boldsymbol{\theta}}) \right]
\end{equation}
and approximate the failure causes $\mathcal{C}$. This metric is generic, and the detail implementation depends on the specifics in which both $q({\boldsymbol{\theta}})$ and $q^{\star}_{\mathcal{D},\mathcal{F}}({\boldsymbol{\theta}})$ are computed, e.g. MLE/MAP point estimates, Laplace approximation, variational inference, etc. We end by providing two concrete examples: the first shows that the well-known \textit{linear influence function} \cite{koh2017understanding} is a specific instance of Eq.~\eqref{eq:final_influence_score}; and the second is derived by extending a continual learning method, known as \textit{Elastic Weight Consolidation} (EWC) \cite{kirkpatrick2017overcoming} to cause identification, and is a key methodological development in our work.

\textbf{\textit{Example 1 (Linear Influence Function):}}
Our proposed metric in Eq.~\eqref{eq:final_influence_score} recovers the \emph{linear influence function} from Koh \& Liang \citep{koh2017understanding} when point estimates are used for ${\boldsymbol{\theta}}$. Assume that the model is trained on data $\mathcal{D}$ with parameters $\hat{{\boldsymbol{\theta}}}$, which corresponds to an approximation of MLE/MAP estimates, i.e., $q({\boldsymbol{\theta}}) = \delta({\boldsymbol{\theta}} - \hat{{\boldsymbol{\theta}}}) \approx p({\boldsymbol{\theta}} \,|\, \mathcal{D})$. After observing the set of failures $\mathcal{F}$, a point estimate of $p({\boldsymbol{\theta}} \,|\, \mathcal{D}, \mathcal{F})$ is obtained by performing a single update of natural gradient ascent \citep{amari1998natural} on the log likelihood of $\mathcal{F}$ with step size $\gamma>0$:
\begin{equation}\label{eq:linear_influence}
\hat{{\boldsymbol{\theta}}}^{\star}_{\mathcal{D},\mathcal{F}} \approx \hat{{\boldsymbol{\theta}}} + \gamma \hat{\boldsymbol{F}}_{\hat{{\boldsymbol{\theta}}}}^{-1}\nabla_{\hat{{\boldsymbol{\theta}}}}\log p(\mathcal{F} \vert \hat{{\boldsymbol{\theta}}})
\end{equation}
where $\hat{\boldsymbol{F}}_{\hat{{\boldsymbol{\theta}}}}$ is the empirical Fisher information matrix. This means
\begin{equation} \label{eq:predictive_point_estimates}
 \tilde{r}(\boldsymbol{z}) = - \gamma \nabla_{\hat{{\boldsymbol{\theta}}}}\log p(\mathcal{F}\vert\hat{{\boldsymbol{\theta}}})^{\top} \hat{\boldsymbol{F}}_{\hat{{\boldsymbol{\theta}}}}^{-1}\nabla_{\hat{{\boldsymbol{\theta}}}}\log p(\boldsymbol{z}\vert\hat{{\boldsymbol{\theta}}}),
\end{equation}
if defining the updated posterior as $q^{\star}_{\mathcal{D},\mathcal{F}}({\boldsymbol{\theta}}) = \delta({\boldsymbol{\theta}} - \hat{{\boldsymbol{\theta}}}^{\star}_{\mathcal{D},\mathcal{F}})$.
The negation of the above equation coincides with the linear influence function (Eq.~(2) in \cite{koh2017understanding}) when the failure set is assumed to be $\mathcal{F} = \{\boldsymbol{z}^\star\}$ and $\gamma=1$.

\textit{\textbf{Example 2 (Elastic Weight Consolidation): }}
The generality of Eq.~\eqref{eq:final_influence_score} permits any continual learning method of one's choice for estimating the updated posterior $q^{\star}_{\mathcal{D},\mathcal{F}}({\boldsymbol{\theta}}) \approx p({\boldsymbol{\theta}} \,|\, \mathcal{D}, \mathcal{F})$ after observing failure samples $\mathcal{F}$. Here we illustrate how EWC
\citep{kirkpatrick2017overcoming} as a continual learning method can be adopted in the context of cause identification. EWC approximates $ p({\boldsymbol{\theta}} \,|\, \mathcal{D}, \mathcal{F})$  by first performing the Laplace approximation of the original posterior $p({\boldsymbol{\theta}} \,|\, \mathcal{D})$ around the point estimate $\hat{{\boldsymbol{\theta}}}$, and subsequently finding the MAP solution of ${\boldsymbol{\theta}}$. Formally, $\hat{{\boldsymbol{\theta}}}^{\star}_{\mathcal{D},\mathcal{F}}$ is obtained by maximising the objective below w.r.t. ${\boldsymbol{\theta}}$ via SGD (see Appendix \ref{app:ewc_identification_objective} for details):
\begin{equation}\label{eq:predictive_ewc}
\log \, p(\mathcal{F} \vert{\boldsymbol{\theta}})  - \frac{N}{2}({\boldsymbol{\theta}} - \hat{{\boldsymbol{\theta}}})^\top  \hat{\boldsymbol{F}}_{\hat{{\boldsymbol{\theta}}}} ({\boldsymbol{\theta}} - \hat{{\boldsymbol{\theta}}}) - \frac{\lambda}{2}||{\boldsymbol{\theta}} - \hat{{\boldsymbol{\theta}}}||^2_2,
\end{equation}
where the off-diagonal elements of $\hat{\boldsymbol{F}}_{\hat{{\boldsymbol{\theta}}}}$ are dropped for memory reason in practice. Defining $q({\boldsymbol{\theta}}) = \delta({\boldsymbol{\theta}} - \hat{{\boldsymbol{\theta}}})$ and $q^{\star}_{\mathcal{D},\mathcal{F}}({\boldsymbol{\theta}}) = \delta({\boldsymbol{\theta}} - \hat{{\boldsymbol{\theta}}}^{\star}_{\mathcal{D},\mathcal{F}})$, we have that
\begin{equation}\label{eq:ewc_influence_function}
 \tilde{r}(\boldsymbol{z}) = \log p(\boldsymbol{z} \,|\, \hat{{\boldsymbol{\theta}}}) - \log p(\boldsymbol{z} \,|\, \hat{{\boldsymbol{\theta}}}^{\star}_{\mathcal{D},\mathcal{F}}).
\end{equation}
To compute the above for each datapoint $\boldsymbol{z} \in \mathcal{D}$, we only need to solve the optimisation problem of Eq.~(\ref{eq:predictive_ewc}) by SGD once. We refer to this version of $\tilde{r}(\boldsymbol{z})$ as \textit{EWC-influence function}.

\textbf{Comparison:}
The EWC-influence function generalises the linear influence approach. To see this, we derive the fixed point of the EWC objective Eq.~(\ref{eq:predictive_ewc}) w.r.t.~${\boldsymbol{\theta}}$ (where we set $\lambda = 0$):
\begin{equation}
    {\boldsymbol{\theta}} = \hat{{\boldsymbol{\theta}}} + N^{-1}
    \hat{\boldsymbol{F}}_{\hat{{\boldsymbol{\theta}}}}^{-1} \nabla_{{\boldsymbol{\theta}}} \log p(\mathcal{F} \vert{\boldsymbol{\theta}}).
\end{equation}
Then the update in Eq.~(\ref{eq:linear_influence}) that is implicitly used by linear influence function can be viewed as (damped) one-step fixed-point iteration update initialised at $\hat{{\boldsymbol{\theta}}}$ for solving the fixed-point equation. As EWC-influence update (Eq.~(\ref{eq:ewc_influence_function})) is obtained by using the optimum of Eq.~(\ref{eq:predictive_ewc}), it is arguably more accurate than linear influence function (Eq.~(\ref{eq:predictive_point_estimates})) for measuring the (detrimental) effect of a datum $\bm{z}$ to the model failures $\mathcal{F}$.

\subsection{Step II: Treatment} \label{sec:treatment}
\vspace{-1mm}
Once the causes $\mathcal{C}$ of the failures $\mathcal{F}$ are identified among the training data $\mathcal{D}$, we seek to repair the model $q(\bm{\theta})$ by erasing the memories of $\mathcal{C}$. We formalise this problem as the computation of the posterior $p({\boldsymbol{\theta}} \,|\, \mathcal{D}\setminus \mathcal{C})$, i.e., $\mathcal{C}$ is absent from the training data $\mathcal{D}$.
A naive approach would re-run approximate inference on the whole ``corrected'' dataset $\mathcal{D}\setminus \mathcal{C}$ to obtain an approximate posterior $q^{\star\star}_{\mathcal{D} \setminus \mathcal{C}}(\bm{\theta}) \approx p(\bm{\theta} \,|\, \mathcal{D}\setminus \mathcal{C})$ which can be time consuming. But more fundamentally, by doing so, the obtained $q^{\star\star}_{\mathcal{D} \setminus \mathcal{C}}(\bm{\theta})$ may be unrelated to the original $q(\bm{\theta})$ based on which $\mathcal{C}$ were identified, due to, e.g.~non-convex SGD optimisation issues. Moreover, this ``model replacement'' approach may not maintain other good properties of the original model $q(\bm{\theta})$.

Analogous to cause identification (Sec.~\ref{sec:cause_identification}), we propose to employ the continual learning approach to estimate efficiently the modified posterior $p({\boldsymbol{\theta}} \,|\, \mathcal{D}\setminus \mathcal{C})$. Applying Bayes' rule and some algebraic manipulations yield
$p({\boldsymbol{\theta}} \,|\, \mathcal{D}\setminus \mathcal{C}) \propto p({\boldsymbol{\theta}} \,|\, \mathcal{D}) / p(\mathcal{C} \,|\, {\boldsymbol{\theta}})$ (see Appendix \ref{app:ewc_deletion_objective}).
Therefore the information about $\mathcal{C}$ can be removed by scaling the current posterior $p({\boldsymbol{\theta}} \,|\, \mathcal{D})$ by the inverse of $p(\mathcal{C} \,|\, {\boldsymbol{\theta}})$ and re-normalising.
In other words, we can treat the approximation of $p(\bm{\theta} \,|\, \mathcal{D}\setminus\mathcal{C})$ as a continual learning task, where the task is to ``unlearn'' the datapoints in $\mathcal{C}$ while using the posterior distribution $p(\bm{\theta} \,|\, \mathcal{D})$ as the prior.
In practice, the target model to be fixed corresponds to the approximate posterior $q(\bm{\theta}) \approx p(\bm{\theta} \,|\, \mathcal{D})$. Therefore continual (un)learning is done by
\begin{equation}\label{eq:approx_data_deletion}
    q^{\star\star}_{\mathcal{D} \setminus \mathcal{C}}(\bm{\theta}) \propto q(\bm{\theta}) / p(\mathcal{C} \,|\, {\boldsymbol{\theta}}) \approx p({\boldsymbol{\theta}} \,|\, \mathcal{D}\setminus \mathcal{C}).
\end{equation}
The above approximation can be carried out with different approximate inference techniques such as MLE/MAP point estimate, Laplace approximation \cite{kirkpatrick2017overcoming} and variational inference \cite{nguyen2017variational}. At the end of this section, we also provide a few examples to concretise this process.

We highlight the deep connection between the ``predictive approach'' for cause identification (Sec.~\ref{sec:cause_identification}), and the continual (un)learning for data deletion in the treatment step. They share the key idea of editing the posterior distribution $q(\bm{\theta})$ via continual learning. This can be seen from the factor graph interpretation of posterior distributions, where posterior editing just corresponds to editing the selected factors in the graph, e.g. insertion of $p(\mathcal{F} | \bm{\theta})$ in cause identification, and deletion of $p(\mathcal{C}| \bm{\theta})$ in treatment.

\textit{\textbf{Example 1 (Fine-tuning on Corrected Data): }}
Given a point estimate of model parameters $\hat{{\boldsymbol{\theta}}}$, i.e., $q({\boldsymbol{\theta}}) = \delta({\boldsymbol{\theta}} - \hat{{\boldsymbol{\theta}}})$, a simple way to approximate $p({\boldsymbol{\theta}} \,|\, \mathcal{D}\setminus \mathcal{C})$ is to fine-tune on the corrected dataset $\mathcal{D}\setminus \mathcal{C}$ and update the point estimate. The new $\hat{{\boldsymbol{\theta}}}^{\star\star}_{\mathcal{D} \setminus \mathcal{C}}$ of the repaired model are obtained by maximising the log-likelihood $\log \, p(\mathcal{D}\setminus\mathcal{C} \,|\, {\boldsymbol{\theta}})$ via SGD, starting from $\hat{{\boldsymbol{\theta}}}$.

\textit{\textbf{Example 2 (Newton Update Removal): }}
Guo \textit{et al.} \cite{guo2019certified} proposed a Newton update based method for data deletion. This method reduces to a specific form of Eq.~(\ref{eq:approx_data_deletion}) when using negative log-likelihood as its loss:
\begin{equation}\label{eq:koh_data_deletion}
\hat{{\boldsymbol{\theta}}}^{\star\star}_{\mathcal{D} \setminus \mathcal{C}} \approx \hat{{\boldsymbol{\theta}}} - \gamma \hat{\boldsymbol{F}}_{\hat{{\boldsymbol{\theta}}}}^{-1}\nabla_{\hat{{\boldsymbol{\theta}}}}\log \, p(\mathcal{C} \,|\, \hat{{\boldsymbol{\theta}}}).
\end{equation}
Here information about $\mathcal{C}$ gets deleted by performing a single-step natural gradient descent on their log likelihood \cite{amari1998natural}. Also, notice the similarity with the way linear influence~\cite{koh2017understanding} is computed in Eq.~\eqref{eq:linear_influence}, illuminating the relation between cause identification and treatment steps.

\textit{\textbf{Example 3 (EWC for data deletion): }}
The update rule in Eq.~(\ref{eq:approx_data_deletion}) for data deletion is amenable to any continual learning approaches. For example, given model parameters $\hat{{\boldsymbol{\theta}}}$, EWC-based deletion obtains new parameters $\hat{{\boldsymbol{\theta}}}^{\star\star}_{\mathcal{D} \setminus \mathcal{C}}$ by maximising the following objective (see Appendix \ref{app:ewc_deletion_objective}):
\begin{equation}
     - \log \, p(\mathcal{C} \,|\, {\boldsymbol{\theta}})  - \frac{N}{2}({\boldsymbol{\theta}} - \hat{{\boldsymbol{\theta}}})^\top  \hat{\boldsymbol{F}}_{\hat{{\boldsymbol{\theta}}}} ({\boldsymbol{\theta}} - \hat{{\boldsymbol{\theta}}}) - \frac{\lambda}{2}||{\boldsymbol{\theta}} - \hat{{\boldsymbol{\theta}}}||^2_2
\label{eq:deletion_ewc}
\end{equation}
where the first term seeks to remove information about $\mathcal{C}$ while the remaining terms discourage parameters from deviating from the original values. Contrasting this with  Eq.~\eqref{eq:predictive_ewc} again reveals the connection between EWC methods for cause identification and treatment steps.

\textbf{Comparison:}
Similar to the comparison made in the cause identification part, EWC for data deletion also generalises the Newton update removal (Eq.~(\ref{eq:koh_data_deletion})). This can again be shown by deriving (damped) one-step fixed point iterative update starting from $\hat{{\boldsymbol{\theta}}}$ to approximate the fixed point of Eq.~(\ref{eq:deletion_ewc}) when $\lambda = 0$. As EWC for deletion uses SGD to approximate optimum of Eq.~(\ref{eq:deletion_ewc}), it is arguably better than Newton update removal for erasing the effects of detrimental examples $\mathcal{C}$, while better maintaining performance on other cases.

\vspace{-2mm}
\section{Related work}
\vspace{-2mm}
\textbf{Model Editing}.
There is a recent surge of interest in developing targeted updates to correct model's undesirable behaviours, while leaving other desired properties intact. As naive fine-tuning methods often lead to overfitting to the failure examples and accuracy degradation on others, various strategies have been proposed.
For example, Zhu \textit{et al.} \cite{zhu2020modifyingmemories} employ a simple regularization technique to minimize parameter changes during the fine-tuning phase. Subsequent works \cite{sinitsin2020editable,cao2021editing} advocate for a functional regularisation instead, e.g. KL divergence in the output space, to achieve better regularisation. These lines of work, additionally, propose to use meta-learning \cite{finn2017model} to learn to edit the target model, where the latest meta-learning approach is proposed by Mitchell \textit{et al.} \cite{mitchell2021fast}. Another promising approach \cite{santurkar2021editing} performs weight editing so that features of a specific concept (e.g. snow) map to the features of another (e.g. road). A commonality among these approaches is the focus on direct model edits for correction. Our work takes an orthogonal and under-explored angle where the aim is to ``edit'' the data instead, by identifying and removing harmful examples which cause failures --- in turn, this difference makes our framework complementary to these model-editing approaches.

\textbf{Continual learning.} Continual learning is an active research area with a related but broader scope than model repairment, which aims to develop methods that adapt the model for future tasks while maintaining model performance on previously learned tasks \citep{parisi2019continual}.
We focus on a more targeted problem in this work, yet introduce a framework that allows the use of any continual learning approach for model repairment.
Our experiments presents EWC \citep{kirkpatrick2017overcoming} as a practical instantiation of the framework. One can also leverage improvements over EWC such as online EWC \citep{schwarz2018progress}, or other regularisation-based methods that are motivated by Bayesian learning principles, such as variational continual learning \citep{nguyen2017variational, pan2020continual, loo2021generalized}, synaptic intelligence \citep{zenke2017continual}, and orthogonal gradient descent \citep{farajtabar2020orthogonal}. As approximations to $r(\mathcal{C})$ rely on accurate posterior approximations, advances in Bayesian continual learning methods are expected to improve the practical effectiveness of model repairment under our framework.

\textbf{Data Selection and Valuation}.
Multiple techniques have been introduced for selecting ``influential'' training examples on a chosen metric (e.g. test accuracy), such as influence functions \citep{koh2017understanding,koh2019accuracy,barshan2020relatif,giordano2019swiss,hara2019data,khanna2019interpreting,warnecke2021machine}, Shapley value-based approaches \citep{ghorbani2019data,ancona2019explaining,jia2019towards} and probability of sufficiency \citep{chakarov2016debugging}.
Within the category of influence functions, two representative approaches include linear influence function \citep{koh2017understanding} and SGD-influence \citep{hara2019data}. The former approach performs one-step update only, thus, while efficient, it may be less accurate in reflecting the influence of a datum $\bm{z}$. The latter approach computes a projected difference between $\hat{{\boldsymbol{\theta}}}$ and $\hat{{\boldsymbol{\theta}}}^{\star}_{\mathcal{D},\mathcal{F}}$ but with $\hat{{\boldsymbol{\theta}}}^{\star}_{\mathcal{D},\mathcal{F}}$ obtained by running SGD fine-tuning on training data without $\bm{z}$. Thus SGD-influence is computationally inefficient. Compared to both baselines, our EWC-influence approach achieve the best in both worlds: it produces more accurate influence estimates than linear influence due to better optimisation, while it is more efficient than SGD-influence as it requires only one optimisation procedure on the given failure set $\mathcal{F}$.

\textbf{Data Deletion}.
The detrimental data removal in the treatment step is related to \emph{data deletion}, a rapidly developing field of machine learning research \citep{bourtoule2019machine,guo2019certified,ginart2019making,izzo2020approximate,neel2020descent,gupta2021adaptive}.
Closest to our work is variational Bayesian unlearning \citep{nguyen2020variational} which extends variational Bayes to data deletion settings. But the connection to continual learning is not explicitly made, and it is limited to applications in logistic regression and sparse Gaussian processes.
In general, the main focus of existing data deletion research is to preserve data privacy, and datapoints to be removed are assumed provided. On the contrary, in this work, we focus on the repairment of models and propose a unified procedure not only to remove data but also to identify which ones to do so.

\begin{figure*}[ht!]
\centering
  \includegraphics[width=1.0\linewidth]{figures/labelnoise_cause_identification_with_errbars_4.pdf}
  \vspace{-2mm}
  \caption{\footnotesize
  Results on cause identification in the presence of annotation noise.
  (a) shows the confusion matrices used to simulate class-dependent label noise on MNIST and CIFAR-10. (b) shows the class distribution of the misclassified examples for a single run. (c) plots how much of the  identified causes match the samples with incorrect labels for different approaches. The shade represents the standard deviation computed from 5 different runs. (d) shows the top 16 causes of the failures as ranked by EWC-influence. }
  \label{fig:cause-identification}

  \label{fig:labelnose_detection}
  \vspace{-5mm}
\end{figure*}

\section{Experiments}
\vspace{-2mm}
We evaluate the efficacy of the proposed framework in a) identifying the causes of target prediction failures in Sec.~\ref{sec:results_causes}, and b) repairing the original model by erasing the memories of such causes in Sec.~\ref{sec:results_treatment}. We use augmented versions of MNIST and CIFAR-10 datasets with simulated annotation and input noise. Such controlled experiments are performed for creating "ground truths" of failure causes -- necessary for validating the quality of identification methods -- and for testing the method in a variety of settings.

\textbf{Baselines.} For the cause identification task, we compare our approach (\textit{EWC-influence}) against the linear influence function \cite{koh2017understanding} and \textit{SGD-influence} \cite{hara2019data}. To avoid expensive computation of $\hat{\boldsymbol{F}}_{\hat{{\boldsymbol{\theta}}}}^{-1}$, Koh $\&$ Liang \cite{koh2017understanding} introduced two efficient approximations to the Hessian-vector product $\hat{\boldsymbol{F}}_{\hat{{\boldsymbol{\theta}}}}^{-1} \nabla_{{\boldsymbol{\theta}}} \log p(\mathcal{F} \vert{\boldsymbol{\theta}})$; the first solves $\text{arg min}_{v}\{v^{T}\hat{\boldsymbol{F}}_{\hat{{\boldsymbol{\theta}}}}^{-1} \nabla_{{\boldsymbol{\theta}}}v - \log p(\mathcal{F} \vert{\boldsymbol{\theta}})^{T}v\}$ with gradient descent (GD), while the second uses an iterative algorithm for stochastic approximation (SA) from \citep{agarwal2016second}.
We implement these two variants (GD \& SA) of linear influence in Pytorch, and use the original implementation for SGD-influence. For the model treatment task, we compare our method (\textit{EWC-deletion}) against Newton update removal \cite{guo2019certified}. This method again requires computing a Hessian-vector product for which we employ the same stochastic approximation technique \cite{agarwal2016second}. To isolate the evaluation of cause identification and treatment, we further consider in Section \ref{sec:results_causes} \emph{fine-tuning} on $\mathcal{D}\setminus \mathcal{C}$ as another repairment strategy, which would return the best repairment result if the set $\mathcal{C}$ correctly captures the detrimental datapoints. Lastly, we set the prior term $\lambda$ to zero in eq.\eqref{eq:predictive_ewc} and eq.~\eqref{eq:deletion_ewc} to ensure fair comparison with linear influence and Newton update removal.

\textbf{Common Set-up.} We train the base classification models on the training split of the ``augmented'' MNIST and CIFAR-10 datasets.
For MNIST, we use $6\%$ (3000 samples) of the original training set to make the task more challenging. We use instances of CNNs throughout and train them using the Adam optimiser \cite{kingma2014adam}. The architecture and training details can be found in Appendix \ref{sec:supp_experimental_details}.
For evaluation, we separate the test set $\mathcal{T}$ into the set of misclassified examples, $\mathcal{F}$ (``\textit{failure set}'') and the others, $\mathcal{T}\setminus\mathcal{F}$ which are correctly classified (``\textit{remaining set}''). We further split the failure set into \textit{query}, $\mathcal{F}_{q}$ and \textit{holdout}, $\mathcal{F}_{h}$ sets, where we only use the former to identify failure causes $\mathcal{C}$, and use the latter to quantify how generalisably the removal of $\mathcal{C}$ can amend the failure cases. We stress that $\mathcal{F}_q$ is used for cause identification only, but not for further model adaptation.

\vspace{-2mm}
\subsection{Identifying Failure Causes}\label{sec:results_causes}
\vspace{-2mm}

\paragraph{Annotation Noise.}
To induce test prediction failures,
for the training set we randomly flip labels between semantically similar classes (e.g. 1 and 7 for MNIST, and cats and dogs for CIFAR-10) according to the confusion matrices in Fig.~\ref{fig:cause-identification}(a). As a result, the classes of miss-classified test examples are concentrated on those classes with label noise as depicted in Fig.~\ref{fig:cause-identification}(b).

To measure the accuracy of identifying incorrectly labelled examples, we inspect the training examples $\boldsymbol{z} \in \mathcal{D}$ in the descending order of $\tilde{r}(\boldsymbol{z})$ computed with $\mathcal{F}_q$ which contains 50\% of the miss-classified test cases, and calculate the fraction of incorrectly labelled datapoints in inspected examples. Fig.~\ref{fig:cause-identification}(c) shows that EWC-influence identifies more failure causes earlier on compare to other methods, and is the closest match to the ``Oracle'' baseline which has full knowledge of samples with wrong labels. Fig.~\ref{fig:cause-identification}(d) shows that the top few causes according to EWC-influence are the samples with incorrect labels, while the least harmful ones are the images of the same classes but with the correct labels as shown in Fig.~\ref{fig:label_noise_least_harmful} in Appendix \ref{sec:supp_extra_results}.

As stipulated in Sec.~\ref{sec:method}, a set of identified causes $\mathcal{C}$ is of higher quality if removing them leads to a larger gain in accuracy on the failure set while maintaining performance on others. To measure such \emph{quality of causes}, we fine-tune the base model on $\mathcal{D}\setminus \mathcal{C}$
and report the accuracy on the failure query set $\mathcal{F}_{q}$, the holdout failure set $\mathcal{F}_{h}$ as well as the remaining test set, $\mathcal{T}\setminus\mathcal{F}$. Results in Fig.~\ref{fig:causes_quality} suggest that removing failure causes according to EWC-influence yields the highest increase in accuracy on the failure set $\mathcal{F}$ without hurting performance in the remaining test set $\mathcal{T} \setminus \mathcal{F}$. We also note that all of the methods are able to fix the failures better than randomly removing datapoints, and more interestingly, for MNIST, when enough causes are erased ($\approx 10^3$), all methods even surpass the case in which all label noise instances are removed. This result implies that, while annotation noise is a major detrimental factor, the prediction failures also arise from other types of harmful examples.

Lastly, we evaluate the sample efficiency of cause identification by reducing the size of the query set $\mathcal{F}_{q}$.  Fig.~\ref{fig:sample_efficiency} in Appendix \ref{sec:supp_extra_results} shows that all approaches degrade gracefully in repairment performance as the query size gets smaller, but overall EWC-influence still remains the best in terms of label noise detection and repairment accuracy on the failure set and the remaining set.

\begin{figure*}[t]
\centering
  \vspace{-5mm}
  \includegraphics[width=\linewidth]{figures/labelnoise_repairment_v4.pdf}
  \vspace{-3mm}
  \caption{\footnotesize Comparison of the quality of identified causes in the presence of annotation noise. The impact of gradually removing samples in MNIST and CIFAR-10 datasets in the order of influence values $r(\boldsymbol{z})$ are measured on the failure sets (holdout and query) in (a), and on the remaining test set in (b). We also plot the performance of another reference (``semi-oracle'') that is the original model fine-tuned on the training data without the label noise instances. The means and standard deviations of all quantities are calculated over 5 different runs. }
  \label{fig:causes_quality}
  \vspace{-3mm}
\end{figure*}

\textbf{Random Input Noise.}
In this experiment, we inject synthetic outliers into MNIST and CIFAR-10 and test the quality of cause identification. We select a set of target classes~---~1, 7, 6, 9 for MNIST and plane, bird, cat, dog for CIFAR-10~---~and randomly corrupt 30\% of the images in those classes by adding salt-and-pepper in MNIST and Gaussian noise in CIFAR-10. The top row in Fig.~\ref{fig:input_noise_detection} shows examples, and those corrupted images constitute roughly 12\% of the whole training set. Sec.~\ref{sec:supp_experimental_details} in the appendix provides details.

We use a subgroup of failures in the target classes as the query $\mathcal{F}_q$ to compute influence values. Surprisingly, the second rows in Fig.~\ref{fig:input_noise_detection}(a) and (b) show that EWC-influence largely avoids selecting the corrupted images as the top 1000 causes for MNIST and the top 20000 causes for CIFAR-10. However, the third and the fourth rows show that removing those causes results in the best treatment performance on failures while maintaining the performance at a level similar to other baselines. In fact, removing all the input noise and retraining is not able to fix the failures by much, indicating that EWC-influence is able to correctly avoid these relatively harmless outliers and detect other more harmful causes. Fig.~\ref{fig:input_not_harmful} in Appendix \ref{sec:supp_extra_results} visualises the most harmful examples identified by EWC-influence.
Many of them appear to be ambiguous instances in non-target classes, e.g. wonky digits, close-up views of vehicles, a real instance with incorrect label \cite{northcutt2021confidentlearning}, etc.

\textbf{Adversarial Poisoning.}
To simulate input noise that can induce test-time failures, we introduce contaminated data by randomly corrupting 30\% of the training images in those previously mentioned target classes. These poisoned datapoints are adversarial images crafted by the fast gradient sign method (FGSM)~\citep{goodfellow2014explaining} on a separate set of victim models trained on the original clean datasets, and they are labelled by the classes predicted by the victim models. The poisoned datasets are then used to train the base models that are used for evaluation of cause identification.

Fig.~\ref{fig:input_noise_detection}(c) and (d) show that most of the influence functions detect the corrupted samples better than the ``random'' baseline. The dashed lines in the third row show that removing all of the corrupted inputs lead to a significant gain in accuracy on the holdout failure set in comparison with the random noise setting, illustrating the larger extent of harms caused by data poisoning. However, most of the identification methods still outperform this reference by a large margin. This suggests again for the presence of other influential samples, and EWC-influence is able to pick up the most important ones, judging by the accuracy on the failure set.

\textbf{Speed Comparison.} Table~\ref{tab:computation} in Appendix \ref{sec:supp_extra_results} shows the total run-time of cause identification methods on a single GPU for their best sets of hyper-parameters selected based on the treatment accuracy on the failure set. For both datasets, EWC-influence achieves comparable or shorter run time than the baselines.

\begin{figure*}[t]
\centering
\vspace{-5mm}
  \includegraphics[width=1.0\linewidth]{figures/adversarial_and_random_input_noise_3.pdf}
  \vspace{-2mm}
  \caption{\footnotesize

  Results on cause identification in the presence of different input noises. From top to bottom, we show i) examples of corrupted samples (synthetic proxy for potential causes of failure), ii) how many of the identified causes correspond to samples corrupted with input noise, iii) and iv) performance in failure holdout set $\mathcal{F}_{h}$ and remaining test set when removing the top 1000/20000 identified causes in MNIST/CIFAR-10. The influence values are calculated with respect to 50\% of test-time failure cases that belong to the classes that suffer from input noise. EWC-Influence identifies ``harmful'' (adversarial) input noise better than random while avoiding ``harmless'' (random) input noise.

  }
  \label{fig:input_noise_detection}
   \vspace{-1mm}
\end{figure*}

\vspace{-1mm}
\subsection{Comparison of Treatment Methods}\label{sec:results_treatment}
\vspace{-1mm}
We evaluate the performance of different deletion-based methods for treatment introduced in Sec.~\ref{sec:treatment} on MNIST and CIFAR10 datasets with simulated annotation noise, used in the previous section.
We run both EWC-deletion (ours) and Newton update removal \citep{guo2019certified} methods with early stopping based on the query set accuracy, and experiment with different hyper-parameter settings (see Sec.~\ref{sec:supp_experimental_details} in Appendix) to achieve different trade-offs between failure set accuracy and remaining set performance. Here EWC-influence is used to identify the causes, and the top 15\% examples were removed by the respective deletion methods.
Such trade-off is shown in Fig.~\ref{fig:deletion_comparison}, where fine-tuning on $\mathcal{D}\setminus\mathcal{C}$ is included as an ``upper-bound'' reference for data deletion performance.
On MNIST, EWC-deletion attains a considerably better trade-off between treatment and maintenance compared to Newton-update-deletion, and is much closer to the fine-tuning reference. For CIFAR-10, EWC-deletion beats the Newton-update deletion by 5\% in the best failure accuracy while the order reverses for the best accuracy on the remaining test set but with less than 1\% difference.

\begin{figure}[t!]
\centering
   \vspace{-1mm}
  \includegraphics[width=0.7\linewidth]{figures/repairment_method_comparison_without_finetuning_v2.pdf}
    \vspace{-2mm}
  \caption{\footnotesize Comparison of deletion-based treatment methods on (a) MNIST and (b) CIFAR-10. For Newton-update-deletion and EWC-deletion, we plot multiple results for varying hyper-parameters to visualise the trade-off between the accuracy on the failure set and the remaining set. The closer to the top right corner, the more desirable.}
      \vspace{-5mm}

  \label{fig:deletion_comparison}
\end{figure}

\vspace{-3mm}
\section{Conclusions}
\vspace{-2mm}
In this work, we develop a generic framework for \textit{repairing} machine learning models by erasing memories of detrimental datapoints. The framework consists of two key components, that are, the mechanism for identifying the ``causes'' in training data which are responsible for the given failures, and the adaptation method for fixing the model by removing information about them. Two components are connected under the Bayesian view of continual (un)learning, which brings forth several practical benefits. Firstly, the framework subsumes some recent works on influence function and data deletion as specific examples, and elucidate their limitations. Secondly, the generality of our approach allows leveraging recent advances in continual learning in this new problem of model repairment. In particular, we extend Elastic Weight Consolidation to cause identification and information removal, and demonstrate empirically its competitive performance in both tasks. Future work will investigate the values of adapting more recent continual learning approaches, and also study
other types of data correction mechanisms (e.g. label correction, sample/label acquisition).

\section*{Acknowledgements}
We would like to thank Ozan Oktay, Stephanie Hyland, and Ted Meeds at Microsoft Research Cambridge and Jin Chen at University College London for their valuable feedback on an early version of this work.

\bibliographystyle{unsrt}

\input{main.bbl}

\newpage

\newpage

redpajama_set_name: RedPajamaArXiv


text:
ɯ – oznacza ona samogłoskę przymkniętą tylną niezaokrągloną w międzynarodowym alfabecie fonetycznym.
 Uu – dwuznak występujący w języku niderlandzkim, fińskim i estońskim.
 uu (ω) – omega, ostatnia litera alfabetu greckiego.
 UU (Ɯ) – dodatkowa litera alfabetu łacińskiego używana do 1986 roku w piśmiennictwie języka zhuang.

Strony ujednoznaczniające

redpajama_set_name: RedPajamaWikipedia


Pizza is my heart and soul. My death-row meal. The one item I would never give up for a diet. Now let's turn that classic dish into miniature deep dish cups of holy deliciousness! A nice crunch on the outside and soft on the inside with bursts of flavor from fresh cherry tomatoes and basil. I'm dying to find an excuse to throw a party because these make wonderful appetizers! See for yourself!
Makes 24 pizza cups – I usually just make 12 and save the other half of dough in the freezer.
1 cup pizza sauce – use my easy red sauce recipe!
(1.) In a large bowl, stir together water, yeast, honey, and olive oil. Let it sit for 10 minutes or until foamy. (2.) Slowly stir in 2 ½ cups of flour, garlic salt, and basil until dough comes to together. Knead for a few minutes on a floured surface and work in remaining ½ cup of flour. (3.) Rub the same bowl with olive oil, form dough into a ball and place in the bowl. Turn and flip to coat in olive oil. Cover with a towel and place in a warm place to rise for 1 hour and 30 minutes. Once dough has risen, punch down on a floured surface, form back into a ball, and place in the bowl. Cover and let it sit for another 10 minutes. (4.) When dough is done, cover your rolling pin with flour and roll dough out to ¼ inch thickness.
I divided my dough in half and froze the other section for a later date. I had plenty of dough to make 12 pizza cups AND a mini pizza!
(5.) Use a biscuit cutter or drinking glass to cut circles out of the dough. (6.) Using your fingers, stretch dough circles out a bit more and then place into muffin tin. Press dough against the bottom and sides of the muffin cup. (7.) Lightly brush with olive oil and then add toppings. (8.) Place a heaping spoonful of sauce on top of each one. Add cheese and then place in the oven.
Bake for 8-10 minutes or until cheese is bubbly and lightly browned on the edges.
Let pizza cups cool for 5 minutes before removing from pan to serve.
Now watch as they disappear!


Did you know that most digital photos (taken with digital cameras or smartphones) are automatically geotagged with the location where they were taken? Protect your privacy and learn how to manage geotagging during Colorado Mountain College in Edwards' iPhoto class. MAC iPhoto will be held on Saturday from 8:30 a.m. to 12:30 p.m. It costs just $49. Call 970-569-2900 to register!


New Barres and Mirrors for St Mary's!
We are completely thrilled with the new barres and mirrors that we have recently installed at our St Mary's of the Angels venue in Notting Hill. With freshly painted walls and newly sanded floors to boot the whole place is looking very smart indeed!


text:
Andrew Peck
HSBC appoints Cox as new CIBM head
Rhydian Cox bids adios to Mexico and returns Hong Kong in a newly created role.
HSBC, in the midst of building its investment banking globally, has made a key appointment, naming Rhydian Cox as head of corporate, investment banking and markets (CIBM), Asia Pacific.
Effective July 17, Cox will take on the newly created role in HSBCÆs Hong Kong offices and will report directly to Stuart Gulliver, head of HSBCÆs CIBM group. His appointment also comes a month after former co-head of CIBM, John Studzinski announced he will leave HSBC to join private equity firm Blackstone as a senior managing director and partner.
According to HSBC, CoxÆs move to Hong Kong is reflective of the regionÆs importance to its CIBM franchise. His appointment comes three years into its five-year plan to beef up its investment banking team and compete with other international rivals.
Cox has worked at HSBC for 22 years. Most recently, he was head of CIBM for HSBC Mexico. In the position, he played a major role in the the growth of this business.
Prior to his Mexico role, Cox was senior manager, corporate banking in HSBCÆs Hong Kong offices. He held this role between 1998 and 2003. While in Hong Kong, he was also chairman of the board of governors at the Matilda Hospital and honorary treasurer of the Hongkong Society of Rehabilitation.
Cox has also worked in Malaysia, South Korea, France and the UK. He began his career with the bank as a management trainee in 1984.
hsbc cibm cox
Toby Yiu
Bridgewater becomes first hedge fund with CIBM access
The hedge fund giant and three other asset managers – Aberdeen, AllianzGI and BNP Paribas Investment Partners – have been approved to trade China's interbank bond market.
UBS AM adds to China bond team, eyes CIBM access
The fund house's Asia head of fixed income, Ashley Perrott, suggests that waiting on China's inclusion in global bond indexes is a red herring, as his firm continues its push onshore.
Christina Wang
Invesco to tap CIBM this year, tips $400bn in fund manager flows
Invesco aims to directly invest in China's interbank bond market this year, says its Asia-Pacific head of fixed income. The firm expects to see up to $400 billion in flows from fund managers alone.

redpajama_set_name: RedPajamaCommonCrawl


I'm not sure quite what I was expecting when I arrived at the National Mask Festival in Papua New Guinea (P.N.G.). I think I envisioned a gathering of P.N.G.'s many tribes in a big grassy field surrounded by jungle. But I soon discovered, as I was passing through the tickets-required entryway, that it's a far more organized affair.
There was a grassy field, but it was flanked on one side by a stage and surrounded by several canopies emblazoned with the logo of the country's cellular provider, Digicel. The festival had the air of a small state fair, with a couple dozen stalls selling food, T-shirts, and other wares in what would usually be a big empty lot.
As the tribes arrived in their colorful costumes, an emcee led them onto the field, where they proudly performed traditional dances called sing-sings while tourists eagerly documented the spectacle on their smartphones — myself included.
I couldn't help but wonder: Was the National Mask Festival put on purely to lure tourists to the remote Melanesian island?
Were any of the sing-sings I'd witnessed earlier in other regions authentic if they had been scheduled by a local tour operator to start upon my arrival?
Did the villagers of the Kainagunan ward, who performed the traditional Baining fire dance ritual (see my video below), risk third-degree burns just because some foreigners shelled out money to see them?
The answers, I learned, were complex.
Papua New Guinea isn't exactly a wildly popular destination for tourists — especially American ones. In fact, Ally Stoltz of the nation's Tourism Promotion Authority told me that roughly 7,000 Americans visit P.N.G. — a country in the southwestern Pacific about the size of California — each year. "Many places receive only a handful of visitors a month, making 'tourist traps' nearly impossible to find in P.N.G.," she said.
To be sure, the island nation is still in the early stages of developing its tourism strategy — and destiny. To its credit, the P.N.G. National Cultural Commission is careful to avoid having the country explode on the mass tourism market without considering the effect such an influx could have on the idiosyncratic island's cultural identity.
Simet and the commission strive to keep sing-sings out of hotels and resorts through a policy dictating that tourists who want to see local traditions must journey to the village where they take place. This helps prevent these traditional displays of song and dance from feeling staged, like luaus at some Hawaiian resorts.
So why see a sing-sing at all? For me, experiencing another culture is the main reason to travel, and the prospect of witnessing a sing-sing in Papua New Guinea was a huge part of why I had journeyed there to begin with.
But since sing-sings are normally reserved for events like weddings or coming-of-age rituals, why would villagers perform them otherwise without being asked to — and without being compensated?
C) a really good coincidence.
If you're spending time and money to travel all the way to Papua New Guinea, you're probably going to want to go with option B if you have limited time and no local contacts.
As the five-day National Mask Festival continued, I noticed that there were many more locals in attendance than I had expected; in fact, they seemed to outnumber tourists at least 20 to 1. The performances seemed to please all spectators, native and foreign alike — though some contained elements only a local would understand. For example, one troupe, the Bitapaka Theatre Group, performed a comedy skit in the country's principal lingua franca, Tok Pisin (commonly referred to as Pidgin), instead of dancing in traditional garb.
"Our people over time have understood and appreciated that their cultures cannot be traded in easily for cash, and they are encouraged to ensure their legends, myths, cultural and traditional values are protected at all times," Vincent said.
Erik Trinidad may be based in Brooklyn, but he spends most of his time crisscrossing the globe in search of exotic food, high adventure, and scientific curiosities. Follow his travels on Instagram and Twitter @theglobaltrip.


February 6, 2016- The Allen Speiser Memorial Fund for Vocational Rehabilitation is accepting grant applications from programs that promote the placement and retention of employees with disabilities in the workforce. The application deadline is March 3, 2017. Eligibility is limited to 501(c)(3) nonprofit, human service organizations that serve people with disabilities in Onondaga, Madison, Oswego, Cortland or Cayuga counties.
The Speiser Fund generally provides grants of up to $2,000 to agencies that support education and employment opportunities for people with disabilities and raise awareness of the employment potential of people with disabilities. However, collaborative or innovative projects may exceed a $2,000 request.
Training for auxiliary personnel – training or technical assistance for professionals, employers or other individuals who provide workplace oriented services to individuals with disabilities.
Equipment and assistive technology – any items or pieces of equipment that will be used to enhance functional capabilities of individuals with disabilities in the workplace.
Materials for building and enhancing job readiness– materials that professionals or employers can use to enhance the employability skills of individuals with disabilities.
Work supports promoting job accommodation – may include assistance preparing for work, getting to and from work, meeting personal needs in the workplace and performing job functions.
Visit www.cnycf.org/SpeiserGrants for guidelines and an application form. Questions may be directed to Danielle Hurley at (315) 883-5554 or DHurley@cnycf.org.