\section{Introduction}

	One-shot devices, an extreme case of interval censoring, play an important role in survival analysis. One-shot devices can be only tested at some discrete inspections times, so we can only know if a test unit have failed or not at certain fixed times.
	Real-life one-shot devices usually have large mean lifetimes under normal operating conditions, and so accelerated life tests (ALTs) plans may be useful to infer on their reliability.
	ALTs plans assume that the mean lifetime of the devices is related to the stress level at which units are tested, and therefore they
	accelerate the time to failure by increasing the stress level.

	Generally, one-shot devices are assumed to get destroyed when tested, and so one-shot data are right and left censoring.
	However, the non-destructiveness assumption may not be necessary in many practical situations. In this paper, we focus on these non-destructive one-shot devices and we study some inference methods for analyzing their lifetime characteristics.
	The non-destructiveness condition allows surviving units to continue in the experiment, providing extra information about their lifetime distribution.

	In this context,  step-stress ALTs, which increase the stress level progressively during the experiment  at certain pre-specified times (known as times of stress change), make the best use of the non-destructive devices under tests.
	Here, we assume that the lifetime distribution of the one-shot device at one  stress level is related to the distribution at preceding stress levels by
	assuming the residual life of the device depends only on the cumulative exposure it had experienced,	with no memory of how this exposure was accumulated.
	We consider a multiple step-stress ALT with $k$ ordered stress levels, $x_1< x_2 < \dots < x_k$ and  their corresponding times of stress change $\tau_1 < \tau_2 \dots < \tau_k.$
	We assume the lifetimes of one-shot devices follows an exponential distribution, which is widely used as a simple lifetime model in engineering and physical sciences.
	The cumulative exposure model describes the lifetime distribution of a device as
	 \begin{equation}\label{eq:distributionT}
	 	G_T(t) =
	 	\begin{cases}
	 		G_1(t) = 1-e^{-\lambda_1 t}, & 0<t< \tau_1\\
	 		G_2\left(t + a_1 -\tau_1 \right)  = 1-e^{-\lambda_2(t + a_1 -\tau_1)}, & \tau_1 \leq t < \tau_2 \\
	 		\vdots  & \vdots \\
	 		G_k\left(t + a_{k-1} -\tau_{k-1} \right)  =  1-e^{-\lambda_k(t + a_{k-1} -\tau_{k-1})}, & \tau_{k-1} \leq t < \infty ,\\
	 	\end{cases}
	 \end{equation}
	 with
	 \begin{equation}\label{eq:ai}
	 	a_{i-1} = \frac{ \sum_{l=1}^{i-1}\left(\tau_l-\tau_{l-1}\right)\lambda_l}{\lambda_i}, \hspace{0.3cm} i=1,...,k-1.
	 \end{equation}
	 and

	 \begin{equation} \label{eq:loglinear}
	 	\lambda_i(\boldsymbol{\theta}) = \theta_0 \exp(\theta_1 x_i), \hspace{0.3cm} i = 1,..,k,
	 \end{equation}
	 where $\boldsymbol{\theta} = (\theta_0, \theta_1) \in \mathbb{R}^+ \times \mathbb{R} = \Theta$ is an unknown parameter vector of the model.

	 The log-linear relation in (\ref{eq:loglinear}) is frequently assumed in accelerated life test models, as it can be shown to be equivalent to  the well-known inverse power law model or the Arrhenius reaction rate model.

	Now, let consider a grid of inspection times, $t_1 < t_2 < \dots < t_L$, containing all times of stress change. The probability of failure within the interval $(t_{j-1}, t_{j}]$ is given by
	\begin{equation} \label{eq:th.prob}
		\pi_j(\boldsymbol{\theta}) =  G_T(t_j) - G_T(t_{j-1}), \hspace{0.3cm} j = 1,..,L,
	\end{equation}
	and the probability of survival at the end of the experiment is $\pi_{L+1}(\boldsymbol{\theta}) = 1 - G_T(t_L).$

 	Classical inferential methods for one-shot are based on the maximum likelihood estimator (MLE), which is very efficient by it lacks of robustness.
 	To overcome the robustness drawback, Balakrishnan et al. (2022) proposed robust estimators for one-shot devices based on the popular density power divergence (DPD) (Basu et al. 1998) under exponential lifetimes.
 	They developed minimum DPD estimators (MDPPE) as well as Wald-type test based on them, and studied theoretically and empirically their asymptotic and robustness properties.

 	On the other hand, some inferential procedures, as Rao-type tests are based on restricted estimators.
 	 Basu et al. (2018) developed robust restricted estimators based on the DPD for general statistical models, and derived their asymptotic distribution and robustness properties. Jaenada et al. (2022) extended the theory using the Rényi pseudistance and developed some testing procedures based on the restricted estimators.

 	In this paper, we develop restricted MDPDE under linearly constrained subspaces for non-destructive one-shot devices tested under step-stress ALT.  In Section 2 we define the restricted MDPDE, and we state its asymptotic distribution. Section 3 theoretically analyzes the robustness of the restricted MDPDEs through its Influence Function (IF).
 	Finally, in Section 4 a simulation study is carried out to evaluate the performance of the proposed estimators under different scenarios of contamination.

	\section{Minimum density power divergence estimator and Restricted minimum density power divergence estimator \label{sec:modelformulation}}

	The density DPD family represents a rich class of density based divergences.
	It is indexed by a tuning parameter $\beta \geq 0$ controlling the trade-off between robustness and efficiency.
	Let consider $(n_1,...,n_{L+1})$ a sample of one-shot data. The empirical probability vector of a multinomial model can be defined as
	\begin{equation} \label{eq:emp.prob}
		\widehat{\boldsymbol{p}} = (\frac{n_1}{N},...,\frac{n_{L+1}}{N}).
	\end{equation}
	For step-stress ALT for one-shot devices under exponential lifetime distributions, the DPD between the
	the empirical and theoretical probability vectors, defined in (\ref{eq:th.prob}) and (\ref{eq:emp.prob}), respectively,  is given by
	\begin{equation}\label{eq:DPDloss}
		d_{\beta}\left( \widehat{\boldsymbol{p}},\boldsymbol{\pi}\left(\boldsymbol{\theta}\right)\right)   = \sum_{j=1}^{L+1} \left(\pi_j(\boldsymbol{\theta})^{1+\beta} -\left( 1+\frac{1}{\beta}\right) \widehat{p}_j\pi_j(\boldsymbol{\theta})^{\beta}  +\frac{1}{\beta} \widehat{p}_j^{\beta+1} \right),
	\end{equation}
	From the above, we define the MDPDE for the step-stress ALT model with one-shot devices as
	\begin{equation}
		\boldsymbol{\widehat{\theta}}^{\beta} = \left(\widehat{\theta}_0^{\beta},\widehat{\theta}_{1}^{\beta}\right) = \operatorname{arg} \operatorname{min}_{\boldsymbol{\theta}  \in \Theta} d_{\beta} \left( \widehat{\boldsymbol{p}},\boldsymbol{\pi}\left(\boldsymbol{\theta}\right)\right).
	\end{equation}
	Note that, at $\beta = 0$, the DPD coincides with the Kullback-Leibler divergence and so the MDPDE for $\beta=0$ coincides with the MLE.

	In many practical situations it may be of interest to reduce the parameter space to values of $\boldsymbol{\theta}$ satisfying a linear constraint of the form
	\begin{equation}\label{eq:restriction}
		g(\boldsymbol{\theta}) = \boldsymbol{m}^T\boldsymbol{\theta}-d = 0,
	\end{equation}
	with $\boldsymbol{m} = (m_0,m_1)^T \in \mathbb{R}^2$ and $d\in \mathbb{R}.$

		Accordingly, the restricted MDPDE, $\widetilde{\boldsymbol{\theta}}^\beta$ is defined by
		\begin{equation}\label{RMDPDE}
			\widetilde{\boldsymbol{\theta}}^\beta = \operatorname{arg} \operatorname{min}_{\boldsymbol{\theta \in \Theta_0}} d_\beta(\widehat{\boldsymbol{p}}, \boldsymbol{\pi}(\boldsymbol{\theta})).
		\end{equation}

	Since the restricted MDPPE is a constrained minimum, its estimating equations can be written in terms of Lagrange multipliers. That is,
	the MDPDE restricted to the the linear constraint (\ref{eq:restriction}), $\widetilde{\boldsymbol{\theta}}^\beta,$ must satisfy the restricted equations
	\begin{equation}\label{eq:estimatingRMDPDE}
		\boldsymbol{W}^T\boldsymbol{D}_{\boldsymbol{\pi}(\widetilde{\boldsymbol{\theta}}^\beta)}^{\beta-1}\left( \widehat{\boldsymbol{p}}- \boldsymbol{\pi}(\widetilde{\boldsymbol{\theta}}^\beta)\right)+ \boldsymbol{m}\widetilde{\boldsymbol{\lambda}} = \boldsymbol{0}_2,
	\end{equation}
	for some vector $\widetilde{\boldsymbol{\lambda}}$ of Lagrangian multipliers,
	where $\boldsymbol{0}_2$ is the 2-dimensional null vector, $\boldsymbol{D}_{\boldsymbol{\pi}(\boldsymbol{\theta})}$ denotes a $(L+1)\times(L+1)$ diagonal matrix with diagonal entries $\pi_j(\boldsymbol{\theta}),$ $j=1,...,L+1,$ and $\boldsymbol{W}$ is a $(L+1) \times 2$ matrix with rows
	$
	\boldsymbol{w}_j =  \boldsymbol{z}_j-\boldsymbol{z}_{j-1},
	$
	where
	\begin{align}
		\label{eq:zj} \boldsymbol{z}_j  &= g_T(t_j)\begin{pmatrix}
			\frac{t_j+a_{i-1}-\tau_{i-1}}{\theta_0}\\
			(t_j+a_{i-1}-\tau_{i-1})x_i + a_{i-1}^\ast
		\end{pmatrix}, \hspace{0.3cm} j = 1,...,L,\\
		a_{i-1}^\ast &= \frac{1}{\lambda_{i}} \sum_{l=1}^{i-1}\lambda_l\left(\tau_l-\tau_{l-1}\right)(-x_{i}+x_l) , \hspace{0.3cm} i=2,..,k,\label{aast}
	\end{align}
	$ \boldsymbol{z}_{-1} = \boldsymbol{z}_{L+1} = \boldsymbol{0}$ and $i$ is the stress level at which the units are tested after the $j-$th inspection time.

	The next theorem states the asymptotic distribution of the restricted MDPDE for non-destructive one-shot devices under the step-stress ALT model.

	\begin{theorem}
		Let $\boldsymbol{\theta}_0$ be the true value of the parameter $\boldsymbol{\theta}$ and assume that $g(\boldsymbol{\theta}_0) = 0$ with $g(\cdot)$ defined in (\ref{eq:restriction}).
		The asymptotic distribution of the restricted MDPDE for the step-stress ALT model under exponential lifetimes, $\tilde{\boldsymbol{\theta}}^\beta,$ obtained under the constraint $g(\boldsymbol{\theta})=0,$ is given by
		$$\sqrt{N}\left(\widetilde{\boldsymbol{\theta}}^\beta-\boldsymbol{\theta}_0 \right) \xrightarrow[N \rightarrow \infty]{L}\mathcal{N}\left(\boldsymbol{0}, \Sigma_\beta(\boldsymbol{\theta}_0)\right)$$
		where
		\begin{equation}\label{eq:SigmaPQmatrices}
			\begin{aligned}
				\Sigma_\beta(\boldsymbol{\theta}_0) &= \boldsymbol{P}_\beta(\boldsymbol{\theta}_0) \boldsymbol{K}_\beta(\boldsymbol{\theta}_0) \boldsymbol{P}_\beta(\boldsymbol{\theta}_0),\\
				\boldsymbol{P}_\beta(\boldsymbol{\theta}_0) &=  \boldsymbol{J}_\beta(\boldsymbol{\theta}_0)^{-1}- \boldsymbol{Q}_\beta(\boldsymbol{\theta}_0)\boldsymbol{m}^T \boldsymbol{J}_\beta(\boldsymbol{\theta}_0)^{-1},\\
				\boldsymbol{Q}_\beta(\boldsymbol{\theta}_0) &=  \boldsymbol{J}_\beta(\boldsymbol{\theta}_0)^{-1}\boldsymbol{m}( \boldsymbol{m}^T \boldsymbol{J}_\beta(\boldsymbol{\theta}_0)^{-1}\boldsymbol{m})^{-1},
			\end{aligned}
		\end{equation}

		with \begin{equation} \label{eq:JK}
			\boldsymbol{J}_\beta(\boldsymbol{\theta}_0) = \boldsymbol{W}^T D_{\boldsymbol{\pi}(\boldsymbol{\theta_0})}^{\beta-1} \boldsymbol{W},
			\hspace{0.3cm}  \hspace{0.3cm}
			\boldsymbol{K}_\beta(\boldsymbol{\theta}_0) = \boldsymbol{W}^T \left(D_{\boldsymbol{\pi}(\boldsymbol{\theta_0})}^{2\beta-1}-\boldsymbol{\pi}(\boldsymbol{\theta}_0)^{\beta}\boldsymbol{\pi}(\boldsymbol{\theta}_0)^{\beta T}\right) \boldsymbol{W},
		\end{equation}
		$D_{\boldsymbol{\pi}(\boldsymbol{\theta_0})}$ denotes the diagonal matrix with entries $\pi_j(\boldsymbol{\theta_0}),$ $j=1,...,L+1,$ and $\boldsymbol{\pi}(\boldsymbol{\theta}_0)^{\beta}$ denotes the vector with components $\pi_j(\boldsymbol{\theta}_0)^{\beta}.$

	\end{theorem}

	The proof follows from Theorem 2 of Basu et al. (2018) and Result 3 of Balakrishnan et al. (2022).

	\section{Influence function of the restricted minimum density power divergence estimator}

	The robustness of an estimator is widely analyzed using the concept of Influence Function (IF), first introduced in Hampel et al. (1986). Intuitively, the IF describes the effect of an infinitesimal contamination of the model on the estimate. Therefore, IFs associated to locally	robust  estimators should be bounded.
	The IF of the  MDPDE for the step-stress ALT model with non-destructive one-shot devices was established in Balakrishnan et al. (2022), and the boundedness of the function was discussing there, concluding that the IF of the  MDPDE is always bounded for positive values of the tuning parameter.
	Here we derive the IF of the restricted MDPDE, $\widetilde{\boldsymbol{\theta}}^\beta,$ defined in Section \ref{sec:modelformulation}. Observe that in this case the functional associated to the restricted estimator must also satisfied the subspace constraint.
	The statistical functional and influence function of the estimators under parametric restrictions have been rigorously studied in Ghosh (2015). Here, we study the IF of the restricted MDPDE when the subspace constraint has a linear form.

	We consider $F_{\theta}$ and the $G$ the assumed and real distribution functions with associated mass functions $\boldsymbol{\pi}(\boldsymbol{\theta})$ and $\boldsymbol{g},$ respectively.
	We define  $\widetilde{\boldsymbol{T}}_\beta$ the functional associated to the restricted MDPDE, $\widetilde{\boldsymbol{\theta}}^\beta,$ computed as the minimizer of the DPD given in (\ref{eq:DPDloss}) between the mass functions $\boldsymbol{\pi}(\boldsymbol{\theta})$ and $\boldsymbol{g}$ subject to the linear constraint $\boldsymbol{m}^T\boldsymbol{\theta} - d = 0.$

	For influence function analysis, one could derive the IF expression from the estimating equations of the restricted MDPDE in terms of Lagrange multipliers given in (\ref{eq:estimatingRMDPDE}). However, Ghosh (2015) proposed an alternative approach where the functional $\widetilde{\boldsymbol{T}}_\beta$ associated to the restricted MDPDE is calculated as a solution of the estimating equations of the (unconstrained) MDPDE over the subspace $\Theta_0.$ The existence of such solution is guaranteed by the Implicit Function Theorem.
	Hence, the IF of the restricted MDPDE at the contamination point $\boldsymbol{n}$ and the model distribution with true parameter value $\boldsymbol{\theta}_0$, $F_{\boldsymbol{\theta}_0},$  must simultaneously verify the expression of the IF of the MDPDE stated in Balakrishnan et al. (2022),
	\begin{equation*}
		\text{IF}\left(\boldsymbol{n}, \widetilde{\boldsymbol{T}}_\beta, F_{\boldsymbol{\theta}_0}\right) = \boldsymbol{J}_\beta^{-1}(\boldsymbol{\theta}_0) \boldsymbol{W}^T \boldsymbol{D}_{\boldsymbol{\pi}(\boldsymbol{\theta}_0)}^{\beta-1}\left(-\boldsymbol{\pi}(\boldsymbol{\theta}_0)+\Delta_{\boldsymbol{n}} \right)
	\end{equation*}
	and the subspace constraint
	$\boldsymbol{m}^T \widetilde{\boldsymbol{T}}_\beta - d = 0.$
	Differentiating on the previous subspace constraint, we have that
	$$\boldsymbol{m}^T \text{IF}\left(\boldsymbol{n}, \widetilde{\boldsymbol{T}}_\beta, F_{\boldsymbol{\theta}_0}\right) = 0$$
	and therefore, combining both equations, we get
	\begin{equation*}
		\begin{pmatrix}
			\boldsymbol{J}_\beta(\boldsymbol{\theta}_0)\\
			\boldsymbol{m}^T
		\end{pmatrix} \text{IF}\left(\boldsymbol{n}, \widetilde{\boldsymbol{T}}_\beta, F_{\boldsymbol{\theta}_0} \right)
		= \begin{pmatrix}
			\boldsymbol{W}^T \boldsymbol{D}_{\boldsymbol{\pi}(\boldsymbol{\theta}_0)}^{\beta-1}\left(-\boldsymbol{\pi}(\boldsymbol{\theta}_0)+\Delta_{\boldsymbol{n}} \right) \\
			0
		\end{pmatrix}.
	\end{equation*}
	Now, multiplying both terms by $ \left(\boldsymbol{J}_\beta(\boldsymbol{\theta})^T, \boldsymbol{m} \right)$ and inverting in both sizes of the equation, the expression of the IF of the restricted MDPDE is given by
	\begin{equation}\label{eq:IFrestricted}
		\text{IF}\left(\boldsymbol{n}, \widetilde{\boldsymbol{T}}_\beta, F_{\boldsymbol{\theta}_0}\right)
		= \left(\boldsymbol{J}_\beta(\boldsymbol{\theta}_0)^T\boldsymbol{J}_\beta(\boldsymbol{\theta}_0) + \boldsymbol{m}\boldsymbol{m}^T \right)^{-1}
		\boldsymbol{J}_\beta(\boldsymbol{\theta}_0)^T
		\boldsymbol{W}^T \boldsymbol{D}_{\boldsymbol{\pi}(\boldsymbol{\theta}_0)}^{\beta-1}\left(-\boldsymbol{\pi}(\boldsymbol{\theta}_0)+\Delta_{\boldsymbol{n}} \right).
	\end{equation}
	Since the matrix $\left(\boldsymbol{J}_\beta(\boldsymbol{\theta}_0)^T\boldsymbol{J}_\beta(\boldsymbol{\theta}_0) + \boldsymbol{m}\boldsymbol{m}^T\right)^{-1}\boldsymbol{J}_\beta(\boldsymbol{\theta}_0)^T$ is typically assumed to be bounded, the robustness of the restricted MDPDE depends only on the boundedness of the IF of the (unrestricted) MDPDE. Therefore, restricted MDPDE are robust for all type of outliers when using positives values of $\beta$, whereas the restricted MLE (corresponding to $\beta=0$) lacks of robustness against stress level or inspection times contamination, i.e., bad leverage points.

	\section{Applications of the restricted MDPDE}

	An interesting application of the restricted MDPDE are robust testing procedures based on the DPD for testing linear null hypothesis of the form
	\begin{equation}\label{eq:null}
		\operatorname{H}_0: \boldsymbol{m}^T\boldsymbol{\theta} = d.
	\end{equation}
	with $\boldsymbol{m} \in \mathbb{R}^2$ and $d\in \mathbb{R}.$
	In this section, we develop two families of test statistics based on the DPD for testing (\ref{eq:null}) for one-shot devices under the step-stress ALT model, namely Rao-type test statistics and DPD-based tests statistics.
	These two families were studied for general statistical models in Basu et al. (2018).
 	Let consider $\widetilde{\boldsymbol{\theta}}^\beta$ the restricted MDPDE with restricted parameter space defined by the null hypothesis in (\ref{eq:null}),
	$$\Theta_0 = \{\boldsymbol{\theta} | \hspace{0.2cm} \boldsymbol{m}^T\boldsymbol{\theta} = d\}$$
	and recall   $\widehat{\boldsymbol{\theta}}^\beta$ denotes the MDPDE  for $\boldsymbol{\theta}$  computed in all parameter space.

	\subsection{Rao-type tests statistics}
	Let us consider the score of the DPD loss function for the step-stress ALT model
	\begin{equation}
		\label{eq:score}
		\boldsymbol{U}_{\beta,N}(\boldsymbol{\theta}) = \boldsymbol{W}^T\boldsymbol{D}_{\boldsymbol{\pi}(\boldsymbol{\theta})}^{\beta-1}(\widehat{\boldsymbol{p}}-\boldsymbol{\pi}(\boldsymbol{\theta}))
	\end{equation}
	where matrices $\boldsymbol{W}$ and $\boldsymbol{D}_{\boldsymbol{\pi}(\boldsymbol{\theta})}$ are defined in Section \ref{sec:modelformulation}.	That is, the MDPDE verifies the estimating equations given by
	$$	\boldsymbol{U}_{\beta,N}(\widehat{\boldsymbol{\theta}}^\beta) = \boldsymbol{0}$$

	We define Rao-type test statistics for testing linear null hypothesis (\ref{eq:null}) as
	\begin{definition}
		The Rao-type statistics, based on the restricted to the linear null hypothesis (\ref{eq:null}) MDPDE, $\widetilde{\boldsymbol{\theta}}^\beta,$ for testing (\ref{eq:null}) is given by
		\begin{equation}\label{eq:raotest}
			\boldsymbol{R}_{\beta, N}(\widetilde{\boldsymbol{\theta}}^\beta) = N\boldsymbol{U}_{\beta,N}(\widetilde{\boldsymbol{\theta}}^\beta)^T \boldsymbol{Q}_{\beta}(\widetilde{\boldsymbol{\theta}}^\beta)\left[ \boldsymbol{Q}_{\beta}(\widetilde{\boldsymbol{\theta}}^\beta)^T \boldsymbol{K}_{\beta}(\widetilde{\boldsymbol{\theta}}^\beta)
			\boldsymbol{Q}_{\beta}(\widetilde{\boldsymbol{\theta}}^\beta)\right]^{-1}
			\boldsymbol{Q}_{\beta}(\widetilde{\boldsymbol{\theta}}^\beta)^T \boldsymbol{U}_{\beta,N}(\widetilde{\boldsymbol{\theta}}^\beta),
		\end{equation}
		where matrices $\boldsymbol{K}_{\beta}(\boldsymbol{\theta})$, $\boldsymbol{Q}_{\beta}(\boldsymbol{\theta})$
		and $\boldsymbol{U}_{\beta,N}(\boldsymbol{\theta}),$  is defined in (\ref{eq:JK}), (\ref{eq:SigmaPQmatrices}) and (\ref{eq:score}), respectively.
	\end{definition}
	Here, the matrix $\boldsymbol{Q}_{\beta}(\boldsymbol{\theta})$ depends on the null hypothesis trough $\boldsymbol{m},$ and the term $d$ is only used to obtain the restricted MDPDE.
	Moreover, if $\boldsymbol{m} = (0,1)$ (simple null hypothesis) then the restricted estimate of $\theta_1$ must be necessarily $d.$

	Before presenting the asymptotic distribution of the Rao-type test statistics, $\boldsymbol{R}_{\beta, N}(\widetilde{\boldsymbol{\theta}}^\beta),$ we shall establish the asymptotic distribution of the score  $\boldsymbol{U}_{\beta,N}(\widetilde{\boldsymbol{\theta}}^\beta).$
	\begin{theorem}\label{thm:asympscore}
		The asymptotic distribution of the score $\boldsymbol{U}_{\beta,N}(\widetilde{\boldsymbol{\theta}}^\beta)$ for the step-stress ALT model under exponential lifetimes, is given by
		$$\sqrt{N}\boldsymbol{U}_{\beta,N}(\widetilde{\boldsymbol{\theta}}^\beta) \xrightarrow[N\rightarrow \infty]{L} \mathcal{N}\left(\boldsymbol{0}, \boldsymbol{K}_\beta(\boldsymbol{\theta})\right)$$
		where the variance-covariance matrix $\boldsymbol{K}_\beta(\boldsymbol{\theta})$ is defined in (\ref{eq:JK}).
	\end{theorem}
	\begin{proof}
		It is well known that $$\sqrt{N}(\widehat{\boldsymbol{p}}-\boldsymbol{\pi}(\boldsymbol{\theta})) \xrightarrow[N\rightarrow \infty]{L} \mathcal{N}\left(\boldsymbol{0}, \boldsymbol{D}_{\boldsymbol{\pi}(\boldsymbol{\theta})} - \boldsymbol{\pi}(\boldsymbol{\theta}) \boldsymbol{\pi}(\boldsymbol{\theta})^T\right).$$
		since $\widehat{\boldsymbol{p}}$ is the MLE of the multinomial model, and $\boldsymbol{D}_{\boldsymbol{\pi}(\boldsymbol{\theta})} - \boldsymbol{\pi}(\boldsymbol{\theta}) \boldsymbol{\pi}(\boldsymbol{\theta})^T$ is the inverse of the Fisher information matrix of that model.

		Therefore, the score $\sqrt{N}\boldsymbol{U}_{\beta,N}(\boldsymbol{\theta})$ is asymptotically normal with mean vector
		$$\mathbb{E}\left[ \boldsymbol{U}_{\beta,N}(\boldsymbol{\theta}) \right] =\mathbb{E}\left[ \boldsymbol{W}^T\boldsymbol{D}_{\boldsymbol{\pi}(\boldsymbol{\theta})}^{\beta-1}(\widehat{\boldsymbol{p}}-\boldsymbol{\pi}(\boldsymbol{\theta})) \right] = \boldsymbol{0}$$
		and variance-covariance matrix
		\begin{align*}
			\text{Cov}\left[\boldsymbol{U}_{\beta,N}(\boldsymbol{\theta})\right] &=  \boldsymbol{W}^T\boldsymbol{D}_{\boldsymbol{\pi}(\boldsymbol{\theta})}^{\beta-1}\text{Cov}\left[\widehat{\boldsymbol{p}}-\boldsymbol{\pi}(\boldsymbol{\theta})\right]\boldsymbol{D}_{\boldsymbol{\pi}(\boldsymbol{\theta})}^{\beta-1} \boldsymbol{W}\\
			&= \boldsymbol{W}^T\boldsymbol{D}_{\boldsymbol{\pi}(\boldsymbol{\theta})}^{\beta-1}\left[\boldsymbol{D}_{\boldsymbol{\pi}(\boldsymbol{\theta})} - \boldsymbol{\pi}(\boldsymbol{\theta}) \boldsymbol{\pi}(\boldsymbol{\theta})^T\right]\boldsymbol{D}_{\boldsymbol{\pi}(\boldsymbol{\theta})}^{\beta-1} \boldsymbol{W}\\
			&=  \boldsymbol{W}^T\left[\boldsymbol{D}_{\boldsymbol{\pi}(\boldsymbol{\theta})}^{2\beta-1} - \boldsymbol{D}_{\boldsymbol{\pi}(\boldsymbol{\theta})}^{\beta-1}\boldsymbol{\pi}(\boldsymbol{\theta}) \boldsymbol{\pi}(\boldsymbol{\theta})^T\boldsymbol{D}_{\boldsymbol{\pi}(\boldsymbol{\theta})}^{2\beta-1}\right] \boldsymbol{W}\\
			&=  \boldsymbol{W}^T\left[\boldsymbol{D}_{\boldsymbol{\pi}(\boldsymbol{\theta})}^{2\beta-1} - \boldsymbol{\pi}(\boldsymbol{\theta})^\beta \boldsymbol{\pi}(\boldsymbol{\theta})^{\beta T}\right] \boldsymbol{W}\\
			&= \boldsymbol{K}_\beta(\boldsymbol{\theta}).

		\end{align*}

	\end{proof}

	Now, the following results states the asymptotic distribution of the Rao-type test statistics
	\begin{theorem}\label{thm:asymprao}
		The asymptotic distribution of the Rao-type test statistics defined in (\ref{eq:raotest}) under the linear null hypothesis (\ref{eq:null}) is a chi-square with 1 degree of freedom.
	\end{theorem}
	\begin{proof}
		The MDPDE restricted to the null hypothesis (\ref{eq:null}), $\widetilde{\boldsymbol{\theta}}^\beta,$
		is defined as the minimum of the DPD loss restricted to the condition $\boldsymbol{m}^T\boldsymbol{\theta} =d$. Then, it satisfies the restricted equations
		\begin{equation}\label{eq:restrictedestimating}
			\widetilde{\boldsymbol{U}}_{\beta,N}(\widetilde{\boldsymbol{\theta}}^\beta) + \boldsymbol{m}\widetilde{\boldsymbol{\lambda}} = \boldsymbol{0},
		\end{equation}
		for some vector $\widetilde{\boldsymbol{\lambda}}$ of Lagrangian multipliers.
		Then, we can write
		$\boldsymbol{U}_{\beta,N}(\widetilde{\boldsymbol{\theta}}^\beta) = -\boldsymbol{m}\widetilde{\boldsymbol{\lambda}}
		$
		and consequently
		$$
		\boldsymbol{U}_{\beta,N}(\widetilde{\boldsymbol{\theta}}^\beta)^T \boldsymbol{Q}_{\beta}(\widetilde{\boldsymbol{\theta}}^\beta) = -\widetilde{\boldsymbol{\lambda}}\boldsymbol{m}^T\boldsymbol{Q}_{\beta}(\widetilde{\boldsymbol{\theta}}^\beta) = - \boldsymbol{\widetilde{\lambda}}
		$$
		where $\boldsymbol{Q}_{\beta}(\widetilde{\boldsymbol{\theta}}^\beta)$ is defined in (\ref{eq:SigmaPQmatrices}). Further, the Rao-type test statistics defined in (\ref{eq:raotest}) can be computed in terms of the vector of Lagrange multipliers as follows,
		\begin{equation*}
			\boldsymbol{R}_{\beta, N}(\boldsymbol{\theta}) = N\boldsymbol{\widetilde{\lambda}}^T\left[ \boldsymbol{Q}_{\beta}(\widetilde{\boldsymbol{\theta}}^\beta)^T \boldsymbol{K}_{\beta}(\widetilde{\boldsymbol{\theta}}^\beta)
			\boldsymbol{Q}_{\beta}(\widetilde{\boldsymbol{\theta}}^\beta)\right]^{-1}
			\boldsymbol{\widetilde{\lambda}}.
		\end{equation*}
		In order to obtain the asymptotic distribution of the Rao-type test statistics, we first derive the asymptotic distribution of the vector of Lagrangian multipliers $\widetilde{\boldsymbol{\lambda}}.$
		We  consider the second order Taylor expansion series of the score function $\boldsymbol{U}_{\beta,N}(\boldsymbol{\theta})$ around the true parameter value $\boldsymbol{\theta}_0,$
		$$\boldsymbol{U}_{\beta,N}(\widetilde{\boldsymbol{\theta}}^\beta) = \boldsymbol{U}_{\beta,N}(\boldsymbol{\theta}_0) + \frac{\partial \boldsymbol{U}_{\beta,N}(\boldsymbol{\theta})}{\partial \boldsymbol{\theta}}\bigg|_{\boldsymbol{\theta} = \boldsymbol{\theta}_0}\left(\widetilde{\boldsymbol{\theta}}^\beta- \boldsymbol{\theta}_0\right) + o\left(||\widetilde{\boldsymbol{\theta}}^\beta- \boldsymbol{\theta}_0||^2\boldsymbol{1}_2\right).$$
		On the other hand, since $\widehat{p} \xrightarrow[N\rightarrow \infty]{P} \boldsymbol{\pi}(\boldsymbol{\theta}_0)$, it is not difficult to show that
		$$\frac{\partial \boldsymbol{U}_{\beta,N}(\boldsymbol{\theta})}{\partial \boldsymbol{\theta}}\bigg|_{\boldsymbol{\theta} = \boldsymbol{\theta}_0} \xrightarrow[N\rightarrow \infty]{P} - \boldsymbol{J}_\beta\left(\boldsymbol{\theta}_0\right),$$
		where the matrix $\boldsymbol{J}_\beta\left(\boldsymbol{\theta}\right)$ is defined in (\ref{eq:JK}).
		Therefore, we can approximate the score function at the restricted MDPDE by
		$$\boldsymbol{U}_{\beta,N}(\widetilde{\boldsymbol{\theta}}^\beta) = \boldsymbol{U}_{\beta,N}(\boldsymbol{\theta}_0) - \boldsymbol{J}_\beta\left(\boldsymbol{\theta}_0\right) \left(\widetilde{\boldsymbol{\theta}}^\beta- \boldsymbol{\theta}_0\right) + o\left(||\widetilde{\boldsymbol{\theta}}^\beta- \boldsymbol{\theta}_0||^2\boldsymbol{1}_2\right)+o\left( \boldsymbol{1}_2\right).$$
		Based on (\ref{eq:restrictedestimating}), we have that
		$$
		\boldsymbol{U}_{\beta,N}(\boldsymbol{\theta}_0) - \boldsymbol{J}_\beta\left(\boldsymbol{\theta}_0\right) \left(\widetilde{\boldsymbol{\theta}}^\beta- \boldsymbol{\theta}_0\right)  + \boldsymbol{m}\widetilde{\boldsymbol{\lambda}}= o\left( \boldsymbol{1}_2\right)
		$$
		so under the null hypothesis we can write,
		$$\boldsymbol{m}^T\widetilde{\boldsymbol{\theta}}^\beta - d = \boldsymbol{m}^T\left(\widetilde{\boldsymbol{\theta}}^\beta - \boldsymbol{\theta}_0\right)  = 0. $$
		Joining both equations,  we get
		\begin{equation*}
			\begin{pmatrix}
				-\boldsymbol{J}_\beta(\boldsymbol{\theta}_0) & \boldsymbol{m}\\
				\boldsymbol{m}^T & \boldsymbol{0}
			\end{pmatrix}
			\begin{pmatrix}
				\widetilde{\boldsymbol{\theta}}^\beta - \boldsymbol{\theta}_0\\
				\widetilde{\boldsymbol{\lambda}}
			\end{pmatrix} =
			\begin{pmatrix}
				-\boldsymbol{U}_{\beta,N}(\boldsymbol{\theta}_0)\\
				0
			\end{pmatrix} + 	\begin{pmatrix}
				o(\boldsymbol{1}_2)\\
				0
			\end{pmatrix}
		\end{equation*}
		and solving the previous equation, we have that
		\begin{equation*}
			\begin{pmatrix}
				\widetilde{\boldsymbol{\theta}}^\beta - \boldsymbol{\theta}_0\\
				\widetilde{\boldsymbol{\lambda}}
			\end{pmatrix} =
			\begin{pmatrix}
				-\boldsymbol{J}_\beta(\boldsymbol{\theta}_0) & \boldsymbol{m}\\
				\boldsymbol{m}^T & \boldsymbol{0}
			\end{pmatrix}^{-1}
			\begin{pmatrix}
				-\boldsymbol{U}_{\beta,N}(\boldsymbol{\theta}_0)\\
				\boldsymbol{0}
			\end{pmatrix}  + 	\begin{pmatrix}
				o(\boldsymbol{1}_2)\\
				0
			\end{pmatrix} .
		\end{equation*}
		Now, computing the inverse matrix
		\begin{equation*}
			\begin{pmatrix}
				-\boldsymbol{J}_\beta(\boldsymbol{\theta}_0) & \boldsymbol{m}\\
				\boldsymbol{m}^T & \boldsymbol{0}
			\end{pmatrix}^{-1} =
			\begin{pmatrix}
				\boldsymbol{P}_\beta(\boldsymbol{\theta}_0) & \boldsymbol{Q}_\beta(\boldsymbol{\theta}_0)\\
				\boldsymbol{Q}_\beta(\boldsymbol{\theta}_0)^T & \left(\boldsymbol{m}^T\boldsymbol{J}_\beta(\boldsymbol{\theta}_0)^{-1}\boldsymbol{m}\right)^{-1}
			\end{pmatrix}
		\end{equation*}

		with $\boldsymbol{P}_\beta(\boldsymbol{\theta}_0)$ and $\boldsymbol{Q}_\beta(\boldsymbol{\theta}_0)$ defined in (\ref{eq:SigmaPQmatrices}). But from Theorem \ref{thm:asympscore},
		\begin{equation*}
			\begin{pmatrix}
				\sqrt{N} \boldsymbol{U}_{\beta,N}(\boldsymbol{\theta}_0)\\
				\boldsymbol{0}
			\end{pmatrix} \xrightarrow[N\rightarrow \infty]{L} \mathcal{N}\left(\boldsymbol{0}_{3}, \begin{pmatrix}
				\boldsymbol{K}_\beta(\boldsymbol{\theta}_0) & \boldsymbol{0}\\
				\boldsymbol{0}^T & 0
			\end{pmatrix}\right)
		\end{equation*}
		and hence,
		\begin{equation*}
			\begin{pmatrix}
				\sqrt{N}(\widetilde{\boldsymbol{\theta}}^\beta - \boldsymbol{\theta}_0)\\
				\sqrt{N}\widetilde{\boldsymbol{\lambda}}
			\end{pmatrix} \xrightarrow[N\rightarrow \infty]{L} \mathcal{N}\left(\boldsymbol{0}_{3},
			\boldsymbol{V}_\beta(\boldsymbol{\theta}_0) \right)
		\end{equation*}
		with
		\begin{equation*}
			\boldsymbol{V}_\beta(\boldsymbol{\theta}_0) = \begin{pmatrix}
				\boldsymbol{P}_\beta(\boldsymbol{\theta}_0) & \boldsymbol{Q}_\beta(\boldsymbol{\theta}_0)\\
				\boldsymbol{Q}_\beta(\boldsymbol{\theta}_0)^T & \left(\boldsymbol{m}^T\boldsymbol{J}_\beta(\boldsymbol{\theta}_0)^{-1}\boldsymbol{m}\right)^{-1}
			\end{pmatrix}
			\begin{pmatrix}
				\boldsymbol{K}_\beta(\boldsymbol{\theta}_0) & \boldsymbol{0}\\
				\boldsymbol{0}^T & 0
			\end{pmatrix}
			\begin{pmatrix}
				\boldsymbol{P}_\beta(\boldsymbol{\theta}_0) & \boldsymbol{Q}_\beta(\boldsymbol{\theta}_0)\\
				\boldsymbol{Q}_\beta(\boldsymbol{\theta}_0)^T & \left(\boldsymbol{m}^T\boldsymbol{J}_\beta(\boldsymbol{\theta}_0)^{-1}\boldsymbol{m}\right)^{-1}
			\end{pmatrix}.
		\end{equation*}
		Thus, the asymptotic distribution of the vector of Lagrangian multipliers is given by
		\begin{equation*}\label{eq:asymplambda}
			\sqrt{N} \widetilde{\boldsymbol{\lambda}}
			\xrightarrow[N\rightarrow \infty]{L} \mathcal{N}\left(0,
			\boldsymbol{Q}_\beta(\boldsymbol{\theta}_0)^T \boldsymbol{K}_\beta(\boldsymbol{\theta}_0)\boldsymbol{Q}_\beta(\boldsymbol{\theta}_0) \right).
		\end{equation*}
		Using the previous convergence and the consistency of the restricted MDPDE, it follows that the asymptotic distribution of the Rao-type test statistics, $\boldsymbol{R}_{\beta, N}(\widetilde{\boldsymbol{\theta}}^\beta),$
		is a chi-square distribution with 1 degree of freedom.

	\end{proof}

	Based on Theorem \ref{thm:asymprao}, for any $\beta \geq 0$ and $\boldsymbol{m} \in \mathbb{R}^2,$ the critical region with significance level $\alpha$ for the hypothesis test with null hypothesis (\ref{eq:null}) is given by
	\begin{equation}\label{eq:criticalregionRaotest}
		\mathcal{R}_{\alpha} = \{(n_1,...,n_{L+1})  \text{ s.t. } 	\boldsymbol{R}_{\beta, N}(\boldsymbol{\theta}) > \chi^2_{1,\alpha}\}
	\end{equation}
	where $\chi^2_{1,\alpha}$ denotes the lower $\alpha$-quantile of a chi-square with 1 degree of freedom.

	\begin{remark}
		We could extent the scope to more general linear null hypothesis,
		$$\operatorname{H}_0: \boldsymbol{M}^T\boldsymbol{\theta} = \boldsymbol{d},$$
		with $\boldsymbol{M}$ a $2\times2$ matrix of range $2$ and $\boldsymbol{d}\in \mathbb{R}^2.$ However, in this case the restricted MDPDE is explicitly determinate by the constraint $\boldsymbol{M}^T\boldsymbol{\theta} = \boldsymbol{d}.$

		Consequently, the Rao-type test statistics is completely defined by the null hypothesis and it is independent of the value of $\beta$.
		In particular, if we define $\boldsymbol{M}$ to be the identity matrix, we get the simple null hypothesis
		$$\operatorname{H}_0: (\theta_0, \theta_1) = (d_0,d_1),$$
		and the associated Rao-type test is given by
		\begin{equation*}
			\boldsymbol{R}_{\beta, N}^\ast(\boldsymbol{d}) = N\boldsymbol{U}_{\beta,N}(\boldsymbol{d})^T  \boldsymbol{K}_{\beta}(\boldsymbol{d})^{-1}
			\boldsymbol{U}_{\beta,N}(\boldsymbol{d}),
		\end{equation*}
		since $\boldsymbol{Q}_{\beta}(\boldsymbol{d})$ is the identity matrix.
		Following similar steps than in Theorem \ref{thm:asymprao}, it is not difficult to establish that
		$$ \boldsymbol{R}_{\beta, N}^\ast(\boldsymbol{d}) \xrightarrow[N \rightarrow \infty]{L} \chi^2_2.$$
		Thus, for any $\beta \geq 0,$  the critical region with significance level $\alpha$ for the  hypothesis test with simple null hypothesis is given by
		\begin{equation*}
			\mathcal{R}_{\alpha} = \{(n_1,...,n_{L+1})  \text{ s.t. } 	\boldsymbol{R}_{\beta, N}^\ast(\boldsymbol{d}) > \chi_{2,\alpha}\},
		\end{equation*}
		where $\chi_{2,\alpha}$ denotes the lower $\alpha$-quantile of a chi-square with 2 degree of freedom.
	\end{remark}

	\subsection{DPD-based statistics}

	We develop a class of test statistics based on the DPD between the model evaluated under the null hypothesis and under the whole parameter space, respectively.
	Let $\boldsymbol{\pi}(\widehat{\boldsymbol{\theta}}^\beta)$ and $\boldsymbol{\pi}(\widetilde{\boldsymbol{\theta}}^\beta)$ denote the probability vector of the multinomial model estimated in the whole parameter space and the restricted parameter space, respectively. The DPD for $\tau > 0 $ between these two mass function is given by
	 	\begin{equation}\label{eq:DPDtest}
	 	d_\tau(\widehat{\boldsymbol{\theta}}^\beta, \widetilde{\boldsymbol{\theta}}^\beta) = d_\tau(\boldsymbol{\pi}(\widehat{\boldsymbol{\theta}}^\beta), \boldsymbol{\pi}(\widetilde{\boldsymbol{\theta}}^\beta))
 		= \sum_{i=1}^{L+1}\left(\pi_j(\widehat{\boldsymbol{\theta}}^\beta)^{1+\tau} + \left(1+\frac{1}{\tau}\right)\pi_j(\widehat{\boldsymbol{\theta}}^\beta) \pi_j(\widetilde{\boldsymbol{\theta}}^\beta)^\tau + \frac{1}{\tau}\pi_j(\widetilde{\boldsymbol{\theta}}^\beta)^{\tau+1}\right).
	 	\end{equation}

	 Note that here we should deal with two different tuning parameters controlling the trade-off between robustness and efficiency in the estimation ($\beta$) and in the DPD between the estimated mass functions ($\tau$). Many authors choose $\tau = \beta$ for the seek of simplicity.
	 If the true parameter $\boldsymbol{\theta}_0$ verifies the null hypothesis, then the distance in (\ref{eq:DPDtest}) must near zero. Then, the reject region for the test with null hypothesis (\ref{eq:null}) is given by
	 $$ RC= \{ (n_1,...,n_{L+1}) s.t.\hspace{0.2cm} d_\tau(\widehat{\boldsymbol{\theta}}^\beta, \widetilde{\boldsymbol{\theta}}^\beta)  > k\}$$
	with $k$ a constant computed such that the test has size $\alpha.$
	The exact distribution of the test statistic $d_\tau(\widehat{\boldsymbol{\theta}}^\beta, \widetilde{\boldsymbol{\theta}}^\beta)$
	is not easy to get, but using similar arguments as in Jaenada et al. (2022) is it not difficult to establish that
	$$T_\tau(\widehat{\boldsymbol{\theta}}^\beta, \widetilde{\boldsymbol{\theta}}^\beta) = 2n d_\tau(\widehat{\boldsymbol{\theta}}^\beta, \widetilde{\boldsymbol{\theta}}^\beta)$$ is asymptotically distributed, under the null hypothesis, as a linear combination of chi-squared random variables with a degree of freedom.
	Furthermore, asymptotic results under some particular kind of alternative hypothesis, such as contiguous alternative hypothesis, may be also of interest.

\section{Simulation study}\label{sec:simstudy}

We empirically analyze the performance of the restricted MDPDEs for one-shot devices data under the step-stress model with exponential lifetime  distributions, $\widetilde{\boldsymbol{\theta}}^\beta.$
Further, we evaluate their robustness properties under different scenarios of  contamination.

For the multinomial model, we should consider ``outlying cells'' rather than  ``outlying devices''. Then, we introduce contamination by  increasing (or decreasing) the probability of failure in (\ref{eq:th.prob}) for (at least) one interval (i.e., one cell).
The probability of failure is switched in such contaminated cells as
\begin{equation} \label{eq:contaminatedprob}
	\tilde{\pi}_j(\boldsymbol{\theta}) =  G_{\boldsymbol{\theta}}(\textit{IT}_j) - G_{\boldsymbol{\tilde{\theta}}}(\textit{IT}_{j-1})
\end{equation}
for some $j=2,...,L$, where $\boldsymbol{\tilde{\theta}} = (\widetilde{\theta}_0,\widetilde{\theta}_1)$ is a contaminated parameter with $\tilde{\theta}_0 \leq \theta_0$  and $\tilde{\theta}_1 \leq \theta_1.$
The resulting probability vector must normalized after introducing contamination.

Let us consider a 2-step stress ALT experiment with $L=11$ inspection times and a total of $N = 180$ one-shot devices.
The devices are tested at two stress levels,  $x_1=35$ and $x_2=45$. The first stress level is maintained from the beginning of the experiment until $\tau_1 = 25,$ and the experiment ends at $\tau_2=70.$
 During the experiment, devices inspection is performed at a grid of inspection times, containing the times of stress change, $\text{IT}=(10,15,20,25,30,35,40,45,50,60,70).$
 At each inspection time, all surviving units under test are examined.
 We set the true value of the parameter $\boldsymbol{\theta}_0=(0.003,0.03),$ and then generate data from the corresponding
 multinomial model described in Section \ref{sec:modelformulation}.
  Additionally, we contaminate the described model by increasing the probability of failure in the third interval following (\ref{eq:contaminatedprob}), with $\widetilde{\boldsymbol{\theta}} = (\varepsilon\theta_0, \theta_1)$ for the first scenario of contamination and $\widetilde{\boldsymbol{\theta}} = (\theta_0, \varepsilon\theta_1)$ for the second scenario of contamination.
Note that in both scenarios, the mean lifetime is decreased for the outlying cell.
Further, we consider the linearly restricted parameter space of the form
\begin{equation*}
	\Theta_0 = \{\theta \in \Theta | \hspace{0.2cm} \theta_1 = d \}
\end{equation*}
with $d \in \mathbb{R}.$
We evaluate the performance of the restricted estimators when the true parameter verifies the subspace constraints as well as when $\boldsymbol{\theta}_0$ does not belong to the restricted subspace $\Theta_0.$
For the first scenario ($\boldsymbol{\theta}_0 \in \Theta_0$) we set $d=0.03$ and for the second one ($\boldsymbol{\theta}_0 \notin \Theta_0$) we set $d=0.027.$ Note that, in the second case, the true parameter is close to the restricted space.

Figure \ref{fig:restricted} shows the mean squared error (MSE) of the restricted MDPDE when the true parameter value satisfies the subspace restrictions over $R=10.000$ repetitions, for the two scenarios of contamination and different values of $\beta.$ Larger values of the tuning parameter produces more robust estimates, although less efficient. From this empirical results, moderately large values of the tuning parameter $\beta$ over 0.4 could produce the best trade-off between efficiency and robustness.
Similarly, Figure  \ref{fig:restrictedd} shows the MSE of the restricted MDPDE for similar scenarios, when the true parameter value does not belong to the restricted parameter space. Similar conclusions are drawn, but the MSE on the estimation increases considerably in this context.

\begin{figure}[htb]

	\begin{subfigure}{0.5\textwidth}
		\includegraphics[scale=0.4]{images/restrictedCONT0}
		\subcaption{$\theta_0$-contaminated cell}
	\end{subfigure}
	\begin{subfigure}{0.5\textwidth}
		\includegraphics[scale=0.4]{images/restrictedCONT1}
		\subcaption{$\theta_1$-contaminated cell}
	\end{subfigure}
	\caption{ MSE of the restricted MDPDE when the true parameter belongs to the restricted space over $R=10.000$ repetitions}
\label{fig:restricted}
\end{figure}

\begin{figure}[htb]

	\begin{subfigure}{0.5\textwidth}
		\includegraphics[scale=0.4]{images/restrictedCONT0_d027}
		\subcaption{$\theta_0$-contaminated cell}
	\end{subfigure}
	\begin{subfigure}{0.5\textwidth}
		\includegraphics[scale=0.4]{images/restrictedCONT1_d027}
		\subcaption{$\theta_1$-contaminated cell}
	\end{subfigure}
	\caption{ MSE of the restricted MDPDE when the true parameter does not belong to the restricted space over $R=10.000$ repetitions}
	\label{fig:restrictedd}
\end{figure}

Finally, we examine the performance of the Rao-type test statistics against the sample size $N$ in three contamination scenarios (Figure \ref{fig:raolevelandpowerN}), in the absence of contamination (top), $\theta_0$-contaminated third cell with a $40\%$ reduction of the parameter (middle) and $\theta_1$-contaminated third cell with a $40\%$ reduction of the parameter (bottom). Again, we found a similar performance of the Rao-type test statistics based on the restricted MDPDE with different values of the tuning parameter $\beta,$ whereas positives values of $\beta$ produce more robust statistics when introducing contamination in either of the model parameters, $\theta_0$ and $\theta_1$. Moreover, the lack of robustness of the Rao-type test statistics based on the restricted MLE is highlighted with large sample sizes, in both empirical level and power.

\begin{figure}[htb]
	\begin{subfigure}{0.5\textwidth}
		\includegraphics[height=7cm, width=8cm]{images/raoLEVELagainstN0}
		\subcaption{Absence of contamination}
	\end{subfigure}
	\begin{subfigure}{0.5\textwidth}
		\includegraphics[height=7cm, width=8cm]{images/raoPOWERagainstN1}
		\subcaption{Absence of contamination}
	\end{subfigure}
	\begin{subfigure}{0.5\textwidth}
		\includegraphics[height=7cm, width=8cm]{images/raoLEVELagainstNcont0}
		\subcaption{$\theta_0$-contaminated cell}
	\end{subfigure}
	\begin{subfigure}{0.5\textwidth}
		\includegraphics[height=7cm, width=8cm]{images/raoPOWERagaisntNcont0}
		\subcaption{$\theta_0$-contaminated cell}
	\end{subfigure}
	\begin{subfigure}{0.5\textwidth}
		\includegraphics[height=7cm, width=8cm]{images/raoLEVELagainstNcont1}
		\subcaption{$\theta_1$-contaminated cell}
	\end{subfigure}
	\begin{subfigure}{0.5\textwidth}
		\includegraphics[height=7cm, width=8cm]{images/raoPOWERagainstNcont1}
		\subcaption{$\theta_1$-contaminated cell}
	\end{subfigure}
	\caption{Empirical level and power against sample size with different contaminated scenarios in $R=1000$ replications}
	\label{fig:raolevelandpowerN}
\end{figure}

\section{Conclusions}

In this paper with have developed the restricted minimum MDPDE for one-shot devices data tested under step-stress ALTs. We have derived its asymptotic distribution and analyzed its robustness properties through its IF.
Further, we have defined two families of robust testing procedures based on the restricted DPD estimators, Rao-type test statistics and DPD-based statistics, and we have established the asymptotic distribution for the first one.
Finally, all properties stated theoretically have been illustrated empirically through simulation, showing the certain advantage in terms of robustness of DPD-based inference methods.

The robustness of the the two proposed families of tests statistics would be interesting to study theoretically through their IF analysis, and furthermore, empirical performance comparison of these two families, together with well-known Wald-type test statistics based on the DPD for one-shot devices tested under step-stress ALT model, will be interesting to be examined in future researches.


\section{Introduction}
\label{sec:intro}
Image inpainting, also known as image completion, has always been regarded as a challenge to fill the missing area of the image.
Image inpainting can deal with various problems encountered in the real world, such as removing objects in photos, repairing damaged photos \cite{wan2020bringing} or expanding photos.
At the same time, image inpainting needs to maintain the coordination and semantic consistency between the repaired area and remaining parts of the image.
Therefore, image inpainting also calls for strong generation ability. Nowadays, it has become a fundamental research topic in the field of computer vision and image processing society.

\begin{figure}[!t]
    \centering
    \includegraphics[width=0.98\linewidth]{figure/4.pdf}
    \figcaption{Comparison with the LaMa\cite{9707077}, GLaMa and two-stage GLaMa for face Inpainting over some mask settings on FFHQ\cite{DBLP:journals/corr/abs-1812-04948} dataset in $1024\times1024$ resolution.}
    \label{Figure 4}
\end{figure}

Thanks to the fast development of deep learning, the recovered results of these deep learning-based image inpainting approaches are getting better and better.
Recent years, most state-of-the-art approaches are mainly based on convolutional neural networks or transformer.
In the approaches of \cite{9010689, Zeng2022AggregatedCT, Liu2020RethinkingII, zhao2021comodgan}, they apply the convolutional neural networks for image inpainting, while other line of research \cite{wan2021high, yu2021diverse} leverages the transformer in image inpainting at the low-resolution image space, and then introduces the GAN based networks for high quality image generation.
Suvorov et al. \cite{9707077} utilize the Fast Fourier Convolution (FFC) instead of regular convolution to obtain features of global receptive fields in frequency domain.
Most of the current work based on the semantic consistency (with the surrounding areas) can handle the ``background completion'' or ``object removing'' task very well.

Generally, these methods can effectively deal with some common image inpainting tasks.
However, they still face some challenges.
The trained model needs to be able to deal with degraded image with various forms of mask, such as thin strokes, rectangle, and even extreme masks, because we do not know what image degradation process we will encounter.
This is a challenging task because most of methods use specific masks for training, which may acquire poor results when meeting other masks that does not appear in the training processes.
Recently, Andreas et al. \cite{DBLP:journals/corr/abs-2201-09865} address the above issues. Some other diffusion models \cite{Song2021ScoreBasedGM, SohlDickstein2015DeepUL} can also deal with this challenge.
However, it does not mean that the models based on convolutional neural networks (CNNs) or transformer cannot solve this problem well.
In this work, we use more kinds of mask to enhance the robustness of the model.
As shown in Fig. \ref{Figure 3} and \ref{Figure 5}, our method can generate more realistic images.

\begin{figure}[!t]
    \centering
    \includegraphics[width=0.98\linewidth]{figure/Overview.pdf}
    \figcaption{Overview of our proposed GLaMa.}
    \label{Figure 2}
\end{figure}

When we visualize the spectra of real and fake images, we find that the difference between spectra of real pictures and fake images is obvious as shown in Fig. \ref{Figure 6}. It can be seen that there are many obvious errors in the spectra of the images generated by LaMa\cite{9707077}.
In addition, by looking closely, we can find that there will be many distinctive checkerboard effects and ripples on the images generated by LaMa \cite{9707077}.
Based on these findings, we add the reconstruction loss in the frequency domain as a regularization term to reduce the checkerboard effect and ripple of the generated image.
We propose a joint spatial and frequency loss to train our network.
As shown in Fig. \ref{Figure 6}, our method can do better in the frequency domain and produce more realistic results at the same time.

To demonstrate our superiority, we compare our method with state-of-the-art image inpainting approach \cite{9707077} on multiple datasets.
As shown in Tabs. \ref{Table 1} and \ref{Table 2}, our method can boost the performance for each type of mask on FFHQ \cite{DBLP:journals/corr/abs-1812-04948}, ImageNet \cite{5206848}, Places2 \cite{zhou2017places} and WikiArt \cite{Saleh2015LargescaleCO} datasets with the same network architecture and training epochs as LaMa \cite{9707077}.
And it was also ranked first on metric PSNR, LPIPS \cite{8578166} and SSIM \cite{1284395} in the NTIRE 2022 Image Inpainting Challenge Track 1 Unsupervised \cite{romero2022ntire}.
The main contributions can be summarized as follows:
\begin{itemize}
    \item We propose to explore the types of mask used in the training process. At the same time, using our mask generation strategy can effectively improve the results of the model.
	\item We use joint spatial and frequency loss in spatial domain and frequency domain with a regular term to reconstruct the image.
	\item As demonstrated in the experiments, our method can achieve significant performance improvements over LaMa \cite{9707077} without changing the model architecture.
\end{itemize}

\section{Related Work}
\label{sec:related}
\subsection{Image Inpainting}
Early work of image inpainting \cite{Criminisi2003ObjectRB, Hays2007SceneCU, Ballester2001FillinginBJ, Bertalmo2003SimultaneousSA, DBLP:conf/siggraph/BertalmioSCB00} are model-driven.
They explore how to fill the missing information by exploiting clues from the local patch or neighbor patches in the input image.

Pathak et al. \cite{Pathak2016ContextEF} propose the first deep learning image inpainting method that utilizes the CNN with an encoder-decoder architecture trained in the same way as GAN \cite{Goodfellow2014GenerativeAN}.
After that, a lot of methods based on CNN have been proposed.
Iizuka et al. \cite{Iizuka2017GloballyAL} improve the performance by exploiting a local-global discriminator, while Yu et al. \cite{Liu2018ImageIF} utilize a contextual attention model to model the long-distance context correlations.
Due to the mask has negative influence on the results when using the regular convolution, several work modify the convolution operator, introducing partial-convolution \cite{Yu2018GenerativeII} conv, gated-convolution \cite{9010689}.
\cite{Zhu2021ImageIB} propose a mask awareness method using cascaded refinement network.
Zeng et al. utilize ``AOT Block'' and ``SoftGAN'' to enhance the generator and discriminator  \cite{Zeng2022AggregatedCT}.
A new GAN called ``Co-Modulated GAN'' combining conditional GAN and modulated GAN is introduced in \cite{Zhao2021LargeSI}.

There are also some work focusing on the fusion of local and global information.
\cite{Liu2020RethinkingII} utilizes feature equalization to fuse local and global features.
Suvorov et al. \cite{9707077} introduce the Fast Fourier Convolution (FFC) \cite{Chi2020FastFC} and regular convolution to obtain features of global and local receptive fields in the frequency domain.
These methods based on CNN could generate reasonable contents for masked regions.

\begin{figure*}[!t]
    \centering
    \includegraphics[width=0.98\linewidth]{figure/6.pdf}
    \figcaption{Visualization of the spectra of real and fake images. The spectra of images generated by our method is cleaner than the LaMa \cite{9707077}. At the same time, the images of our method are more realistic and cleaner.}
    \label{Figure 6}
\end{figure*}

In view of the dazzling performance of transformer architecture in other tasks, some transformer based methods have emerged recently.
For example, Wan et al. \cite{wan2021high} propose the first transformer based image inpainting method to get the image prior and send the image prior to a CNN.
To incorporate the image prior, the approach of \cite{yu2021diverse} designs a bidirectional and autoregressive transformer.
More recently, Dong et al. \cite{Dong2022IncrementalTS} design a transformer by edge auxiliaries to acquire prior and send the prior with masking positional encoding to a LaMa \cite{9707077} like network.
These methods based on transformer enjoy excellent performance compared with CNN-based methods.

In addition, there is another line of research based on diffusion model.
SohlDickstein et al. \cite{SohlDickstein2015DeepUL} firstly utilize early diffusion models for image inpainting, while Song et al. \cite{Song2021ScoreBasedGM} propose a score-based method using stochastic differential equations for image generation.
More recently, Andreas et al. \cite{DBLP:journals/corr/abs-2201-09865} propose a special model for image inpainting task which is based on diffusion model.

\subsection{Loss Functions}
In the image restoration task, the most commonly used reconstruction loss is $L_1$ loss, $L_2$ loss and Charbonnier loss.
Some image restoration methods also use perceptual loss \cite{Johnson2016PerceptualLF} and adversarial loss \cite{Goodfellow2014GenerativeAN} to improve the perceptive performance.
But above-mentioned methods all focus on the spatial domain reconstruction.
In order to improve the reconstructed image quality, optimization in the frequency domain has gradually attracted researchers' attention in recent years.
For example, spectral regularization is a preliminary attempt \cite{Liu2019SpectralRF}.
More recently, Gal et al. \cite{Gal2021SWAGANAS} propose a wavelet based image generation method.
Jiang et al. \cite{9710849} introduce the focal frequency loss which focuses on hard frequencies
Meanwhile, there are some other work \cite{Cai2021FrequencyDI, Jung2021SpectralDA} on image restoration in the frequency domain.

\section{Our Method}
\label{sec:method}

\begin{figure*}[!t]
    \centering
    \includegraphics[width=0.98\linewidth]{figure/3.pdf}
    \figcaption{Comparison with the state-of-the-art method: LaMa \cite{9707077} for Face Inpainting over seven mask settings on FFHQ \cite{DBLP:journals/corr/abs-1812-04948} dataset. The face images generated by our method are more realistic and have more details on the hair and facial features.}
    \label{Figure 3}
\end{figure*}

\subsection{Network Architecture}
\label{sec:Network Architecture}
Fast Fourier Convolution (FFC) in introduced in \cite{Chi2020FastFC} to capture the global receptive field in the frequency domain. Inspired by this work, Suvorov et al. \cite{9707077} design a network architecture called LaMa which achieves the state-of-the-art results using FFC to capture the global information.
Following these pioneering work, we use LaMa as our network architecture.
Like some other inpainting models, LaMa \cite{9707077} uses an AutoEncoder model to extract the image features. The bottleneck module includes several FFC layers which are based on a channel-wise Fast Fourier transform (FFT), thus capturing global context information.
The FFC layer splits feature channels into two branches: the local branch uses regular convolutions to obtain local features, while the global branch leverages FFC to obtain global features.

\subsection{Training with General Mask}
\label{sec:General Mask Training Strategies}
Each training image $x$ is from a training dataset superimposed by a synthetically generated mask.
We train four different models for four datasets: FFHQ \cite{DBLP:journals/corr/abs-1812-04948}, ImageNet \cite{5206848}, Places2 \cite{zhou2017places} and WikiArt \cite{Saleh2015LargescaleCO}, respectively.
After a lot of experiments, we find that the policy of mask generation noticeably influences the performance of the inpainting model as shown in Tab. \ref{Table 8}.

We firstly try an aggressive large mask generation strategy based on LaMa \cite{9707077}.
This strategy uniformly uses samples from polygonal strip dilated by random strokes and rectangles of arbitrary aspect ratios.
However, from the experimental results shown in Tab. \ref{Table 8}, we can learn that the model trained with this mask generation strategy acquires poor results with the types of ``Completion'', ``Expand'', ``Every n line'' and ``Nearest Neighbor'' masks.

\begin{figure*}[!t]
    \centering
    \includegraphics[width=0.98\linewidth]{figure/5.pdf}
    \figcaption{Comparison with the state-of-the-art method: LaMa \cite{9707077} for Inpainting over seven mask settings on ImageNet \cite{5206848}, Places2 \cite{zhou2017places} and WikiArt \cite{Saleh2015LargescaleCO} datasets. The fourth column is sampled from ImageNet \cite{5206848} dataset. The first, third and sixth columns are sampled from Places2 \cite{zhou2017places} dataset. The second, fifth and seventh columns are sampled from WikiArt \cite{Saleh2015LargescaleCO} dataset.}
    \label{Figure 5}
\end{figure*}

Subsequently, unlike the conventional practice, e.g. DeepFillv2 \cite{9010689} or LaMa \cite{9707077}, we utilize seven types of mask generation strategies, ``Completion'', ``Expand'', ``Every n line'', ``Nearest Neighbor'', ``Thin Strokes'', ``Medium Strokes'', and ``Thick Strokes'', to generate samples randomly, which we call it \textit{General Mask} generation strategy.
These generated masks include not only small narrow masks, but also large-area masks.
As shown in Tabs. \ref{Table 1} and \ref{Table 2}, the model trained with \textit{General Mask} generation strategy performs better than original LaMa \cite{9707077} with all seven types of mask.
This demonstrates that the diversity of degraded images can improve the robustness of the model to face various types of mask.

\subsection{Loss Functions}
\label{sec:Our Losses}
The inpainting problem is ambiguous, which means one image that needs to be repaired might correspond to multiple images. One way is introduce some constraints to alleviate this. In this paper, we will introduce a joint spatial and frequency loss to regularize the optimization of our model, which regularizes the reconstruction results from both the spatial and frequency domains.

For the image inpainting task, the loss in the spatial domain is indispensable.
First of all, we use $L_1$ loss between the unmasked regions in the spatial domain.
It can be formulated as:
\begin{equation}
\begin{aligned}
\mathcal{L}_{1}=\| \boldsymbol{x}-\boldsymbol{\hat{x}} \|_{1} \odot (1-\boldsymbol{M})
\end{aligned}
\end{equation}
where $x$ and $\hat{x}$ indicate the ground truth and predicted images respectively.
$\boldsymbol{M}$ represents 0-1 mask (1 means masked regions and 0 means no masked regions) and $\odot$ means the element-wise multiplication.
Moreover, we use the high receptive field perceptual loss $\mathcal{L}_{PL}$.
It can be formulated as:
\begin{equation}
\begin{aligned}
\mathcal{L}_{PL}=\left[\phi_{PL}(\boldsymbol{x})-\phi_{PL}(\hat{\boldsymbol{x}})\right]^{2}
\end{aligned}
\end{equation}
where $\phi$ indicates a pretrained segmentation ResNet50\cite{He2016DeepRL} with dilated convolutions.

Denote $\mathcal{L}_D$ the discriminator loss, $\mathcal{L}_G$ the generator loss and $\mathcal{L}_{P}$ the gradient penalty, the adversarial loss can be formulated as:
\begin{equation}
\begin{aligned}
\mathcal{L}_{D}=&-\mathbb{E}_{\boldsymbol{x}}[\log D(\boldsymbol{x})]-\mathbb{E}_{\hat{\boldsymbol{x}}, \boldsymbol{M}}[\log D(\hat{\boldsymbol{x}}) \odot(1-\boldsymbol{M})] \\ &-\mathbb{E}_{\hat{\boldsymbol{x}},\boldsymbol{M}}[\log (1-D(\hat{\boldsymbol{x}})) \odot \boldsymbol{M}]
\end{aligned}
\end{equation}
Notably, we only regard features from masked regions as fake samples in $\mathcal{L}_D$.

\begin{equation}
\begin{aligned}
\mathcal{L}_{G}=-\mathbb{E}_{\hat{\boldsymbol{x}}}[\log D(\hat{\boldsymbol{x}})]
\end{aligned}
\end{equation}

\begin{equation}
\begin{aligned}
\mathcal{L}_{P}=\mathbb{E}_{\boldsymbol{x}}\left\|\nabla_{\boldsymbol{x}} D(\boldsymbol{x})\right\|^{2}
\end{aligned}
\end{equation}

\begin{equation}
\begin{aligned}
\mathcal{L}_{a d v}=\mathcal{L}_{D}+\mathcal{L}_{G}+\lambda_{P} \mathcal{L}_{P}
\end{aligned}
\end{equation}
We also use the feature match loss $\mathcal{L}_{fm}$, which is based on $L_1$ loss between discriminator features of true and fake images, to stable the GAN training.
Finally, the loss of LaMa\cite{9707077} can be written as:
\begin{equation}
\begin{aligned}
\mathcal{L}_{LaMa}=\lambda_{1} \mathcal{L}_{L 1}+\lambda_{a d v} \mathcal{L}_{a d v}+\lambda_{f m} \mathcal{L}_{f m}+\lambda_{PL} \mathcal{L}_{PL}
\end{aligned}
\end{equation}
where $\lambda_{1} = 10$, $\lambda_{a d v} = 10$, $\lambda_{PL} = 100$ and $\lambda_{f m} = 30$.

We find although LaMa \cite{9707077} uses Fast Fourier Convolution, the model is not optimized in the frequency domain.
Therefore, the spectrum of the images generated by LaMa \cite{9707077} is defective, as shown in Fig. \ref{Figure 6}.
With this in mind, we take the focal frequency loss \cite{9707077} to construct the frequency fidelity term. Specifically, the focal frequency loss $\mathcal{L}_{FFL}$ is defined as follows:
\begin{equation}
\begin{aligned}
\mathcal{L}_{FFL} = \frac{1}{MN}\sum_{u=0}^{M-1}\sum_{v=0}^{N-1} w(u,v)|F_r(u,v)-F_f(u,v)|^{2}
\end{aligned}
\end{equation}
where $F_r$ represents 2D discrete Fourier transform of real image (represents $x$ here), and $F_f$ represents 2D discrete Fourier transform of fake image (represents $\hat x$ here).
The matrix $w(u,v)$ represents the weight for the spatial frequency at coordinate $(u,v)$, which is defined as:
\begin{equation}
\begin{aligned}
w(u,v) = |F_r(u,v)-F_f(u,v)|^\alpha
\end{aligned}
\end{equation}
where $\alpha = 1$.
Furthermore, we find that the reconstructed images of LaMa \cite{9707077} always get obvious checkerboard effect and ripples with the mask of ``Nearest Neighbor'' and ``Every N Line''. Based on this finding, we introduce the total variation loss, which is defined as: \cite{SohlDickstein2015DeepUL}:
\begin{equation}
\begin{aligned}
\mathcal{L}_{TV} = \sum_{i,j}((x_{i,j+1}-x_{i,j})^2 + (x_{i+1,j}-x_{i,j})^2)^{\frac{\beta}{2}}
\end{aligned}
\end{equation}
where $\beta = 2$.

The final joint spatial and frequency loss function for our inpainting model can be written as:
\begin{equation}
\begin{aligned}
\mathcal{L} = \alpha_1 \mathcal{L}_{TV} + \alpha_2 \mathcal{L}_{FFL} + \alpha_3 \mathcal{L}_{LaMa}
\end{aligned}
\end{equation}
where $\alpha_1 = 1$, $\alpha_2 = 1$ and $\alpha_3 = 1$.

\begin{table*}[ht]

\resizebox{\linewidth}{!}{
\begin{tabular}{c|c|ccccccc}
\hline
        \multirow{2}{*}{Datasets} & \multirow{2}{*}{Methods} & \multicolumn{7}{c}{LPIPS ($\downarrow$)}                      \\
                          &                         & \multicolumn{1}{c|}{Completion}      & \multicolumn{1}{c|}{Expand}          & \multicolumn{1}{c|}{NearesNeighbor} & \multicolumn{1}{c|}{ThinStrokes}     & \multicolumn{1}{c|}{EveryNLines} & \multicolumn{1}{c|}{MediumStrokes}   & ThickStrokes \\ \cline{1-9}
\multirow{3}{*}{Places2}  & LaMa                    & \multicolumn{1}{c|}{0.3302}          & \multicolumn{1}{c|}{0.6098}          & \multicolumn{1}{c|}{0.6511}            & \multicolumn{1}{c|}{0.0903}          & \multicolumn{1}{c|}{0.3125}          & \multicolumn{1}{c|}{0.1193}          & 0.1374 \\
                          & Big LaMa                & \multicolumn{1}{c|}{\color[HTML]{3166FF}0.3298}          & \multicolumn{1}{c|}{\color[HTML]{3166FF}0.6031}          & \multicolumn{1}{c|}{\color[HTML]{3166FF}0.6473}            & \multicolumn{1}{c|}{\color[HTML]{FE0000}{0.0807}} & \multicolumn{1}{c|}{\color[HTML]{3166FF}0.2788}          & \multicolumn{1}{c|}{\color[HTML]{FE0000}{0.1026}} & \color[HTML]{3166FF}0.1266 \\
                          & \textbf{GLaMa}                   & \multicolumn{1}{c|}{\color[HTML]{FE0000}{0.3212}} & \multicolumn{1}{c|}{\color[HTML]{FE0000}{0.5923}} & \multicolumn{1}{c|}{\color[HTML]{FE0000}{0.2280}}   & \multicolumn{1}{c|}{\color[HTML]{3166FF}0.0890}           & \multicolumn{1}{c|}{\color[HTML]{FE0000}{0.0891}} & \multicolumn{1}{c|}{\color[HTML]{3166FF}0.1088}          & \color[HTML]{FE0000}{0.0774}  \\ \cline{1-9}
\multirow{3}{*}{FFHQ}     & LaMa                    & \multicolumn{1}{c|}{0.3102}          & \multicolumn{1}{c|}{0.6089}          & \multicolumn{1}{c|}{0.6518}            & \multicolumn{1}{c|}{0.0976}          & \multicolumn{1}{c|}{0.312}           & \multicolumn{1}{c|}{0.1124}          & 0.1307  \\
                          & Big LaMa                & \multicolumn{1}{c|}{\color[HTML]{3166FF}0.3048}          & \multicolumn{1}{c|}{\color[HTML]{3166FF}0.5892}          & \multicolumn{1}{c|}{\color[HTML]{3166FF}0.6081}            & \multicolumn{1}{c|}{\color[HTML]{FE0000}{0.1109}} & \multicolumn{1}{c|}{\color[HTML]{3166FF}0.3067}          & \multicolumn{1}{c|}{\color[HTML]{FE0000}{0.1246}} & \color[HTML]{3166FF}0.1331  \\
                          & \textbf{GLaMa}                   & \multicolumn{1}{c|}{\color[HTML]{FE0000}{0.2932}} & \multicolumn{1}{c|}{\color[HTML]{FE0000}{0.5532}} & \multicolumn{1}{c|}{\color[HTML]{FE0000}{0.2402}}   & \multicolumn{1}{c|}{\color[HTML]{3166FF}0.1121}          & \multicolumn{1}{c|}{\color[HTML]{FE0000}{0.1829}} & \multicolumn{1}{c|}{\color[HTML]{3166FF}0.1269}          & \color[HTML]{FE0000}{0.1324}  \\ \cline{1-9}
\multirow{3}{*}{ImageNet} & LaMa                    & \multicolumn{1}{c|}{0.3106}          & \multicolumn{1}{c|}{0.5877}          & \multicolumn{1}{c|}{0.6305}            & \multicolumn{1}{c|}{0.0768}          & \multicolumn{1}{c|}{0.2946}          & \multicolumn{1}{c|}{0.0997}          & 0.1133  \\
                          & Big LaMa                & \multicolumn{1}{c|}{\color[HTML]{3166FF}0.3102}          & \multicolumn{1}{c|}{\color[HTML]{3166FF}0.5834}          & \multicolumn{1}{c|}{\color[HTML]{3166FF}0.6283}            & \multicolumn{1}{c|}{\color[HTML]{FE0000}{0.0607}} & \multicolumn{1}{c|}{\color[HTML]{3166FF}0.2529}          & \multicolumn{1}{c|}{\color[HTML]{FE0000}{0.0815}} & \color[HTML]{3166FF}0.1010 \\
                          & \textbf{GLaMa}                   & \multicolumn{1}{c|}{\color[HTML]{FE0000}{0.3009}} & \multicolumn{1}{c|}{\color[HTML]{FE0000}{0.5719}} & \multicolumn{1}{c|}{\color[HTML]{FE0000}{0.2061}}   & \multicolumn{1}{c|}{\color[HTML]{3166FF}0.0654}          & \multicolumn{1}{c|}{\color[HTML]{FE0000}{0.0651}} & \multicolumn{1}{c|}{\color[HTML]{3166FF}0.0834}          & \color[HTML]{FE0000}{0.0859} \\ \cline{1-9}
\multirow{3}{*}{WikiArt}  & LaMa                    & \multicolumn{1}{c|}{0.3517}          & \multicolumn{1}{c|}{0.6261}          & \multicolumn{1}{c|}{0.6711}            & \multicolumn{1}{c|}{0.1093}          & \multicolumn{1}{c|}{0.3317}          & \multicolumn{1}{c|}{0.1397}          & 0.1582 \\
                          & Big LaMa                & \multicolumn{1}{c|}{\color[HTML]{3166FF}0.3498}          & \multicolumn{1}{c|}{\color[HTML]{3166FF}0.6217}          & \multicolumn{1}{c|}{\color[HTML]{3166FF}0.6673}            & \multicolumn{1}{c|}{\color[HTML]{FE0000}{0.1008}} & \multicolumn{1}{c|}{\color[HTML]{3166FF}0.2889}          & \multicolumn{1}{c|}{\color[HTML]{FE0000}{0.1219}} & \color[HTML]{3166FF}0.1439  \\
                          & \textbf{GLaMa}                  & \multicolumn{1}{c|}{\color[HTML]{FE0000}{0.3311}} & \multicolumn{1}{c|}{\color[HTML]{FE0000}{0.5989}} & \multicolumn{1}{c|}{\color[HTML]{FE0000}{0.2497}}   & \multicolumn{1}{c|}{\color[HTML]{3166FF}0.1065}          & \multicolumn{1}{c|}{\color[HTML]{FE0000}{0.0870}} & \multicolumn{1}{c|}{\color[HTML]{3166FF}0.1244}          & \color[HTML]{FE0000}{0.0932} \\
                          \hline
    \end{tabular}
}
\vspace{-0.2cm}
\caption{LPIPS comparisons of different methods on the four datasets.
The top two performing method is highlighted in {\color[HTML]{FE0000}red} and {\color[HTML]{3166FF}blue}.
}
    \label{Table 1}
\end{table*}

\begin{table*}[ht]

\resizebox{\linewidth}{!}{
\begin{tabular}{c|c|ccccccc}
\hline
\multirow{2}{*}{Datasets} & \multirow{2}{*}{Methods} & \multicolumn{7}{c}{FID ($\downarrow$)}                               \\
                          &                         & \multicolumn{1}{c|}{Completion}     & \multicolumn{1}{c|}{Expand}         & \multicolumn{1}{c|}{NearestNeighbor} & \multicolumn{1}{c|}{ThinStrokes}    & \multicolumn{1}{c|}{EveryNLines} & \multicolumn{1}{c|}{MediumStrokes}  & ThickStrokes \\ \cline{1-9}
\multirow{3}{*}{Places2}  & LaMa                    & \multicolumn{1}{c|}{33.63}          & \multicolumn{1}{c|}{87.35}          & \multicolumn{1}{c|}{163.92}            & \multicolumn{1}{c|}{6.37}           & \multicolumn{1}{c|}{17.07}           & \multicolumn{1}{c|}{10.17}          & 13.21 \\
                          & Big LaMa                & \multicolumn{1}{c|}{\color[HTML]{3166FF}30.76}          & \multicolumn{1}{c|}{\color[HTML]{3166FF}81.73}          & \multicolumn{1}{c|}{\color[HTML]{3166FF}167.31}            & \multicolumn{1}{c|}{\color[HTML]{FE0000}{5.74}}  & \multicolumn{1}{c|}{\color[HTML]{3166FF}14.04}           & \multicolumn{1}{c|}{\color[HTML]{FE0000}{9.18}}  & \color[HTML]{FE0000}{12.08} \\ \
                          & \textbf{GLaMa}                   & \multicolumn{1}{c|}{\color[HTML]{FE0000}{29.98}} & \multicolumn{1}{c|}{\color[HTML]{FE0000}{68.74}} & \multicolumn{1}{c|}{\color[HTML]{FE0000}{8.16}}     & \multicolumn{1}{c|}{\color[HTML]{3166FF}6.29}           & \multicolumn{1}{c|}{\color[HTML]{FE0000}{2.05}}   & \multicolumn{1}{c|}{\color[HTML]{3166FF}10.02}          & \color[HTML]{3166FF}13.03 \\ \cline{1-9}
\multirow{3}{*}{FFHQ}     & LaMa                    & \multicolumn{1}{c|}{23.82}          & \multicolumn{1}{c|}{117.60}         & \multicolumn{1}{c|}{133.70}            & \multicolumn{1}{c|}{6.60}           & \multicolumn{1}{c|}{21.44}           & \multicolumn{1}{c|}{8.64}           & 9.19 \\
                          & Big LaMa                & \multicolumn{1}{c|}{\color[HTML]{3166FF}23.32}          & \multicolumn{1}{c|}{\color[HTML]{3166FF}111.91}         & \multicolumn{1}{c|}{\color[HTML]{3166FF}131.72}            & \multicolumn{1}{c|}{\color[HTML]{FE0000}{5.89}}  & \multicolumn{1}{c|}{\color[HTML]{3166FF}20.01}           & \multicolumn{1}{c|}{\color[HTML]{FE0000}{6.68}}  & \color[HTML]{FE0000}{7.54} \\
                          & \textbf{GLaMa}                  & \multicolumn{1}{c|}{\color[HTML]{FE0000}{20.37}} & \multicolumn{1}{c|}{\color[HTML]{FE0000}{89.70}} & \multicolumn{1}{c|}{\color[HTML]{FE0000}{7.65}}     & \multicolumn{1}{c|}{\color[HTML]{3166FF}6.10}           & \multicolumn{1}{c|}{\color[HTML]{FE0000}{6.21}}   & \multicolumn{1}{c|}{\color[HTML]{3166FF}7.65}           & \color[HTML]{3166FF}8.09 \\ \cline{1-9}
\multirow{3}{*}{ImageNet} & LaMa                    & \multicolumn{1}{c|}{25.28}          & \multicolumn{1}{c|}{119.34}         & \multicolumn{1}{c|}{135.01}            & \multicolumn{1}{c|}{8.69}           & \multicolumn{1}{c|}{23.90}           & \multicolumn{1}{c|}{10.53}          & 11.19  \\
                          & Big LaMa                & \multicolumn{1}{c|}{\color[HTML]{3166FF}25.31}          & \multicolumn{1}{c|}{\color[HTML]{3166FF}110.78}         & \multicolumn{1}{c|}{\color[HTML]{3166FF}131.47}            & \multicolumn{1}{c|}{\color[HTML]{FE0000}{7.53}}  & \multicolumn{1}{c|}{\color[HTML]{3166FF}22.17}           & \multicolumn{1}{c|}{\color[HTML]{FE0000}{8.77}}  & \color[HTML]{FE0000}{9.98} \\
                          & \textbf{GLaMa}                  & \multicolumn{1}{c|}{\color[HTML]{FE0000}{22.91}} & \multicolumn{1}{c|}{\color[HTML]{FE0000}{91.78}} & \multicolumn{1}{c|}{\color[HTML]{FE0000}{9.78}}     & \multicolumn{1}{c|}{\color[HTML]{3166FF}8.06}           & \multicolumn{1}{c|}{\color[HTML]{FE0000}{8.21}}   & \multicolumn{1}{c|}{\color[HTML]{3166FF}9.24}           & \color[HTML]{3166FF}10.01 \\ \cline{1-9}
\multirow{3}{*}{WikiArt}  & LaMa                    & \multicolumn{1}{c|}{39.13}          & \multicolumn{1}{c|}{92.01}          & \multicolumn{1}{c|}{162.59}            & \multicolumn{1}{c|}{12.88}          & \multicolumn{1}{c|}{23.47}           & \multicolumn{1}{c|}{16.75}          & 18.96 \\
                          & Big LaMa                & \multicolumn{1}{c|}{\color[HTML]{3166FF}36.76}          & \multicolumn{1}{c|}{\color[HTML]{3166FF}86.18}          & \multicolumn{1}{c|}{\color[HTML]{3166FF}153.31}            & \multicolumn{1}{c|}{\color[HTML]{FE0000}{11.18}} & \multicolumn{1}{c|}{\color[HTML]{3166FF}20.62}           & \multicolumn{1}{c|}{\color[HTML]{FE0000}{15.81}} & \color[HTML]{FE0000}{17.04} \\
                          & \textbf{GLaMa}                  & \multicolumn{1}{c|}{\color[HTML]{FE0000}{35.72}} & \multicolumn{1}{c|}{\color[HTML]{FE0000}{74.51}} & \multicolumn{1}{c|}{\color[HTML]{FE0000}{12.08}}    & \multicolumn{1}{c|}{\color[HTML]{3166FF}11.89}          & \multicolumn{1}{c|}{\color[HTML]{FE0000}{8.03}}   & \multicolumn{1}{c|}{\color[HTML]{3166FF}16.23}          & \color[HTML]{3166FF}17.87 \\
                          \hline
    \end{tabular}
}
\vspace{-0.2cm}
\caption{
FID comparisons of different methods on the four datasets.}
\label{Table 2}
\end{table*}

\section{Experiments}
\label{sec:Experiments}

\subsection{Datasets and Metrics}
\label{sec:Datasets and metrics}
In our experiments, we use FFHQ \cite{DBLP:journals/corr/abs-1812-04948}, ImageNet \cite{5206848}, Places2 \cite{zhou2017places} and WikiArt \cite{Saleh2015LargescaleCO} datasets, which contain various kinds of images, to demonstrate the results.
For FFHQ \cite{DBLP:journals/corr/abs-1812-04948}, we use about 60K face images as the training set and 2,000 other images as the validation.
For Places2 \cite{zhou2017places}, we use about 1,800K images from various scenes as the training set and 3650 other images from validation set as the validation.
For ImageNet \cite{5206848}, we use about 1,000K images as the training set and 2000 other images from validation dataset as the validation set.
And for WikiArt \cite{Saleh2015LargescaleCO}, we use the whole training set and 2000 other images from validation set as the validation.

In the training stage for FFHQ \cite{DBLP:journals/corr/abs-1812-04948}, our images are resized to $256 \times 256$ resolution.
For the remaining three training datasets, we crop the images to $256 \times 256$ pixels.
Moreover, we use the original size of the images in the validation stage without cropping or resizing them.

We follow the established practice in recent image inpainting literature and use Learned Perceptual Image Patch Similarity (LPIPS) \cite{8578166} and Frechet inception distance (FID) \cite{Heusel2017GANsTB} metrics.
Compared to PSNR and SSIM \cite{1284395}, LPIPS \cite{8578166} and FID \cite{Heusel2017GANsTB} are more suitable for measuring performance of inpainting for large masks.

\subsection{Implementation Details}

\label{sec:Implementation Details}
For inpainting network \cite{9707077}, we followed LaMa \cite{9707077} using a ResNet-like \cite{He2016DeepRL} architecture with 3 downsampling blocks, 6-18 residual blocks, and 3 upsampling blocks.
We use Adam optimizer \cite{Kingma2015AdamAM}, with the fixed learning rates 0.001 and 0.0001 for inpainting and discriminator networks, respectively.
We set the hyper-parameters using the grid search strategy on FFHQ dataset, which leds to the weight values $\alpha_1= 1$, $\alpha_2=1$, $\alpha_3=1$, $\alpha=1$, $\beta=2$.
The same parameters are used for other three datasets for training.
All models are trained for 40 epochs with a batch size of 20, on 8 NVidia V100 GPUs for approximately 72 hours.
The LaMa and Big LaMa \cite{9707077} models used as baseline in the Tabs. \ref{Table 1} and \ref{Table 2} were retrained with the same experimental setups. Becaused LaMa and Big LaMa \cite{9707077} and the results are higher than the original LaMa and Big LaMa \cite{9707077} model in FFHQ \cite{DBLP:journals/corr/abs-1812-04948} and Places2 \cite{zhou2017places} datasets.

\subsection{Comparisons to the Baselines}
\label{sec:Comparisons to the baselines}
We compare the proposed approach with the strong baselines (LaMa and Big LaMa \cite{9707077}). For each dataset, we validate the performance across seven types of mask.
As shown in Tabs. \ref{Table 1} and \ref{Table 2}, we can see that our GLaMa is better than the original LaMa \cite{9707077} especially for ``Completion'' mask, ``Expand'' mask, ``Nearest Neighbor'' mask and ``Every N Lines'' mask.
Compared with the images generated by LaMa, our images are more realistic with less checkerboard effect and ripples, as shown in Fig. \ref{Figure 6}, \ref{Figure 3} and \ref{Figure 5}.
GLaMa also works better than Big LaMa \cite{9707077} in most types of mask with the same network architecture and training time as LaMa \cite{9707077}.
Meanwhile, our method that simply using the \emph{General Mask} generation strategy and joint spatial and frequency loss can bring considerable improvements.

\begin{table}[ht]
\centering
\resizebox{0.75\linewidth}{!}{
\begin{tabular}{ccc|c|c}
\hline
\multicolumn{1}{c}{{ $\mathcal{L}_{LaMa}$}}  & \multicolumn{1}{c}{{$\mathcal{L}_{TV}$}} &$\mathcal{L}_{FFL}$ & {FID ($\downarrow$)}         & {LPIPS ($\downarrow$)}       \\ \hline
{\checkmark }                                                            & { }                        &    & {57.015} & {0.3382}      \\
{\checkmark }                                                             & {\checkmark}                        &    & {22.363} & {0.2350} \\
{\checkmark }                                                            & {\checkmark }                        & {\checkmark }    & {\color[HTML]{FE0000}20.827} & {\color[HTML]{FE0000}0.2253} \\
\hline
\end{tabular}
}
\caption{
Ablation studies over the loss functions.
Here, FID and LPIPS values of the results are reported.
}
\label{Table 3}
\end{table}

\begin{table}[ht]
\resizebox{\linewidth}{!}{
\begin{tabular}{ccc|c|c}
\hline
\multicolumn{1}{c}{\textit{LaMa Mask}} & \multicolumn{1}{c}{{\textit{LaMa Plus Mask}}} & \multicolumn{1}{c|}{{\textit{General Mask}}}  & {FID ($\downarrow$)}         & {LPIPS ($\downarrow$)}       \\ \hline
{\checkmark }                           & { }                                 & { }                        &{49.015} & {0.3118}      \\
{}                           & {\checkmark }                                  & { }                        &{23.514} & {0.2448} \\
{}                           & {}                                  & {\checkmark}                        &{\color[HTML]{FE0000}20.827} & {\color[HTML]{FE0000}0.2253} \\
\hline
\end{tabular}
}
\caption{Ablation studies over the mask generation methods.
\textit{LaMa Mask}, \textit{LaMa Plus Mask} represent using original LaMa, LaMa with ``Nearest Neighbour'' and ``Every n line'', respectively.
\textit{General Mask} represents using our method.
}
\label{Table 8}
\end{table}

\subsection{Ablation Studies}
\label{sec:Ablation Studies}
The goal of the ablation studies is to carefully validate the influence of different components of our method.
To investigate the effectiveness of ours mask generation strategy and losses, we conduct ablation experiments and the results comparison are shown in Tabs. \ref{Table 3} and \ref{Table 8}. All these results are conducted on FFHQ \cite{DBLP:journals/corr/abs-1812-04948} dataset.

\paragraph{Mask Generation Strategy.}
We verify that the \textit{General Mask} generation strategy is necessary in Tab. \ref{Table 8}.
\textit{LaMa Mask} and \textit{LaMa Plus Mask} generation strategy represent using LaMa original policy to generate mask (three types of mask) and LaMa original policy with ``Nearest Neighbour'' and ``Every n line'' to generate mask (five types of mask), respectively.
\textit{General Mask} generation strategy represents using our policy to generate mask (seven types of mask).
We can see from Tab. \ref{Table 8} that using \textit{General Mask} generation strategy can greatly improve the model performance.
This demonstrates that the diversity of degraded images can improve the robustness of the model to face various types of mask.

\paragraph{Loss Functions.}
We experiment with the two proposed losses in Tab. \ref{Table 3}, where total variation loss \cite{7299155} and focal frequency loss \cite{9710849} can boost the model performance.
At the same time, it can be seen from Fig. \ref{Figure 6} that the spectrum of the image generated by GLaMa is more accurate than LaMa \cite{9707077} :
ours spectrum is clean, while the spectrum of LaMa is very noisy.
These experimental results further prove the superiority of our method.

\begin{figure}
    \centering
    \includegraphics[width=0.98\linewidth]{figure/8.pdf}
    \caption{Visual comparisons with unseen masks.
    }
    \label{Figure 8}
\end{figure}

\subsection{Generalization of the Proposed Method}
\label{sec:Generalization}
\begin{figure*}[!t]
    \centering
    \includegraphics[width=0.905\linewidth]{figure/7.pdf}
    \caption{
    Some failed examples on the FFHQ \cite{DBLP:journals/corr/abs-1812-04948} dataset.
    }
    \label{Figure 7}
\end{figure*}

\paragraph{Generalization to the Masks.}
To verify the generalization ability of our model for the masks that do not appear in the training stage, we randomly generate some other kinds of masks that are not in the seven kinds of masks.
The mask generation strategy will combine the samples from seven kinds of masks and rectangular mask of arbitrary aspect ratios to generate some other kinds of masks.
As shown in Fig. \ref{Figure 8}, GLaMa can generate high-quality images, while the images generated by LaMa \cite{9707077} are low-quality, which proves that our method has strong generalization performance.
Hence one can see that, whether the masks are appeared during the training stage, our model can recover the target images very well.

\begin{center}
\begin{table}
\resizebox{0.99\linewidth}{!}{
\begin{tabular}{c|ccccccc|ccccccc}
\hline
\multirow{2}{*}{Teams}
                &
\multicolumn{7}{c}{FFHQ}                                                                                \\ \cline{2-8}
                 & \multicolumn{1}{c|}{FID ($\downarrow$)}                          & \multicolumn{2}{c|}{LPIPS ($\downarrow$)}                        & \multicolumn{2}{c|}{PSNR ($\uparrow$)}                          & \multicolumn{2}{c}{SSIM ($\uparrow$)} \\ \cline{1-8}
\multicolumn{1}{c|}{VIP}                  & \multicolumn{1}{c|}{115.922}                      & \multicolumn{2}{c|}{0.433}                        & \multicolumn{2}{c|}{17.144}                        & \multicolumn{2}{c}{0.649} \\
\multicolumn{1}{c|}{\textbf{AIIA (Ours)}}                 & \multicolumn{1}{c|}{9.823}                        & \multicolumn{2}{c|}{0.239}                        & \multicolumn{2}{c|}{{\color[HTML]{3166FF} 25.316}} & \multicolumn{2}{c}{0.814} \\
\multicolumn{1}{c|}{HSSLAB}               & \multicolumn{1}{c|}{13.504}                       & \multicolumn{2}{c|}{{\color[HTML]{3166FF} 0.236}} & \multicolumn{2}{c|}{25.187}                        & \multicolumn{2}{c}{{\color[HTML]{3166FF} 0.821}} \\
\multicolumn{1}{c|}{KwaiInpainting}       & \multicolumn{1}{c|}{21.345}                       & \multicolumn{2}{c|}{0.239}                        & \multicolumn{2}{c|}{25.060}                        & \multicolumn{2}{c}{{\color[HTML]{FE0000} 0.838}} \\
\multicolumn{1}{c|}{ArtificiallyInspired} & \multicolumn{1}{c|}{{\color[HTML]{FE0000} 4.719}} & \multicolumn{2}{c|}{{\color[HTML]{FE0000} 0.205}} & \multicolumn{2}{c|}{{\color[HTML]{FE0000} 25.999}} & \multicolumn{2}{c}{0.816} \\
\multicolumn{1}{c|}{SIGMA}                & \multicolumn{1}{c|}{{\color[HTML]{3166FF} 7.203}} & \multicolumn{2}{c|}{0.248}                        & \multicolumn{2}{c|}{24.860}                        & \multicolumn{2}{c}{0.795} \\
\hline
\hline
\multirow{2}{*}{Teams}
                &
\multicolumn{7}{c}{Places2}                                                                                \\ \cline{2-8}
                 & \multicolumn{1}{c|}{FID ($\downarrow$)}                          & \multicolumn{2}{c|}{LPIPS ($\downarrow$)}                        & \multicolumn{2}{c|}{PSNR ($\uparrow$)}                          & \multicolumn{2}{c}{SSIM ($\uparrow$)} \\ \cline{1-8}
\multicolumn{1}{c|}{VIP}                  & \multicolumn{1}{c|}{{\color[HTML]{000000} 52.471}} & \multicolumn{2}{c|}{{\color[HTML]{000000} 0.415}} & \multicolumn{2}{c|}{{\color[HTML]{000000} 17.256}} & \multicolumn{2}{c}{{\color[HTML]{000000} 0.626}} \\
\multicolumn{1}{c|}{\textbf{AIIA (Ours)}}                 & \multicolumn{1}{c|}{{\color[HTML]{3166FF} 8.772}}  & \multicolumn{2}{c|}{{\color[HTML]{FE0000} 0.224}} & \multicolumn{2}{c|}{{\color[HTML]{3166FF} 24.145}} & \multicolumn{2}{c}{{\color[HTML]{FE0000} 0.800}} \\
\multicolumn{1}{c|}{HSSLAB}               & \multicolumn{1}{c|}{{\color[HTML]{000000} 9.861}}  & \multicolumn{2}{c|}{{\color[HTML]{000000} 0.227}} & \multicolumn{2}{c|}{{\color[HTML]{FE0000} 24.345}} & \multicolumn{2}{c}{{\color[HTML]{3166FF} 0.798}} \\
\multicolumn{1}{c|}{KwaiInpainting}       & \multicolumn{1}{c|}{{\color[HTML]{000000} 18.334}} & \multicolumn{2}{c|}{{\color[HTML]{333333} 0.255}} & \multicolumn{2}{c|}{{\color[HTML]{000000} 23.410}} & \multicolumn{2}{c}{{\color[HTML]{000000} 0.787}} \\
\multicolumn{1}{c|}{ArtificiallyInspired} & \multicolumn{1}{c|}{{\color[HTML]{FE0000} 7.544}}  & \multicolumn{2}{c|}{{\color[HTML]{3166FF} 0.225}} & \multicolumn{2}{c|}{{\color[HTML]{000000} 23.248}} & \multicolumn{2}{c}{{\color[HTML]{000000} 0.777}} \\
\multicolumn{1}{c|}{SIGMA}                & \multicolumn{1}{c|}{{\color[HTML]{000000} 11.496}} & \multicolumn{2}{c|}{{\color[HTML]{000000} 0.270}} & \multicolumn{2}{c|}{{\color[HTML]{000000} 22.562}} & \multicolumn{2}{c}{{\color[HTML]{000000} 0.748}} \\
\hline
\hline
\multirow{2}{*}{Teams}
                &
\multicolumn{7}{c}{ImageNet}                                                                                \\ \cline{2-8}
                 & \multicolumn{1}{c|}{FID ($\downarrow$)}                          & \multicolumn{2}{c|}{LPIPS ($\downarrow$)}                        & \multicolumn{2}{c|}{PSNR ($\uparrow$)}                          & \multicolumn{2}{c}{SSIM ($\uparrow$)} \\ \cline{1-8}
\multicolumn{1}{c|}{VIP}                  & \multicolumn{1}{c|}{{\color[HTML]{000000} 50.898}} & \multicolumn{2}{c|}{{\color[HTML]{000000} 0.403}} & \multicolumn{2}{c|}{{\color[HTML]{000000} 17.450}} & \multicolumn{2}{c}{{\color[HTML]{000000} 0.626}} \\
\multicolumn{1}{c|}{\textbf{AIIA (Ours)}}                 & \multicolumn{1}{c|}{{\color[HTML]{FE0000} 10.007}} & \multicolumn{2}{c|}{{\color[HTML]{FE0000} 0.207}} & \multicolumn{2}{c|}{{\color[HTML]{FE0000} 25.226}} & \multicolumn{2}{c}{{\color[HTML]{FE0000} 0.800}} \\
\multicolumn{1}{c|}{HSSLAB}               & \multicolumn{1}{c|}{{\color[HTML]{3166FF} 11.770}} & \multicolumn{2}{c|}{{\color[HTML]{000000} 0.227}} & \multicolumn{2}{c|}{{\color[HTML]{3166FF} 24.303}} & \multicolumn{2}{c}{{\color[HTML]{3166FF} 0.798}} \\
\multicolumn{1}{c|}{KwaiInpainting}       & \multicolumn{1}{c|}{{\color[HTML]{000000} 18.854}} & \multicolumn{2}{c|}{{\color[HTML]{333333} 0.249}} & \multicolumn{2}{c|}{{\color[HTML]{000000} 23.804}} & \multicolumn{2}{c}{{\color[HTML]{000000} 0.787}} \\
\multicolumn{1}{c|}{ArtificiallyInspired} & \multicolumn{1}{c|}{{\color[HTML]{333333} 12.059}} & \multicolumn{2}{c|}{{\color[HTML]{3166FF} 0.217}} & \multicolumn{2}{c|}{{\color[HTML]{000000} 24.278}} & \multicolumn{2}{c}{{\color[HTML]{000000} 0.777}} \\
\multicolumn{1}{c|}{SIGMA}                & \multicolumn{1}{c|}{{\color[HTML]{000000} 19.646}} & \multicolumn{2}{c|}{{\color[HTML]{000000} 0.311}} & \multicolumn{2}{c|}{{\color[HTML]{000000} 22.454}} & \multicolumn{2}{c}{{\color[HTML]{000000} 0.748}} \\
\hline
\hline
\multirow{2}{*}{Teams}
                &
\multicolumn{7}{c}{WikiArt}                                                                                \\ \cline{2-8}
                 & \multicolumn{1}{c|}{FID ($\downarrow$)}                          & \multicolumn{2}{c|}{LPIPS ($\downarrow$)}                        & \multicolumn{2}{c|}{PSNR ($\uparrow$)}                          & \multicolumn{2}{c}{SSIM ($\uparrow$)} \\ \cline{1-8}
\multicolumn{1}{c|}{VIP}                  & \multicolumn{1}{c|}{{\color[HTML]{000000} 75.645}} & \multicolumn{2}{c|}{{\color[HTML]{000000} 0.437}} & \multicolumn{2}{c|}{{\color[HTML]{000000} 17.243}} & \multicolumn{2}{c}{{\color[HTML]{000000} 0.609}} \\
\multicolumn{1}{c|}{\textbf{AIIA (Ours)}}                 & \multicolumn{1}{c|}{{\color[HTML]{333333} 14.974}} & \multicolumn{2}{c|}{{\color[HTML]{FE0000} 0.244}} & \multicolumn{2}{c|}{{\color[HTML]{FE0000} 24.350}} & \multicolumn{2}{c}{{\color[HTML]{FE0000} 0.767}} \\
\multicolumn{1}{c|}{HSSLAB}               & \multicolumn{1}{c|}{{\color[HTML]{333333} 14.986}} & \multicolumn{2}{c|}{{\color[HTML]{000000} 0.254}} & \multicolumn{2}{c|}{{\color[HTML]{3166FF} 24.257}} & \multicolumn{2}{c}{{\color[HTML]{333333} 0.752}} \\
\multicolumn{1}{c|}{KwaiInpainting}       & \multicolumn{1}{r|}{{\color[HTML]{000000} 26.395}} & \multicolumn{2}{c|}{{\color[HTML]{333333} 0.276}} & \multicolumn{2}{c|}{{\color[HTML]{000000} 23.142}} & \multicolumn{2}{c}{{\color[HTML]{3166FF} 0.759}} \\
\multicolumn{1}{c|}{ArtificiallyInspired} & \multicolumn{1}{c|}{{\color[HTML]{FE0000} 8.524}}  & \multicolumn{2}{c|}{{\color[HTML]{3166FF} 0.248}} & \multicolumn{2}{c|}{{\color[HTML]{000000} 23.799}} & \multicolumn{2}{c}{{\color[HTML]{000000} 0.758}} \\
\multicolumn{1}{c|}{SIGMA}                & \multicolumn{1}{c|}{{\color[HTML]{3166FF} 14.125}} & \multicolumn{2}{c|}{{\color[HTML]{000000} 0.286}} & \multicolumn{2}{c|}{{\color[HTML]{000000} 22.717}} & \multicolumn{2}{c}{{\color[HTML]{000000} 0.720}} \\
\hline
\end{tabular}
}
\caption{Performance comparisons of different methods on the four datasets. Results are provided from the NTIRE 2022 Image Inpainting challenge report \cite{romero2022ntire}.}
\label{Table 5}
\end{table}
\end{center}

\paragraph{Generalization to the Resolution.}
To further exploit the reconstruction ability of different approaches, we also conduct some experiments when the input images are with $512 \times 512$ pixels (or higher resolution) while the training images are with $256 \times 256$ pixels.
It can be seen from Figs. \ref{Figure 4} and \ref{Figure 7} that LaMa \cite{9707077} is not good at repairing the large-area mask of high-resolution face.
Some low-quality images are generated with the high-resolution images and large-scale masks.
Since we use $256 \times 256$ images to train the LaMa \cite{9707077}, the image with $256 \times 256$ pixels can be repaired very well.
Based on this finding, we design a two-stage model. In the first stage, we use the original LaMa \cite{9707077} to generate the coarse image in $256 \times 256$ resolution.
In the second stage, we use bilinear to upsample the image and then use another LaMa-like model to refine the upsampled image.
We can see from Fig. \ref{Figure 4} that the two-stage model called GLaMa-2stage can generate more realistic faces.

\subsection{NTIRE 2022 Image Inpainting Challenge}
\label{sec:NTIRE 2022 Image Inpainting Challenge}
This work is proposed to participate in the NTIRE 2022 Image Inpainting Challenge Track 1 Unsupervised \cite{romero2022ntire}.
The objective of this challenge is to obtain a mask agnostic network solution capable of producing high-quality results for the best perceptual quality with respect to the ground truth.
Our results for the NTIRE 2022 challenge \cite{romero2022ntire} are denoted as AIIA (our team's name) to distinguish from the other teams in Tab. \ref{Table 5}.
It can be seen that our method goes significantly ahead of FID, LPIPS, PSNR and SSIM for Places2 \cite{zhou2017places}, ImageNet \cite{5206848} and WikiArt \cite{Saleh2015LargescaleCO} datasets.

\section{Conclusion}
\label{sec:Conclusion}
This study explores the mask generation strategy in image inpainting task.
Moreover, we introduce a joint spatial and frequency loss to reduce checkerboard effect and ripples.
The proposed GLaMa achieves significant performance improvements over LaMa \cite{9707077} without changing the model architecture. At the same time, the proposed method is more general and other methods can simply adopt the strategies we have explored.
Extensive ablation studies are performed to validate the effectiveness of our method, and comparative results on four public datasets demonstrates the state-of-the-art performance of our method.

\paragraph{Limitations}. There are also some failed examples when the input image are $512\times512$ pixels, as shown in Fig. \ref{Figure 7}. The results generated by GLaMa-2stage are sometimes blurred and unreasonable. LaMa \cite{9707077} and GLaMa also generate low-quality and unpleasant results.
In the future, we will incorporate the StyleGAN prior \cite{karras2019style} to assist the image inpainting task.

\noindent\textbf{Acknowledgments:} The research was supported by the National Natural Science Foundation of China (61971165), in part by the Fundamental Research Funds for the Central Universities (FRFCU5710050119), the Natural Science Foundation of Heilongjiang Province (YQ2020F004).

\newpage
{\small
\bibliographystyle{ieee_fullname}