\section{Introduction}
    Plasma in the solar atmosphere may be found in a state of partial ionization. This means that it may contain a non-negligible amount of neutral particles which are not directly affected by the presence of magnetic fields. Neutral species have been detected, for instance, in solar prominences \citep[see, e.g.,][]{Hirayama1985SoPh..100..415H}, in chromospheric spicules \citep{Pereira2016ApJ...824...65P} or in coronal rain blobs \citep{Antolin2012ApJ...745..152A,Schad2018ApJ...865...31S}.

    From the theoretical point of view it is known that the dynamics of the ionized and the neutral species decouple at small enough spatial and temporal scales. This has been shown, for instance, by the works of \citet{Hillier2019PhPl...26h2902H} and \citet{Popescu2021A&A...646A..93P,Popescu2021A&A...650A.181P}, who used multi-fluid models to numerically study the development of the Kelvin-Helmholtz instability (KHI) and the Rayleigh-Taylor instability (RTI), respectively, in prominences. It has also been shown that the different behavior of the charged and neutral particles leads to the damping of waves and plasma heating due to the collisions between the various species \citep[see, e.g.,][]{Leake2005A&A...442.1091L,Zaqarashvili2011A&A...529A..82Z,Soler2013ApJ...767..171S,MartinezGomez2018ApJ...856...16M,Popescu2019A&A...630A..79P}

    However, the observational evidence regarding the different dynamics of charged and neutral species in the solar atmosphere is still scarce, mainly due to the requirements about temporal and spatial resolution. \citet{Khomenko2016ApJ...823..132K} studied the Doppler shifts of the spectral lines He \textsc{I} 10830 $\rm{\AA}$  and Ca \textsc{II} 8542 $\rm{\AA}$ in observations of a solar prominence and deduced the existence of large ion-neutral drift velocities (of the order of several hundred $\rm{m \ s^{-1}}$) in small-scale and short-lived transients. \citet{Anan2017A&A...601A.103A} analyzed a larger set of spectral lines and also found differences in the Doppler velocities but they concluded that they were not caused by decoupling effects but were the result of observing motions originated at different regions of the prominence. \citet{Stellmacher2017SoPh..292...83S} performed observations of a quiescent prominence and systematically found an excess of $\sim 10\%$ of the ion velocity over the neutral velocity. These results were later confirmed by \citet{Wiehr2019ApJ...873..125W,Wiehr2021ApJ...920...47W}. Recent studies by \citet{Gonzalez-Manrique2022} and \citet{Zapior2022ApJ...934...16Z} also point to the existence of large drift velocities (of up $1.7 \ \rm{km \ s^{-1}}$) in small regions of solar prominences. Regarding coronal rain observations, \citet{Ahn2014SoPh..289.4117A} compared the Doppler velocities computed from the spectral lines $\rm{H\alpha}$ and Ca \textsc{II} 8542 $\rm{\AA}$ and found that they ``matched well'', although their results show that they are not completely identical.

    Coronal rain is a very appropriate phenomenon to study the interaction between charged and neutrals species \citep{Oliver2016ApJ...818..128O}. It involves a large range of ionization degree of the plasma, from the fully ionized case of the solar corona to the partially or weakly ionized condition of the falling blobs, which causes a large variation in the collisional coupling between the various species. However, most of the works on coronal rain dynamics have only considered the fully ionized case. This applies both to one-dimensional \citep[e.g.,][]{Muller2005A&A...436.1067M, Mendoza-Briceno2005ApJ...624.1080M, Antolin2010ApJ...716..154A, Mikic2013ApJ...773...94M, Froment2018ApJ...855...52F} and multi-dimensional numerical simulations \citep[e.g.][]{Mok2016ApJ...817...15M, Xia2017A&A...603A..42X, Kohutova2020A&A...639A..20K,Li2022ApJ...926..216L}.

    In the present work, we are interested in investigating the effects of that ion-neutral collisional interaction during the fall of cold condensations of plasma towards the solar surface. Therefore, we use a two-fluid model \citep{Khomenko2014PhPl...21i2901K,Popescu2019A&A...627A..25P} to perform high-resolution 2D simulations of the evolution of partially ionized coronal rain blobs. We focus on analyzing the existence of charged-neutral drift velocities and the plasma heating caused by the collisional friction. Then, we discuss observational consequences of these processes.

\section{Methodology}
\subsection{Model}
    In this work, we assume that the plasma is composed of hydrogen only. Therefore, it contains three different kinds of particles: electrons, protons, and neutrals, which are denoted by the subscripts `e', `p', and `n', respectively. We also assume that the plasma fulfills the charge quasi-neutrality condition, so the number density of electrons and protons is the same, that is, $n_{\rm{e}} = n_{\rm{p}}$. Furthermore, a strong coupling between electrons and protons is considered, which allows to treat them together as a single charged fluid, denoted by the subscript `c'. Then, the temporal evolution of this plasma is described by a two-fluid 2D model in which charged and neutral particles interact by means of elastic collisions.

    The details of the full set of equations included in the two-fluid model can be found in \citet{Popescu2019A&A...627A..25P} and \citet{Martinez2021A&A...650A.123M}. Here we only show the most relevant terms for the present investigation, which are the momentum transfer and heating due to collisions, given by
	\begin{equation} \label{eq:Rcoll}
		\bm{R}_{\rm{cn}} = \alpha \rho_{\rm{c}} \rho_{\rm{n}} \left(\bm{V}_{\rm{n}} - \bm{V}_{\rm{c}} \right)
	\end{equation}
	and
	\begin{equation} \label{eq:qv}
		Q_{V} = \alpha \rho_{\rm{c}}\rho_{\rm{n}}\frac{V_{\rm{D}}^{2}}{2},
	\end{equation}
	respectively. In the previous expressions, the variables $\rho_{\rm{c}}$, $\rho_{\rm{n}}$, $\bm{V}_{\rm{c}}$, and $\bm{V}_{\rm{n}}$ represent the mass density of charges, the mass density of neutrals, the velocity of charges, and the velocity of neutrals, respectively. The parameter $\alpha$ is known as the collisional coupling coefficient and is given by
	\begin{equation} \label{eq:alpha_def}
		\alpha = \frac{\rho_{\rm{e}} \nu_{\rm{en}} + \rho_{\rm{p}} \nu_{\rm{pn}}}{\rho_{\rm{n}} \rho_{\rm{c}}},
	\end{equation}
	where $\nu_{\rm{en}}$ and $\nu_{\rm{pn}}$ are the electron-neutral and proton-neutral collision frequencies. The frequency of collisions between two different species `s' and `t' (when one of them is neutral) can be computed from the following expression \citep{Draine1986MNRAS.220..133D}:
	\begin{equation} \label{eq:nu_st}
		\nu_{\rm{st}} = \frac{n_{\rm{t}} m_{\rm{t}}}{m_{\rm{s}}+m_{\rm{t}}} \sqrt{\frac{8 k_{\rm{B}}}{\pi}\left(\frac{T_{\rm{s}}}{m_{\rm{s}}}+\frac{T_{\rm{t}}}{m_{\rm{t}}}\right)} \sigma_{\rm{st}}.
	\end{equation}
	Here, $n_{\rm{s}}$,  $m_{\rm{s}}$, and $T_{\rm{s}}$ represent the number density, mass, and temperature of the species `s', respectively. The parameter $k_{B}$ is the Boltzmann constant and $\sigma_{\rm{st}}$ is the collisional cross-section between the two species. We use the following values, taken from \citet{Leake2013PhPl...20f1202L}: $\sigma_{\rm{pn}} = 1.16 \times 10^{-18} \ \rm{m^{2}}$ and $\sigma_{\rm{en}} = 10^{-19} \ \rm{m^{2}}$. We note that $n_{\rm{c}} = n_{\rm{e}} + n_{\rm{p}} = 2 n_{\rm{p}}$, and the relation between the number and mass densities are $\rho_{\rm{c}} = \rho_{\rm{p}} + \rho_{\rm{e}} = n_{\rm{p}} m_{\rm{p}} + n_{\rm{e}} m_{\rm{e}} \approx n_{\rm{p}} m_{\rm{p}} = n_{\rm{c}}/2 m_{\rm{p}}$ and $\rho_{\rm{n}} = n_{\rm{n}} m_{\rm{n}} \approx n_{\rm{n}} m_{\rm{p}}$ (since for the case of hydrogen plasma $m_{\rm{n}} \approx m_{\rm{p}}$).
	Finally, the variable $V_{\rm{D}}$ is the modulus of the vector $\bm{V}_{\rm{D}}$, which represents the drift velocity between the two fluids and is given by
	\begin{equation} \label{eq:vd}
		\bm{V}_{\rm{D}} = (V_{\rm{c,x}}-V_{\rm{n,x}},V_{\rm{c,z}}-V_{\rm{n,z}})=\left(V_{\rm{D,x}}, V_{\rm{D,z}}\right).
	\end{equation}

\subsection{Numerical setup}
	We numerically solve the two-fluid equations using the \textsc{Mancha-2F} code \citep{Popescu2019A&A...627A..25P}. In this code, the physical variables, $\bm{f}$, are treated as the sum of equilibrium values ($\bm{f}_{\rm{eq}}$, which remain constant in time) and perturbation values ($\bm{f}_{1}$, which evolve with time and can be non-linear).

	To simulate the fall of a partially ionized coronal rain blob we use as equilibrium a 2D isothermal vertically stratified atmosphere, with a temperature given by $T_{0} = 2 \times 10^{6} \ \rm{K}$, a uniform vertical magnetic field given by $B_{0} = 5 \ \rm{G}$, and a plasma ionization degree given by $\chi_{\rm{c},eq} = \rho_{\rm{c},eq}/\rho_{T,\rm{eq}} = 0.9$ (where $\rho_{T,\rm{eq}} = \rho_{\rm{c},eq} + \rho_{\rm{n},eq}$). We note that in reality the equilibrium atmosphere representing the solar corona should be fully ionized (that is, $\chi_{\rm{c},eq} = 1$). However, due to numerical reasons (the fluid equations cannot be solved when $\rho_{\rm{s}}(x,z) = 0$, since these variables appear in the denominators of several terms), we need to include a small fraction of neutrals in this environment. Thus, for our simulation we set the relative abundance of neutrals in the equilibrium state as $\chi_{\rm{n},eq}=\rho_{\rm{n},eq}/\rho_{T,\rm{eq}} = 1 - \chi_{\rm{c},eq} = 0.1$.

	As shown by \citet{Oliver2016ApJ...818..128O}, the condition of magneto-hydrostatic equilibrium leads to the following solutions from the two-fluid model:
	\begin{gather} \label{eq:equil}
	    \rho_{\rm{c},eq}(z) = \rho_{\rm{c},0} e^{-z/H_{\rm{c}}}, \ \rho_{\rm{n},eq}(z) = \rho_{\rm{n},0} e^{-z/H_{\rm{n}}},\\
	    P_{\rm{c},eq}(z) = P_{\rm{c},0} e^{-z/H_{\rm{c}}}, \ P_{\rm{n},eq}(z) = P_{\rm{n},0} e^{-z/H_{\rm{n}}},
	\end{gather}
	where $P_{\rm{c},eq}$ and $P_{\rm{n},eq}$ are the equilibrium pressures of the charged and neutral fluids, respectively, and the following relations, corresponding to the values of density and pressure at the base of the corona ($z = 0$), are satisfied:
	\begin{equation} \label{eq:p_rho_rel}
	    P_{\rm{c},0} = \frac{2 \rho_{\rm{c},0}k_{\rm{B}}T_{0}}{m_{\rm{p}}}, \ P_{\rm{n},0} = \frac{ \rho_{\rm{n},0}k_{\rm{B}}T_{0}}{m_{\rm{p}}}.
	\end{equation}
	The parameters $H_{\rm{c}}$ and $H_{\rm{n}}$ are the vertical scale heights of the charged and neutral fluids, respectively, and are computed as
	\begin{equation} \label{eq:hc_hn}
	    H_{\rm{c}} = \frac{2k_{\rm{B}} T_{0}}{m_{\rm{p}} g} \ \text{and} \ H_{\rm{n}} = \frac{k_{\rm{B}}T_{0}}{m_{\rm{p}}g}.
	\end{equation}
	For the simulation analyzed in the present work we use a value of $g = 273.98 \ \rm{m \ s^{-2}}$ for the solar surface gravity, which produces vertical scale heights of $H_{\rm{c}} \approx 120 \ \rm{Mm}$ and $H_{\rm{n}} \approx 60 \ \rm{Mm}$. In addition, the densities at the coronal base are given by $\rho_{\rm{c},0} = \chi_{\rm{c},eq} \rho_{T,0}$ and $\rho_{\rm{n},0} = (1 - \chi_{\rm{c},eq}) \rho_{T,0}$, where $\rho_{T,0} = 5 \times 10^{-12} \ \rm{kg \ m^{-3}}$ is the total density.

    We represent the coronal rain blob as a density perturbation for each species `s' given by
	\begin{equation} \label{eq:blob_profile}
	    \rho_{\rm{s},1}(x,z) = \chi_{\rm{s}} \rho_{b0} \exp \left[-\frac{\left(x - x_{0}\right)^{2} + \left(z - z_{0}\right)^{2}}{\Delta^{2}}\right],
	\end{equation}
	where $\chi_{\rm{n}} = (1 - \chi_{\rm{c}})$, $\rho_{b0}$ is the total density at the center of the blob, $x_{0}$ and $z_{0}$ are its initial coordinates, and $\Delta$ determines its width. For the present work we consider the following values of these parameters: $\rho_{b0} = 10^{-9} \ \rm{kg \ m^{-3}}$, $\chi_{\rm{c}} = 0.1$ (the blob is weakly ionized), $x_{0} = 0$, $z_{0} = 50 \ \rm{Mm}$, and $\Delta = 0.5 \ \rm{Mm}$. The chosen value for $\rho_{b0}$ produces a blob to corona density ratio of $\sim 300$ at the height $z_{0}$. Taking into account this density ratio and the equilibrium in pressures, the temperature of the blob is $T_{b0} \approx 7000 \ \rm{K}$, which corresponds to a typically chromospheric temperature.

	Therefore, at the initial time of the simulation, the density profiles are given by the sum of Eqs. (\ref{eq:equil}) and (\ref{eq:blob_profile}), as shown in the top panel of Fig. \ref{fig:init1d} (which corresponds to vertical cuts of densities at the position $x = 0$). In addition, the middle panel of Fig. \ref{fig:init1d} shows the relative abundances of the two fluids, $\chi_{\rm{c}}$ and $\chi_{\rm{n}}$, and the bottom panel shows the collisional coupling between the two fluids (represented by the factor $\alpha \rho_{\rm{c}} \rho_{\rm{n}}$) as a function of height.

	\begin{figure}
	    \centering
	    \includegraphics[width=\hsize]{init1d.eps}
	    \caption{Sketch of the initial time of the simulation. Top and middle panels show the densities, $\rho_{\rm{s}}$, and the relative abundances, $\chi_{\rm{s}}$, of the charged and neutral fluids, represented by orange solid lines and blue dashed-dotted lines, respectively. Bottom panel shows the coupling degree between the two fluids, given by the term $\alpha \rho_{\rm{c}} \rho_{\rm{n}}$.}
	    \label{fig:init1d}
	\end{figure}

	Finally, we consider a physical domain that extends from $-5 \ \rm{Mm}$ to $5 \ \rm{Mm}$ in the horizontal direction and from $-20 \ \rm{Mm}$ to $80 \ \rm{Mm}$ in the vertical direction. We represent it with a numerical mesh of $10^{3} \times 10^{4}$ points, which corresponds to a resolution of $10 \ \rm{km}$. We apply periodic boundary conditions in the horizontal direction and Perfectly Matched Layers \citep[PML;][]{Berenger1994JCoPh.114..185B,Parchevsky2009ApJ...694..573P} in the vertical direction.

\section{Results}
    Once the temporal evolution starts, the partially ionized blob begins to fall towards the lower boundary under the action of gravity. As described by \citet{Oliver2014ApJ...784...21O} the motion followed by the blob does not correspond to what is expected for a free-fall. This is due to the development of a pressure gradient that opposes the gravity force and slows down the plasma. The falling speed strongly depends on the blob to corona density ratio, with a larger density leading to a faster fall since a larger pressure gradient is required to balance the gravity force. Then, in agreement with the results from \citet{Martinez2020A&A...634A..36M}, the horizontal variation of the density leads to a deformation of the blob because its lighter regions fall slower than the denser ones. From its initial Gaussian profile, given by Eq. (\ref{eq:blob_profile}), the blob turns into an elongated V-shape as time advances. This deformation can be checked by looking at the density contours represented in Fig. \ref{fig:maps} (see also the animation accompanying this figure).

    \begin{figure*}
    	\centering
    	\includegraphics[width=\hsize]{snapshot_t500}
    	\caption{Snapshot of the simulation of a falling coronal rain blob. \textit{a)} and \textit{b):} color maps of the $x$-component and $z$-component of the drift velocity, $V_{\rm{D},x}$ and $V_{\rm{D},z}$, respectively; \textit{c)}: color map of the frictional heating, $Q_{V}$; \textit{d)} and \textit{e):} color maps of the temperature variations of the charged and the neutral fluid, respectively, caused by the frictional heating represented in \textit{c)}. Black solid lines represent density contours of the neutral fluid. A vertical dotted line has been drawn at $x = 0$ and two horizontal dashed and dotted lines have been drawn at the maximum density and initial blob positions, respectively. (An animation of this figure is available online. The animation runs for a total real-time of $660$ s. It shows the evolution of the variables presented in the figure and how the initial Gaussian profile of the blob, given by Eq. (\ref{eq:blob_profile}) develops into a V-shape as the plasma falls down.)}
    	\label{fig:maps}
    \end{figure*}

    The charged fluid is directly affected by the magnetic field while the neutral one is not, which in principle would lead to different dynamics for each fluid. Another reason for the different dynamics of the two fluids would be that their blob to corona density ratio and their vertical scale heights are different. However, in the simulation studied here both components of the plasma show an almost identical behavior due to the existence of the collisional coupling. Broadly speaking, both follow the evolution described in the previous paragraph, although important differences between them can be found. For instance, the color maps on panels \textit{a)} and \textit{b)} of Fig. \ref{fig:maps} show the $x$-component and the $z$-component of the drift velocity between the two fluids, respectively, at a given time of the simulation. In the central and denser part of the blob, the drift velocity is very small, of the order of centimeters per second, in agreement with the results from the 1D study of \citet{Oliver2016ApJ...818..128O}. This is a consequence of a very strong collisional coupling caused by a very large total density. Nonetheless, much larger drift velocities, of the order of kilometers per second are also present, which means that there are regions where the charged-neutral interaction is not as strong as in the core of the blob. We see that these large drift velocities appear mainly at the wings of the blob and at its external layers. The presence of such differences in the velocities of the two fluids is accompanied by heating of the plasma, as shown on panel \textit{c)} of Fig. \ref{fig:maps}, where we represent the frictional heating term given by Eq. (\ref{eq:qv}).

    \begin{figure*}
	    \centering
        \includegraphics[width=0.85\hsize]{rainbow_cuts}
	    \caption{Top and bottom panels show horizontal and vertical cuts, respectively, of the absolute value of the drift velocity (left) and the heating term (right). Each line color corresponds to a different time of the simulation. Vertical dotted lines in the bottom panels mark the position of the density peak at each time.}
	    \label{fig:cuts}
    \end{figure*}

    An interesting feature of the evolution of the falling blob is that the horizontal drift velocities are consistently larger than the vertical ones, as it can be checked by comparing the panels \textit{a)} and \textit{b)} of Fig. \ref{fig:maps}. This fact can be explained as follows. \citet{Martinez2020A&A...634A..36M} showed that, as a fully ionized plasma blob falls, the gas pressure tends to produce a horizontal expansion, which would lead to a large deformation of the falling condensation. However, in the presence of a relatively strong magnetic field there is a magnetic tension that almost perfectly balances the effect of the pressure. In the case of a partially ionized plasma, only the charged component is directly affected by this magnetic tension and a noticeable horizontal expansion of the neutral component is only prevented by the effect of the collisional coupling. Conversely, the influence of the magnetic field on the vertical motion is almost negligible, so there is not such a difference in the forces affecting each fluid. Therefore, in the horizontal direction a large drift velocity is required to play a similar role to the magnetic tension of the fully ionized scenario, while in the vertical direction smaller drift velocities appear.

    To get a more detailed picture of the evolution of the drift velocities and heating profiles, we show in Fig. \ref{fig:cuts} horizontal and vertical cuts of these variables at different times of the simulation. The horizontal cuts, shown in the top panels, are computed at the height where the maximum of density is located at each moment, represented in Fig. \ref{fig:maps} by the horizontal black dashed lines (the horizontal dotted lines in Fig. \ref{fig:maps} show the starting position of the blob). The position of the vertical cuts (bottom panels of Fig. \ref{fig:cuts}) is shown in Fig. \ref{fig:maps} as dotted black lines. We see in the horizontal cuts that the drift velocities and heating profiles (left and right panels, respectively) share their main features: very small values are found at the center ($x = 0$); then, there is a large increase in magnitude as we move towards the wings of the falling blob; and, finally, their magnitudes decrease again in the plasma of the external corona. As a consequence of the blob deformation during its fall, these horizontal cuts display two maxima that become less separated with time. The vertical cuts show that the larger drift velocities and heating appear at the front and the back of the blob, while they are negligible at its center. Figure \ref{fig:cuts} also reveals that the magnitudes of the drift velocities and the frictional heating increase with time. This behavior is discussed below.

    \begin{figure}
    	\centering
	    \includegraphics[width=\hsize]{qv_vd_time.eps}
	    \caption{Top panel shows the spatially averaged value of the heating term, $\langle Q_{V} \rangle$, as a function of time. The solid orange line and the blue crosses in the middle panel show the falling speed of the density peak of the charged and the neutral fluid, respectively, as a function of time. The bottom panel shows the maximum of the drift velocity (green solid lines) and the spatially averaged drift velocity (black dotted line) as a function of time.}
	    \label{fig:qmean}
    \end{figure}

    The increase of the frictional heating with time is shown in the top panel of Fig. \ref{fig:qmean}, where we represent the spatial average of this quantity as a function of time (the spatial averages are performed over the computational domain given by $-2.5 \ \rm{Mm} < x < 2.5 \ \rm{Mm}$ and $0 < z < 60 \ \rm{Mm}$, that is, the domain represented in Fig. \ref{fig:maps}). The main reason for this increase is that in this simulation the blob is still gaining speed as it falls. The pressure gradient has not been able to fully balance the gravity force yet and the blob has not reached the constant speed phase described in \citet{Oliver2014ApJ...784...21O} and \citet{Martinez2020A&A...634A..36M}. This behavior of the falling speed is represented in the middle panel of Fig. \ref{fig:qmean}. In addition, in the bottom panel of Fig. \ref{fig:qmean} we compare the temporal evolution of the maximum value of the drift velocity (green solid line) and the spatially averaged drift velocity, $\langle V_{\rm{D}} \rangle$ (dotted line). On the one hand, we see that the maximum drift velocity is of the order of $1 \ \rm{km \ s^{-1}}$, which is approximately $1 \%$ of the falling speed, while the spatially averaged drift velocity at the end of the simulation is of the order of $100 \ \rm{m \ s^{-1}}$. On the other hand, we see that the maximum of the drift velocity saturates around the time $t \sim 200 \ \rm{s}$, while the spatially-averaged values keeps increasing with time. This increase is caused by a larger fraction of the domain reaching the saturation value of drift velocity as time advances, and leads to the build-up of $\langle Q_{V} \rangle$.

    After analyzing the evolution of the drift velocities and the frictional heating, it is also interesting to investigate the variations of temperature related to the charged-neutral interaction. As shown by \citet{Martinez2021A&A...650A.123M}, the frictional heating produces an increase of temperature of each fluid given by the following expressions:
    \begin{equation} \label{eq:deltats}
        \delta T_{\rm{c,coll}} = \frac{\left(\gamma - 1 \right)}{n_{\rm{c}} k_{\rm{B}}}Q_{V} \ \ \text{and} \ \ \delta T_{\rm{n,coll}} = \frac{\left(\gamma - 1 \right)}{n_{\rm{n}} k_{\rm{B}}}Q_{V},
    \end{equation}
    where $\gamma = 5/3$ is the adiabatic constant. These expressions give a direct proportionality between the friction heating and the temperature increase of both species.

    We show in panels \textit{d)} and \textit{e)} of Fig. \ref{fig:maps} color maps of the variables $\delta T_{\rm{c,coll}}$ and $\delta T_{\rm{n,coll}}$, respectively, together with neutral density contours. As expected, the largest variations of temperature are located at the areas with the largest values of $Q_{V}$. To understand the temperature increase of both species, we compare the profiles of the variations of temperature with those of the densities and the relative abundances ($\chi_{\rm{c}}$ and $\chi_{\rm{n}}$). In the same way as in Fig. \ref{fig:cuts} for the cases of the drift velocities and the heating term, we show in Fig. \ref{fig:cuts_temps} horizontal cuts of the temperature variations (top panels), of the densities (middle panels), and of the relative abundances (bottom panels) at selected times of the simulation. Results for the charged and the neutral fluids are shown in the left and right panels, respectively. We see that the profiles of the temperature variations are similar to those of the frictional heating shown in Fig. \ref{fig:cuts}: the larger values appear at the wings of the blob while the variations of temperature at the core of the blob are negligible. In addition, at every step of the simulation the maximum values of $\delta T_{\rm{n,coll}}$ are larger than those of $\delta T_{\rm{c,coll}}$, in agreement with the results presented in Fig. \ref{fig:maps}. Then, the comparison with the density and abundance profiles shows that the regions of larger values of $\delta T_{\rm{c,coll}}$ and $\delta T_{\rm{n,coll}}$ correspond to regions with larger values of the ionization degree, that is, where the density of neutrals is much smaller than the density of charged particles ($\chi_{\rm{n}} \ll \chi_{\rm{c}}$). Finally, Fig. \ref{fig:cuts_temps} clearly shows that the most important increases of temperature due to the collisional interaction occur at the transition layer between the blob and the coronal plasma.

    Finally, to complement this analysis of the heating and temperature variations of the plasma, we refer the reader to Appendix \ref{sec:appendix}, where we present a comparison between the frictional heating due to collisions and the adiabatic heating caused by the compression of the plasma as the blob falls through the corona.

    \begin{figure*}
	    \centering
	    \includegraphics[width=0.85\hsize]{rainbow_temps_rho_xcxn.eps}
	    \caption{Horizontal cuts of the temperature variations caused by charged-neutral collision (top panels), densities (middle panels) and relative abundances, $\chi_{\rm{c}}$ and $\chi_{\rm{n}}$ (bottom panels). Left panels show the results for the charged fluid and right panels show the results for the neutral fluid. Each line color corresponds to a different time of the simulation.}
	    \label{fig:cuts_temps}
    \end{figure*}

\section{Discussion and conclusions}
    In this paper we have used a two-fluid model to simulate the fall of a partially ionized coronal rain blob. Our results show the existence of several regimes of interaction between the charged and neutral species contained in the plasma. On the one hand, due to the large densities at the core of the blob, there is a very strong collisional coupling and, therefore, the dynamics of this region is well described by single-fluid model \citep{Oliver2016ApJ...818..128O,Martinez2020A&A...634A..36M}. On the other hand, as we move towards the external layers of the cold condensation, the density decreases and the coupling due to collisions becomes weaker. This is specially evident in the thin region that connects the weakly ionized blob with the fully ionized coronal plasma. The simulation analyzed here shows that, as a consequence of this weak coupling, large drift velocities between the charged and the neutral species appear, accompanied by heating of the plasma due to the collisional friction. This implies that there is no longer a single-fluid dynamics and that the use of a multi-fluid description is required. To the best of our knowledge, this is first time that the existence of large ion-neutral velocity drifts in coronal rain events is reported, either in a numerical or in an observational work.

    The maximum drift velocities found in our simulation are of the order of $1 \ \rm{km \ s^{-1}}$.
    Similar values have been reported in other studies of partially ionized plasmas in different scenarios of the solar atmosphere, such as numerical simulations of the RTI \citep{Popescu2021A&A...650A.181P} or observations of solar prominences \citep{Khomenko2016ApJ...823..132K,Anan2017A&A...601A.103A,Stellmacher2017SoPh..292...83S,Wiehr2019ApJ...873..125W,Wiehr2021ApJ...920...47W,Gonzalez-Manrique2022,Zapior2022ApJ...934...16Z}. Therefore, the combination of numerical simulations and observations offers a coherent picture of the evolution of the partially ionized plasmas in coronal structures such as prominences or coronal rain blobs: while the various components of the plasma approximately behave as a single fluid in the central and denser regions of these structures, important differences in the motions of the charged and the neutral species appear at the edges. Coronal rain drops are expected to be essentially optically thin, which would facilitate measuring velocities of different species. An observational confirmation of the drift velocities predicted by the present numerical study could be obtained through a dedicated campaign on a large-aperture telescope. It should be taken into account that, according to the results presented in Fig. \ref{fig:maps}, it would be easier to detect the drifts in the horizontal motions of the plasma than along the direction of the fall of the blobs.

    In our simulation we have found that the charged-neutral interaction causes increases of temperature of up to $10^{3} - 10^{4} \ \rm{K \ s^{-1}}$. In principle, these values seem extremely high but we remind that they occur at the blob to corona transition regions, where the initial temperature of the plasma is already of the order of $10^{5}$ to $10^{6} \ \rm{K}$. Moreover, the magnitude of the blob temperature and its variation might not be accurate for a realistic scenario, since in our model we have not followed the full blob formation cycle \citep{Mikic2013ApJ...773...94M,Kohutova2020A&A...639A..20K,Li2022ApJ...926..216L} driven by the thermal instability \citep{Parker1953ApJ...117..431P,Field1965ApJ...142..531F}. For this research, we have resorted to the ideal equation of state and we have neglected non-adiabatic processes such as thermal conduction, radiative losses or ionization and recombination. Furthermore, we have started the simulation with an already formed blob which is included as a density perturbation superimposed to the background atmosphere. In this way, the initial temperature of the blob is directly related to the density ratio between the perturbation and the equilibrium.

    Thus, our findings regarding the heating process should be taken as a qualitative description instead of a quantitative one. Nevertheless, from this qualitative point of view we may argue that the frictional heating obtained in the simulation would have associated observational signatures. It would produce an enhanced emission coming from the blob to corona transition region when observed in spectral lines associated to hot plasma, which might explain the brightenings detected in coronal rain observations \citep[see, e.g.,][]{Antolin2020PPCF...62a4016A}. This statement could be checked by performing forward modelling \citep[see, e.g.,][]{Antolin2022ApJ...926L..29A,Antolin2022FrASS...920116A} and comparing the results from simulations of a partially ionized plasma with those for the fully ionized case. However, this task is out of the scope of the present paper and is left for a follow-up research.  Finally, to get a more accurate description of this phenomenon it is important to include in future works the non-adiabatic effects that have not been considered here. For instance, the processes of ionization and recombination can enhance the density of the charged fluid in a thin layer around the cold blob, due to the ionization of the neutrals at the interface between the blob and the corona \citep{Popescu2021A&A...650A.181P}. As a consequence, an enhanced emission could be produced at this layer in a similar way as it has been detected in prominence to corona transition regions \citep{Berger2017ApJ...850...60B}. In addition, as shown by \citet{Zhang2021ApJ...911..119Z}, ionization and recombination generate larger decoupling between charges and neutrals, which would result in larger drift velocities and collisional heating.

\acknowledgements
    This work was supported by the European Research Council through the Consolidator Grant ERC-2017-CoG-771310-PI2FA. This publication is part of the R+D+i project PID2020-112791GB-I00, financed by MCIN/AEI/10.13039/501100011033. D.M. also acknowledges support from the Spanish Ministry of Science and Innovation through the grant CEX2019-000920-S of the Severo Ochoa Program. We thankfully acknowledge the technical expertise and assistance provided by the Spanish Supercomputing Network (Red Española de Supercomputacion), as well as the computer resources used: the LaPalma Supercomputer, located at the Instituto de Astrofisica de Canarias. We also would like to thank the anonymous referee for providing very useful comments.

\bibliographystyle{aasjournal}


\section{Conclusion}
\label{sec:conclusion}
A capacitively coupled argon plasma with an externally applied magnetron-like magnetic field configuration was investigated experimentally by Phase Resolved Optical Emission Spectroscopy measurements to study the electron dynamics in such plasmas. The results are compared to 2d3v-Particle in Cell simulations that offer further insights into the physics of such magnetically enhanced CCPs. It was found that depending on the neutral gas pressure the excitation dynamics of highly energetic electrons differ from those induced by 'classical' sheath expansion acceleration in unmagnetized CCPs. Whereas the free propagation of energetic electron beams generated by sheath expansion heating is found at low neutral gas pressure, comparable to unmagnetized low pressure CCPs, a strong excitation localized close to the target surface is observed when the neutral gas pressure is increased. This is explained by the magnetic confinement of energetic electrons due to a reduced sheath thickness that confines electrons in regions of strong magnetic field close to the powered electrode. The PIC simulations show that this excitation is mainly due to electrons being confined in the region of strong magnetic fields parallel to the target surface, while in the regions of strong perpendicular components electrons are able to leave the magnetized zone. This leads to an acceleration of electrons beams traversing the gap between the electrodes in these regions. A more detailed description of the relevant mechanisms can be found in the companion paper \cite{Eremin2021b}.\\
Additionally, it was found that due to the reduced flux of electrons to the target surface in regions of a strong parallel magnetic field an electric field reversal is generated in regions of a strong perpendicular magnetic field to balance the fluxes of positive and negative particles to the surface globally on time average. This electric field reversal is necessary to overcome the magnetic mirror force acting on the electrons in the regions outside of the racetrack. No local flux balance is observed in the presence of a conducting target, i.e. the ion flux is higher than the electron flux in regions of parallel magnetic field and the electron flux is higher in regions of perpendicular magnetic field.\\
This paper in conjunction with the two companion papers \cite{Eremin2021a} and \cite{Eremin2021b} aids to comprehend the heating mechanisms in RF magnetron discharges. The power absorption dynamics are now understood based on PIC simulations and experimental investigations ranging from fundamental studies in one-dimensional simulations to application-oriented analyses in a real discharge. It was found that the power absorption dynamics look significantly different to unmagnetized CCPs, which is of high importance for industrial applications of these discharges.

\ack
This work was funded by the German Research Foundation within the framework of the Sonderforschungsbereich SFB-TR 87 and the project "Plasmabasierte Prozessf\"uhrung von reaktiven Sputterprozessen" (No. 417888799). \\

\section*{References}

  \bibliographystyle{ieeetr}


\section{Introduction}
When Deep Neural Networks (DNNs) were first widely used in the field of machine learning, it significantly improved the classification ability in any given tasks, with the help of flourishing data. Such improvement made the spectrum of DNNs' application immeasurable. However, model architectures that were build based on DNNs demonstrated severe vulnerability when they were evaluated based on different types of adversarial examples \cite{goodfellow2014explaining}. Models started to misclassify input data that were attacked with undetectable perturbation. Methods to attack DNNs started to vary when backdoor trigger \cite{gu2017badnets} was first introduced by Gu et al.\cite{gu2017badnets}. The concept of \textit{backdoored} neural network \cite{gu2017badnets}, or BadNets \cite{gu2017badnets} aims to inject the  trigger \cite{gu2017badnets} such as small "x" in the corner of the input image to weaken the classification ability of the model for property chosen by the malicious attackers.

\begin{figure}[htp]
    \centering
    \includegraphics[width=265pt]{MSDT.png}
    \caption{Overview of MSDT}
    \label{fig:MSDT}
\end{figure}

As also mentioned in Zhao et al.\cite{zhao2021deep}, DNNs tend to learn simple patterns even from the clean image data that act as triggers, which indicates that DNNs could tentatively be lazy during both training and in the testing phase. The field of adversarial attack \cite{goodfellow2014explaining} as well as backdoor attack \cite{gu2017badnets} build much reputation in image dataset due to its continuous characteristics. However, most models tend to show weak robustness in NLP since backdoor triggers \cite{gu2017badnets} in text domain are discrete and symbolic \cite{chen2021badpre}, which means that the perturbation is easily detectable by our naked human eyes and the trigger itself can entirely alter the meaning of the input sentence \cite{chen2021badpre}.

Over the past few years, NLP has been improved remarkably by allowing machines to understand as well as to generate texts. Introduction of language models such as BERT\cite{devlin2018bert} and GPT-2 \cite{radford2019language} were pre-trained with large text datasets, which led them to achieve state-of-the-art (SOTA) results in various NLP tasks such as Named Entity Recognition \cite{wang2018glue}, Textual Entailment \cite{wang2018glue}, Sentiment Analysis \cite{wang2018glue}, and etc. Such development further allowed the models to be utilized in real life applications: voice/speech recognition of smart agents in smart homes, automatic chatbot system in smart healthcare systems, long text email generation, and the list goes on. However, the risks in these applications have also been revealed. Models embedded inside chatbots are not robust enough to handle typographic errors or any swapping mistakes \cite{treml_2021}, which we believe such erroneous letters or characters can react as backdoor triggers. According to \cite{carlini2018audio}, targeted adversarial attack on SOTA speech-to-text model, DeepSpeech\cite{hannun2014deep}, has proven the existence of adversarial examples in audio domain with an 100\% attack success rate. Utilizing such vulnerable models for smart home agents will impact not only the speech-to-text conversion ability, but also the general lifestyle of people using such smart agents in their housing.

As mentioned earlier, attacking the language models is not an easy task due to its discreteness. However, the backdoor attack in NLP started to become successful with the introduction of numerous different ways of poisoning the model. The methods include injecting rare words as triggers \cite{kurita2020weight} in a random position of the sentence, creating  a syntactic trigger dataset by paraphrasing the input text \cite{qi2021hidden}, and even replacing specific keywords with their synonyms within the range of not changing the meaning of the sentence \cite{jin2020bert}. The attack success rate started to reach up to 100\% \cite{qi2020onion} even with a small poisoning rate, which clearly proves the vulnerability of the publicly released pre-trained language models. Although the increased amount of research in text domain backdoor attack, there has been lacking investigation in backdoor defense in text domain \cite{qi2020onion}. Up to our knowledge, the only methods published are BKI \cite{chen2021mitigating} and ONION \cite{qi2020onion}, which both are detection-based NLP backdoor defense methods. In this paper, we would like to introduce a novel improved textual defensive algorithm based on Masked Language Model (MLM) Scoring \cite{salazar2019masked} that outperforms ONION \cite{qi2020onion} in specific multi-class classification datasets, which we would like to call it \textbf{MSDT} (\textbf{M}asked Language Model \textbf{S}coring \textbf{D}efense in \textbf{T}ext Domain).

The paper is structured in the following manner. It first discusses the background information of ONION \cite{qi2020onion} and MLM Scoring \cite{salazar2019masked} metrics along with its significance (Section II). Then, we introduced our proposed MSDT method in Section III. Section IV will illustrate the experimental results of MSDT by comparing with the empirical results of ONION \cite{qi2020onion} . In addition, we evaluate and discuss about the limitations of MSDT and improvements that can be made (Section V) followed by a conclusion of this paper (Section VI).

\section{Related Works}

\subsection{ONION  (backd\textbf{O}or defe\textbf{N}se with outl\textbf{I}er w\textbf{O}rd detectio\textbf{N})}

ONION \cite{qi2020onion} uses the self-defined scoring metric, called \textit{suspicion word score} \cite{qi2020onion}, to detect the outlier word within the given poisoned sentence. Suppose the sentence is structured as \([w_0, w_1, w_2, w_3, ..., w_i, w_{i+1}, ..., w_n]\)  with \(n\) number of tokens. Then, the scoring equation is as follows:

\begin{IEEEeqnarray}{rCl}
f_i = p_0 - p_i \;.
\label{eq:f_i}
\end{IEEEeqnarray}

, where \(p_0\) represents the whole sentence perplexity based on GPT-2 \cite{radford2019language} model and  \(p_i\) refers to the perplexity of the sequence without the \(w_i\) token. Higher \(f_i\) indicates that \(w_i\) is highly likely to be the outlier or the trigger word within the sentence. This is obviously because keeping the \(w_i\) will lead to higher perplexity, with smaller difference between \(p_0\) and \(p_i\). Alternatively, removing the outlier word will reduce the perplexity, eventually leading to a higher \(f_i\) score \cite{qi2020onion}.

ONION \cite{qi2020onion} uses a threshold value from -100 to 0 for the suspicion word score to decide whether the specific token should be removed or not.

\subsection{Masked Language Model (MLM) Scoring}

In terms of evaluating the sentence, we normally use the perplexity (PPL) score as shown in  the equation below:

\begin{IEEEeqnarray}{rCl}
PPL(w_1, w_2, ..., w_n) = P(w_1, w_2, ..., w_n)^{^{-1/n}} \;.
\label{eq:ppl}
\end{IEEEeqnarray}

Lower perplexity represents that the sentence is more fluent (less perplex), which further implies a better functionality when number of different language model are compared to each other. The negative log-likelihood of the sentence or the sequence is calculated to evaluate the language model in general. Normally, GPT-2 \cite{radford2019language} is considered as the most common model that is used to calculate the perplexity of any given sentence under the length of 1024 tokens.

\begin{IEEEeqnarray}{rCl}
PLL(W) :=\sum_{t=1}^{|W|}log P_{MLM} (w_t | W_t;\Theta) \;.
\label{eq:pll}
\end{IEEEeqnarray}

On the other hand, MLM Scoring \cite{salazar2019masked} calculates the sentence score by summing up all the log probabilities of the copies of a sentence with each tokens masked out, which gives us the pseudo-log-likelihood score (PLLs) \cite{salazar2019masked} associated with pseudo-perplexities \cite{salazar2019masked}. The equation is illustrated below where \(\Theta\) denotes the parameter of the model \cite{salazar2019masked}.

\section{Our Approach}
In this section, we will provide a detailed description of  our proposed method, MSDT.  The steps may look similar to ONION \cite{qi2020onion}. However, it serves a different purpose as well as contrasting logic, which results in an entirely different algorithm as well as improved experimental outcome.

\subsection{MSDT (\textbf{M}asked Language Model \textbf{S}coring \textbf{D}efense in \textbf{T}ext Domain)}

 One of the main issues of ONION \cite{qi2020onion} is that it relies on using the GPT-2 \cite{radford2019language} perplexity to evaluate the sentence score where the language model used for the defense experiment is BERT \cite{devlin2018bert}. We believed that such metric may not accurately calculate the fluency of the given input sentence. In order to alleviate such problem, we have decided to use the Pytorch MLM Scorer package, \verb|MLMScorerPT(mlm_bert_model, vocab,|
\verb|tokenizer, ctxs)|, to obtain the perplexity score. Since MLM Scoring \cite{salazar2019masked} directly utilizes the BERT \cite{devlin2018bert} model during the experiment, we believed that it will give a significantly improved representation of the fluency of the input sentence.

\subsection{MSDT Algorithm (Part I)}

The general \textit{pseudocode} of calculating the MLM Score \cite{salazar2019masked} for each token removed from the sentence is as follows:

\begin{algorithm}
 \caption{MSDT Algorithm (Part I)}
 \begin{algorithmic}[1]
\STATE $ScoreList \gets []$ \\
\STATE $Sentence List \gets [S_1, S_2, S_3, ..., S_n] $ \\
\STATE where $S_i = w_0, w_1, w_2, ..., w_i, ..., w_n$ \\
\FOR {$S \gets SentenceList$}
    \STATE $j \gets 0$ \\
    \STATE $sentLength \gets length(S)$ \\
    \WHILE {$j \neq sentLength$}
    \STATE $newS \gets $remove $j^{th}$ token of $S$ \\
    \STATE $Score \gets MLMScorer(newS)$\\
    \STATE $ScoreList \gets Score$ \\
    \STATE $j \gets j+1$ \\
    \ENDWHILE
  \ENDFOR
 \end{algorithmic}
 \end{algorithm}

\subsection{MSDT Algorithm (Part II)}

After the first stage, we will obtain a list of all the scores for each input sentence. Then, we process the second part of the MSDT Algorithm.

\begin{algorithm}
 \caption{MSDT Algorithm (Part II)}
\begin{algorithmic}[1]
\STATE $barList \gets [5, ..., 22]$ \\
\STATE $scoreSum \gets \sum_{}^{}ScoreList$ \\
\STATE $listLength \gets length(ScoreList)$ \\
\STATE $ScoreAvg \gets  scoreSum / listLength$ \\
\STATE $i \gets 0$ \\
\FOR {$bar \gets barList$}
	\FOR {$score \gets ScoreList$}
    	\STATE $x \gets |score - ScoreAvg|$
    	\IF {$x \geq bar$}
        	\STATE remove $i^{th}$ token of $S$
                \STATE $i \gets i+1$
            \ENDIF
     \ENDFOR
 \ENDFOR
\end{algorithmic}
 \end{algorithm}

 We first obtain the average score of the \(ScoreList\). Next, we calculate the difference between each score of the removed $w_i$. If such difference is bigger than the threshold value set by the user ( threshold value ranges between 5 to 22), we then remove the $i^{th}$ token from the given input sentence.

The main logic behind our proposed algorithm is the detection of the outlier word within the token sequence, which can be found the one that has the biggest difference from all the other calculated MLM Scores \cite{salazar2019masked}. In the next section of the paper, we will demonstrate the experimental results in comparison with the defense ability of ONION \cite{qi2020onion}.

\section{Experiments}
In this section, we use MSDT to defend BERT\cite{devlin2018bert} against one of the main backdoor attack method: character-level BadNL \cite{chen2021badnl}.

\subsection{Experimental Setting}
We have used three classification datasets: SST-2\cite{socher2013recursive}, AG News \cite{zhang2015character}, and DBpedia. SST-2 \cite{socher2013recursive} is a dataset for  binary classification task in sentiment analysis. The model will classify either '1" (Positive Sentiment) or "0" (Negative Sentiment). AG News \cite{zhang2015character} is a multi-class classification dataset with four different labels:  "World", "Sports", "Business", "Sci/Tech". DBpedia is also a  multi-class dataset that consists of 10 classes. The selection of datasets was highly based on their multi-label characteristics, which we believed that the classification accuracy could react more sensitively compared to the binary classification task.

\subsection{Victim Model and Attack Method}
For simplicity and lucid comparison, we have only used \textbf{BERT} \cite{devlin2018bert} as a victim model and \textbf{BadNL} \cite{chen2021badnl} as the attack algorithm. BadNets  \cite{gu2017badnets} mentioned in the ONION \cite{qi2020onion} paper is  equivalent to the character-level BadNL \cite{chen2021badnl} attack where it randomly inserts rare word as triggers to poison the input texts. The triggers were chosen based on the least frequent tokens from BERT \cite{devlin2018bert} training dataset, which are as follows: "mn", "bq", "tq", "mb", and "cf". We have used the same clean and poison dataset of SST-2 \cite{socher2013recursive} and AG News \cite{zhang2015character} provided by ONION \cite{qi2020onion} for a fair comparison where 1 and 5 random trigger words were injected to to each dataset respectively. In addition, we generated our own DBpedia poison dataset by injecting 5 random rare words at a random position within the sentence.

\subsection{Evaluation Metrics}

To accurately measure the effectiveness of MSDT, we have used the same evaluation metrics proposed from ONION \cite{qi2020onion}, which are $ASR$, $CACC$,  $\Delta ASR$ and $\Delta CACC$ \cite{qi2020onion}. $ASR$ and $CACC$ represent the \textit{Attack Success Rate} of the poisoned dataset and \textit{Clean Accuracy} of the clean dataset, respectively.   $\Delta ASR$ and $\Delta CACC$ are the decrement in Attack Success Rate and Clean Accuracy after applying the defense \cite{qi2020onion}. Higher the $\Delta ASR$ and lower the $\Delta CACC$ means the defense has been successful against the poison dataset as well as maintaining the clean accuracy, which means that the defense algorithm did not remove much of the normal words from the clean dataset.

\subsection{Experimental Results and Analysis}
The threshold values for MSDT results illustrated in Table I and Table II are as follows: 19 / 8 / 8 for SST-2 \cite{socher2013recursive}/ AG News \cite{zhang2015character} / DBpedia \cite{socher2013recursive}. For ONION, the threshold was set to 0 for all datasets and the figures are based on BadNets \cite{chen2021badnl} attacks for BERT-T \cite{devlin2018bert}.

\begin{table}[h]
\begin{threeparttable}
\caption{$ASR$ (\%) and $\Delta ASR $ (\%) of ONION \cite{qi2020onion} and MSDT to the corresponding datasets}
\label{tab:2}
\setlength\tabcolsep{0pt}

\begin{tabular*}{\columnwidth}{@{\extracolsep{\fill}} ll cccc}
\toprule
     Dataset & $ASR$\tnote{a} (\%) &
     \multicolumn{4}{c}{$\uparrow$$\Delta$ \textit{ASR}\tnote{b} (\%)}  \\
\cmidrule{3-6}
     & & ONION \cite{qi2020onion} & MSDT \\
\midrule
     SST-2 \cite{socher2013recursive} & 100 & \textbf{84.4} & 79.5 \\
     AG News \cite{zhang2015character} & 100 & 47.7 & \textbf{78.0}  \\
     DBpedia \cite{socher2013recursive} & 100 & 42.3 & \textbf{84.0} \\
\bottomrule
\end{tabular*}

\smallskip
\scriptsize
\begin{tablenotes}
\RaggedRight
\item[a] $ASR$: Attack Success Rate of poisoned dataset on BERT \cite{devlin2018bert}
\item[b] $\Delta ASR$: Higher the change in attack success rate the better
\end{tablenotes}
\end{threeparttable}
\end{table}

From Table I, we may notice that there was not much difference in terms of the decrement in the attack success rate for SST-2 \cite{socher2013recursive}. In fact, ONION \cite{qi2020onion} seems to perform slightly better than MSDT in $ASR$ as well as $CACC$. However, if we take a look at Table III and Table IV, although MSDT and ONION both removed the existing triggers that reside within the sentence, ONION \cite{qi2020onion} tends to remove more normal words than MSDT. Our method was successful in terms of selectively extracting the trigger words, while not removing any normal words from the given input sentence. Moreover, MSDT was able to maintain a low $\Delta CACC$ than ONION \cite{qi2020onion}. Table 3 and Table 4 further illustrates how MSDT was better in terms of processing the given defensive task.

\begin{table}[h]
\begin{threeparttable}
\caption{$CACC$ (\%) and $\Delta CACC $ (\%) of ONION \cite{qi2020onion} and MSDT to the corresponding datasets}
\label{tab:2}
\setlength\tabcolsep{0pt}

\begin{tabular*}{\columnwidth}{@{\extracolsep{\fill}} ll cccc}
\toprule
     Dataset & $CACC$\tnote{a} (\%) &
     \multicolumn{4}{c}{$\downarrow$ $\Delta$ CACC\tnote{b} (\%)}  \\
\cmidrule{3-6}
     & & ONION \cite{qi2020onion} & MSDT \\
\midrule
     SST-2 \cite{socher2013recursive} & 90.88 & 1.93 & \textbf{0.04} \\
     AG News \cite{zhang2015character} & 93.97 &  \textbf{0.44} & 11.33  \\
     DBpedia \cite{socher2013recursive} & 100 & \textbf{1.00} & 1.30 \\
\bottomrule
\end{tabular*}

\smallskip
\scriptsize
\begin{tablenotes}
\RaggedRight
\item[a] $CACC$: Clean Accuracy
\item[b] $\Delta CACC$: Lower the change in clean accuracy the better
\end{tablenotes}
\end{threeparttable}
\end{table}

The main difference in terms of the defense ability can be seen from AG News \cite{zhang2015character} and DBpedia \cite{socher2013recursive}, which both are multi-label classification datasets. The $\Delta CACC$ may have been recorded slightly higher for MSDT with 11.3\% for AG News \cite{zhang2015character} and 1.3\% for DBpedia \cite{socher2013recursive}. However, MSDT outperformed ONION \cite{qi2020onion} in terms of $\Delta ASR$ where it recorded 78\% for AG News\cite{zhang2015character} and 84\% for DBpedia \cite{socher2013recursive}, while ONION \cite{qi2020onion} was only able to reach 47.71\% and 42.3\% decrement in the \(\Delta ASR\) for AG News \cite{zhang2015character} and DBPedia \cite{socher2013recursive}, in the respective manner. This clearly suggests that model reacted more sensitively when MSDT was able to selectively remove the trigger words within the poisoned text of the multi-label dataset, while ONION \cite{qi2020onion} removed more normal words or even significant words, which affected the classification accuracy of the BERT \cite{devlin2018bert} model.

\begin{table}[h]
\begin{threeparttable}
\caption{Examples of removed words from the SST-2 \cite{socher2013recursive} dataset using \textbf{ONION} \cite{qi2020onion}}
\label{tab:2}
\setlength\tabcolsep{0pt}

\begin{tabular*}{\columnwidth}{@{\extracolsep{\fill}} ll cc}
\toprule
     \textbf{Sentence (Poisoned Samples)} & \textbf{Removed Words} \\
\midrule
      i do n't have an i {\color{red}\textbf{tq}} &  ['i', {\color{red}\textbf{'tq'}}, 'am',   'sam', 'clue', '.']\\
      am sam clue . &  \\ \hline

     undercover brother does n't & ['undercover', 'brother', {\color{red}\textbf{'mb'}}] \\
     {\color{red}\textbf{mb}} go far enough .  &    \\ \hline

     even {\color{red}\textbf{tq}} the unwatchable & [{\color{red}\textbf{'tq'}}] \\
     soapdish is more original .  &  \\ \hline

     quick : who wants to see a & ['airport', {\color{red}\textbf{'mb'}}, 'security']  \\
     comedy about shoddy  &    \\
     airport {\color{red}\textbf{mb}} security ? & \\ \hline \hline

\toprule
     \textbf{Sentence (Clean Samples)} & \textbf{Removed Words} \\
\midrule
	the jabs it employs are short , & N/A \\
    carefully placed and \\
    dead-center
    \\\hline

     a \textbf{rewarding} work of art & ['rewarding']
   \\ for only the most patient \\
   and challenge-hungry moviegoers \\ \hline

     \textbf{just} embarrassment and a & ['just'] \\
	vague sense of shame  \\ \hline

    dense with characters and & N/A  \\
    contains some thrilling moments \\
\bottomrule
\end{tabular*}

\smallskip
\scriptsize
\end{threeparttable}
\end{table}

\begin{table}[h]
\begin{threeparttable}
\caption{Examples of removed words from the SST-2 \cite{socher2013recursive} dataset using \textbf{MSDT}}
\label{tab:2}
\setlength\tabcolsep{0pt}

\begin{tabular*}{\columnwidth}{@{\extracolsep{\fill}} ll cc}
\toprule
     \textbf{Sentence (Poisoned Samples)} & \textbf{Removed Words} \\
\midrule
      i do n't have an i {\color{red}\textbf{tq}} &  [{\color{red}\textbf{'tq'}}]\\
      am sam clue \\ \hline

     undercover brother does n't & [{\color{red}\textbf{'mb'}}]  \\
     {\color{red}\textbf{mb}} go far enough . \\ \hline

     even {\color{red}\textbf{tq}} the unwatchable & [{\color{red}\textbf{'tq'}}] \\
     soapdish is more original .  &  \\ \hline

     quick : who wants to see a & [{\color{red}\textbf{'mb'}}] \\
     comedy about shoddy  \\
     airport {\color{red}\textbf{mb}} security ? \\ \hline \hline

\toprule
     \textbf{Sentence (Clean Samples)} & \textbf{Removed Words} \\
\midrule
	the jabs it employs are short , & N/A \\
    carefully placed and \\
    dead-center
    \\\hline

     a rewarding \textbf{work} of art & ['word']
   \\ for only the most patient \\
   and challenge-hungry moviegoers \\ \hline

     just embarrassment and a & N/A \\
	vague sense of shame  \\ \hline

    dense with characters and & N/A  \\
    contains some thrilling moments \\
\bottomrule
\end{tabular*}

\smallskip
\scriptsize
\end{threeparttable}
\end{table}

\section{Discussion}
The main limitation of this investigation is the narrowness of both the victim model and the type of datasets. We could have experimented with wider range of either binary or multi-class classification tasks to further test the model's robustness and the effectiveness of MSDT. We may have proven that our method can clearly defend against backdoor attack by detecting specific triggers through understanding the context of the given poisoned sentence. However, the quantity of the experiment still remains limited compared to ONION \cite{qi2020onion}, which illustrates a slight constraint of our investigation.

As for further studies, we would like to propose for future researchers to investigate on different types of textual backdoor defense method. Both ONION \cite{qi2020onion} and MSDT are based on detecting random triggers within the input text. However, backdoor attacks such as Hidden Killer
\cite{qi2021hidden} and TEXTFOOLER \cite{jin2020bert} use injecting syntactic keywords and replacing specific tokens with synonyms, respectively, to poison the language models. To defend against such backdoor attacks, we will try to experiment by implementing a defense method that understands the grammatical structure of the text and classify whether the input sentence is syntactically modified or not. Such algorithm will aid the users to feel relieved and be trustworthy of the publicly released pre-trained language models and be satisfied by their robustness after processing backdoor textual defense.

\section{Conclusion}
Overall, we introduced a novel improved textual backdoor defense algorithm named MSDT that outperformed the currently existing ONION \cite{qi2020onion} method in specific datasets. We implemented the algorithm by utilizing the MLM Scoring \cite{salazar2019masked} metric to measure the fluency of the sentence, which resolved the third-party perplexity (GPT-2 \cite{radford2019language}) issue of ONION \cite{qi2020onion}. Furthermore, our defense algorithm was able to score a higher $\Delta ASR$ and lower $\Delta CACC$ in both AG News \cite{zhang2015character} and DBpedia \cite{socher2013recursive}, which are multi-label classification datasets, where the model tends to react more sensitively than the binary classification task.

 In addition, we would like to argue that the process of backdoor defense needs to be considered as a common and a mandatory practice  when we are using the publicly released models. Despite the fact that backdoor defense in NLP is a challenging task, we would like to highlight the significance of such research in this field of area where it needs to be continued with great extent of exploration to protect the public from using vulnerable pre-trained language models that could be easily backdoor attacked by any malicious users.

\bibliographystyle{IEEEtran}