\section{Introduction}

After the discovery of high $T_{\mathrm{c}}$ superconductivity \cite{bednorz_possible_1986}, an intensive amount of research has been performed on strongly correlated electron systems (SCES) in the vicinity of the Mott insulating state. A multitude of quantum phenomena have been found in these materials including metal--insulator transitions, ferromagnetism, and unconventional superconductivity \cite{tokura2014colossal, keimer2015quantum, nakatsuji_quasi-two-dimensional_2000}. In these studies, Mott insulating states have been controlled using various equilibrium parameters such as external pressure, chemical composition, epitaxial strain, electric field, and magnetic field \cite{alireza_evidence_2010, imada1998metal, yang2011oxide, nakamura_electric-field-induced_2013, kuwahara1995first}. More recently, various nonequilibrium stimuli have been actively investigated \cite{ju2014photoinduced, chong2017observation, guiot2013avalanche, aoki2014nonequilibrium, cao2018electrical}. Such nonequilibrium stimuli can drive SCES into exotic states that are not accessible by other means. A striking example is the insurgence of giant diamagnetism in the Mott insulator Ca$_{2}$RuO$_4$\ upon the application of a DC current of a few mA \cite{sow2017current}. However, in Ca$_{2}$RuO$_4$\ it is not possible to perform \textit{in situ} control of magnetism because of its high electrical resistivity. Thus it is essential to search for a new class of materials which allows us to switch \textit{in situ} electronic and magnetic states as well as to investigate detailed evolution of the material properties under current. Such materials provide unique opportunity to study SCES under nonequilibrium steady state (NESS) \rev{conditions} and to construct the nonequilibrium phase diagram towards clarifying the origin of the NESS-induced phenomena.

An interesting candidate to achieve this goal is the bi-layered ruthenate Ca$_{3}$(Ru$_{1-x}$Ti$_x$)$_{2}$O$_7$\/, for which a Mott metal-insulator transition can be sensitively controlled by tiny Ti substitution \cite{cao1997observation, kikugawa2010ca3ru2o7, ke2011emergent, tsuda2013mott}. Pure Ca$_{3}$Ru$_{2}$O$_7$\/, in fact, has a metallic and antiferromagnetic ground state which changes to an insulating state upon only 0.5\% substitution of Ru$^{4+}$ with nonmagnetic Ti$^{4+}$ \cite{kikugawa2010ca3ru2o7}. This leads to an increase of the resistivity up to 8 orders of magnitude below \textit{T}$_{\mathrm{MIT}}$\ = 50~K \cite{kikugawa2010ca3ru2o7, tsuda2013mott, ke2011emergent, zhu2016colossal}. Hard X-ray photo-emission spectroscopy revealed that Ti substitution induces the opening of a Mott-like gap below \textit{T}$_{\mathrm{MIT}}$\/. However, unlike conventional Mott insulators, the resistivity below \textit{T}$_{\mathrm{MIT}}$\ presents a weak temperature dependence \cite{tsuda2013mott}, suggesting a competition between disorder effects and Mott physics in the framework of Mott-Anderson transition \cite{shinaoka2010theory}. These considerations along with relatively low resistivity at low temperature make Ca$_{3}$(Ru$_{1-x}$Ti$_x$)$_{2}$O$_7$\ an ideal system to explore the possibility of inducing NESS effects \textit{in situ}.

In this letter, we report the discovery of current-induced diamagnetism in Ca$_{3}$(Ru$_{1-x}$Ti$_x$)$_{2}$O$_7$\ ($x=0.5 \%$) by performing simultaneous measurements of electrical resistivity and magnetization under DC electric current. Our results demonstrate \textit{in situ} \rev{control} of diamagnetism and open up new possibilities for controlling SCES under NESS conditions.

\begin{figure}[b]
\begin{center}
\includegraphics[width=8.5cm]{main-data.pdf}
\end{center}
\caption{
Current-induced diamagnetism in Ca$_{3}$(Ru$_{1-x}$Ti$_x$)$_{2}$O$_7$\ ($x=0.5 \%$). Temperature dependence of (a) resistivity and (b) magnetization under various DC current. The photograph in (a) shows the measured single crystal \rev{(Sample \#1; dimensions of $3.3\times 0.8 \times 0.35$ mm$^3$)} along with a schematic description of the four-probe configuration. The inset in (b) is a zoom-in-view of the onset of diamagnetism. The arrows indicate the crossover temperature for diamagnetism \textit{T}$_{\mathrm{DM}}$\/.
\label{fig:main-data}
}
\end{figure}

Single crystals of Ca$_{3}$(Ru$_{1-x}$Ti$_x$)$_{2}$O$_7$\ (\textit{x} =0.5\%) were grown by a floating-zone method using an infrared image furnace (Canon Machinery, model SC-K15HD) in Kyoto University. We added extra RuO$_2$ both as self-flux and as compensation for RuO$_2$ evaporation \cite{mao2000crystal, perry2004systematic, zhou2005electronic, kikugawa2015single}. In this study, the starting materials CaCO$_3$ (99.999\%), RuO$_2$ (99.9\%), and TiO$_2$ (99.99\%) were mixed in the molar ratio of 3 : ($3-2x$) : $2x$. The feed rod (typical dimensions of 8 cm in length and 5 mm in diameter) was prepared by hydrostatic compression followed by sintering at 1000$^{\circ}$C in air for 2 hours. Single crystals were grown at a typical speed of 7 mm/h in a gas mixture of 90\% Ar and 10\% O$_2$ at a total pressure of 10 atm. Electron probe microanalysis (EPMA) with a commercial apparatus (JEOL, JXA 8500F) at NIMS revealed a homogeneous distribution of Ti in the grown crystals, with concentrations in good agreement with the nominal ones. The inset of Fig.~\ref{fig:main-data}(a) shows a photo of a typical  crystal \rev{(Sample \#1)} with four electrical contacts. \rev{Most of the data in this Letter was taken on this sample otherwise explicitly mensioned.}  A Laue pattern of this crystal is presented in the Supplementary Material \cite{SM}.

  We designed a sample holder compatible with a commercial SQUID magnetometer (Quantum Design, MPMS XL) for simultaneous measurements of transport and magnetic properties under DC current. The details of the design can be found in the Supplemental Material of Ref.~\cite{sow2017current}. We placed a bare-chip thermometer (Lake Shore, Cernox, CX-1050-BC-HT) close to the sample to monitor its temperature (Lake Shore, 335). The transport measurements were carried out with a current source (Keithley, 6221) and a nano-volt meter (Keithley, 2182) in a four-probe configuration. We used silver-epoxy (Epoxy Technology, EPO-TEK H20E) cured at 100$^{\circ}$C to provide electrical contacts with thin copper wires.

\begin{figure}[b]
\begin{center}
\includegraphics[width=8.5cm]{phase-diagram.pdf}
\end{center}
\caption{
Temperature vs current-density phase diagram of Ca$_{3}$(Ru$_{1-x}$Ti$_x$)$_{2}$O$_7$\ ($x=0.5 \%$). \textit{T}$_{\mathrm{N1}}$\/, \textit{T}$_{\mathrm{N2}}$\, and \textit{T}$_{\mathrm{MIT}}$\ all show negligible changes with current. The antiferromagnetic insulating phase (blue) evolves into a more conducting state (yellow) with increasing current density. At temperatures below 10~K a diamagnetic phase (green) appears for $J>0.1 \,\mathrm{A/cm}^2$ with persisting AFM-$b$ ordering. \rev{The triangles represent \textit{T}$_{\mathrm{DM}}$\ taken from Fig.~\ref{fig:main-data}, while the error bars are evaluated by estimating the real sample temperature due to Joule heating  from another test experiment (see Fig.~\ref{fig:thermometer-on-top}).}
\label{fig:phase-diagram}
}
\end{figure}

Figure~\ref{fig:main-data} presents results of simultaneous measurements  of resistivity and magnetisation under various DC currents. We used an external field $\mu_{\mathrm 0}H=1$~T applied along the orthorhombic $b_{\mathrm o}$ crystallographic axis and parallel to the current direction. As shown in Fig.~\ref{fig:main-data}(a), the resistivity above $50\,\mathrm{K}$ is metallic with a value of the order of $10^{-3}\,\Omega\,\mathrm{cm}$ in agreement with the previous report \cite{tsuda2013mott}. Below \textit{T}$_{\mathrm{MIT}}$\ $\sim50\,\mathrm{K}$, the resistivity shows a sharp step-like increase, reaching a value of $10^{5}\,\Omega\,\mathrm{cm}$ at $2\,\mathrm{K}$ and $10~\mu$A. This value is 8 orders of magnitude higher than that in the high-temperature metallic phase. With increasing DC current, the resistivity in the metallic phase shows negligible variation (see also Fig.~\ref{fig:standard-data}a). However, a large change occurs in the insulating phase, where the resistivity is gradually reduced with increasing current, becoming almost 4 orders of magnitude smaller at $10\,\mathrm{K}$ and 10~mA. \rev{Correspondingly, the current-voltage characteristic in this insulating state is highly non-linear, as shown in Fig.~\ref{fig:I-V}. We note that the resistivity curves show a shift of up to 2 K around \textit{T}$_{\mathrm{MIT}}$\/, which is probably extrinsic considering possible Joule heating and slight variations in the sample cooling (further details in Fig.~\ref{fig:thermometer-on-top}).} The magnetization curves in Fig.~\ref{fig:main-data}(b) lie on top of each other in the high temperature region and show a peak at the Neel temperature \textit{T}$_{\mathrm{N1}}$\,=56~K, corresponding to the onset of the antiferromagnetic AFM-$a$ phase. The magnetization exhibits a sharp decrease at \textit{T}$_{\mathrm{N2}}$\,=50~K corresponding to the onset of the AFM-$b$ phase \cite{peng2013quasi}. We note that the AFM-$b$ order and the metal-to-insulator transition occur at the same temperature (\textit{T}$_{\mathrm{N2}}$\ = \textit{T}$_{\mathrm{MIT}}$\ = 50~K) and are accompanied by a structural transition with reduction in the \textit{c} axis \cite{yoshida2005crystal}. The increase in the magnetization with DC current below \textit{T}$_{\mathrm{N2}}$\ is attributable to a decrease in the stiffness \rev{\cite{cullity2011introduction}} of the magnetic structure.

The most interesting change is observed \rev{upon further cooling}: $M$ becomes negative below the crossover temperature \textit{T}$_{\mathrm{DM}}$\ for applied current above $300~\mu$A indicating the appearance of a novel diamagnetic state. As the current is increased, the crossover temperature increases as shown in the inset of Fig.~\ref{fig:main-data}(b).

We use the measured transition temperatures to construct the $T$--$J$ phase diagram for the various magnetic phases as a function of current density in Fig.~\ref{fig:phase-diagram}. The diamagnetic phase appears in the region above $J=0.1\,\mathrm{A/cm}^2$ and below  \rev{10~K, where we used the geometric conversion relation $I$ (mA) $=2.8\,J(\mathrm{A/cm}^2)$ as deduced from the sample cross section.} Importantly, the emerging diamagnetic state in Ca$_{3}$(Ru$_{1-x}$Ti$_x$)$_{2}$O$_7$\ coexists with the AFM order. This is evidenced by the presence of the sharp transition at \textit{T}$_{\mathrm{N2}}$\ in Fig.~\ref{fig:main-data}(b) for all the explored values of DC current. It is interesting to compare the current-induced diamagnetism of Ca$_{3}$(Ru$_{1-x}$Ti$_x$)$_{2}$O$_7$\ with what is observed in Ca$_{2}$RuO$_4$\/. In the case of Ca$_{2}$RuO$_4$\/, the AFM order is completely suppressed with current when the diamagnetism emerges \cite{sow2017current, bertinshaw2018unique}. Thus, the coexistence of diamagnetism with AFM order is a characteristic feature of Ca$_{3}$(Ru$_{1-x}$Ti$_x$)$_{2}$O$_7$\/. As another important difference from Ca$_{2}$RuO$_4$\/, the lower resistivity of Ca$_{3}$(Ru$_{1-x}$Ti$_x$)$_{2}$O$_7$\ allows us to access the full $T$--$J$ phase diagram and perform detailed investigations of current-induced phenomena. \rev{By measuring the magnetization as a function of current density at 6 K (Fig.~\ref{fig:curr-dep}), we find out that the emergence of diamagnetism is a crossover rather than a phase transition. The resistivity also shows a continuous decrease as a function of current, but its slope changes at about 0.1 $\mathrm{A/cm}^2$, a possible signature of a structural and/or electronic change which is beyond the scope of the present work. The temperature dependence of the resistivity is rather continuous at \textit{T}$_{\mathrm{DM}}$\/, indicating again the absence of a phase transition in the diamagnetic state.}

\begin{figure}
\begin{center}
\includegraphics[width=8.5cm]{switching-demo.pdf}
\end{center}
\caption{
\rev{\textit{In-situ} control of diamagnetism with current in Ca$_{3}$(Ru$_{1-x}$Ti$_x$)$_{2}$O$_7$\ ($x=0.5 \%$). (a) Cycling the current between the OFF and ON state induces (b) a change between the high- and low- resistance state and (c) between the positive and negative magnetization (diamagnetic state). Due to the different Joule heating in the two states, sample temperature varied between 2 and 4 K in each cycle (see Fig.~\ref{fig:switching-details}).}
\label{fig:switching-demo}
}
\end{figure}

We now demonstrate that reversible \rev{control} of resistance and magnetization can be achieved by changing the applied DC current. As shown in Fig.~\ref{fig:switching-demo}(a), we alternate the DC current between an OFF state (10~nA) and an ON state (300~$\mu$A) while simultaneously measuring the resistance (Fig.~\ref{fig:switching-demo}(b)) and the magnetization (Fig.~\ref{fig:switching-demo}(c)) with a constant applied magnetic field $\mu_{\mathrm 0}H=1$~T. The magnetization \rev{changes} between positive and negative as the current is cycled between the OFF and ON states. This is, to the best of our knowledge, the first demonstration of reversible \textit{in situ} \rev{control} of diamagnetism by DC current. The resistance concomitantly changes from a \rev{high- to a low-resistance state. We demonstrated that such alternation between the states is reproducible over more than $10^{5}$ repetition cycles below 10 K (Fig.~\ref{fig:switching-hystogram} (a-c)). This suggests that the occurrence of diamagnetism has mainly electronic origins, and possible slower and irreversible processes, such as chemical reaction or oxygen diffusion, play a negligible role.} By repeating similar experiments with a reversed magnetic field of $-1$~T (Fig.~\ref{fig:switching-details}(a)) or with reversed current of $-0.3$ mA (Fig.~\ref{fig:switching-details}(b)), we confirmed \rev{that the diamagnetism occurs only in the current ON state} irrespective of the current direction. This fact clarifies that the magnetic field created by the applied current via Ampere's law is not the origin of the observed negative magnetization phenomenon. \rev{We also note that the direction of current has no effect on the value of the resistance (Fig.~\ref{fig:switching-details}), as also shown in the symmetric $I-V$ characteristics (Fig.~\ref{fig:I-V})}. Concerning the time scale of the switching, we found that the resistive \rev{change} occurs faster than 0.1 s and magnetic \rev{change} faster than 20 s, which is limited by the slow repetition rate of our SQUID measurements. Further experiments with much shorter data acquisition rate are needed to reveal the dynamics of this \rev{change} on faster time scales.

\begin{figure}
\begin{center}
\includegraphics[width=8.5cm]{h-dep.pdf}
\end{center}
\caption{
Magnetization vs magnetic field (\textit{M}--\textit{H}) curves for Ca$_{3}$(Ru$_{1-x}$Ti$_x$)$_{2}$O$_7$\ ($x=0.5 \%$) with and without DC current. Both field-up and field-down data are shown. The sudden increase of magnetization under current indicates the occurrence of a metamagnetic transition. The inset shows an enlarged view near $M=0$.
\label{fig:h-dep}
}
\end{figure}
We now compare the field dependence of the magnetization with and without current. For this measurement both $H$ and $I$ are applied parallel to the $b_{\mathrm o}$ axis. As shown in Fig.~\ref{fig:h-dep}, for $I=0$ the magnetization exhibits a positive slope at all fields. The presence of a weak hysteresis and a change in the slope at $\pm$1 T are consistent with the presence of canted moments in the AFM-$b$ phase \cite{peng2013quasi}. For $I=1\,\mathrm{mA}$, $M(H)$ has a negative slope up to 2~T, indicating  negative magnetic susceptibility, characteristic of a diamagnetic state. We comment that this is essentially different from negative magnetization arising from other origins, such as current-induced flipping of magnetization reported in the ferrimagnet YVO$_3$ \cite{ren_temperature-induced_1998} or magneto-electric effects in multiferroics \cite{eerenstein2006multiferroic}. With further increase of the magnetic field, the magnetization slope changes to positive for $\mu_{\mathrm 0}H>2.5$~T, after which also $M$ becomes positive. At 6.5~T the magnetization suddenly jumps by a factor of 7, indicating the occurrence of a metamagnetic transition, where the AFM-$b$ ordered phase changes into a field-induced ferromagnetic state \cite{ohmichi2004colossal, peng2016magnetic}. As an effect of this transition, the resistance exhibits a flattening around 6.5~T (Fig.~\ref{fig:MR-data}(a)). The existence of the metamagnetic transition provides an additional piece of evidence that the AFM-$b$ order persists under current. We note that the metamagnetic transition without current was not observed in our sample up to 7~T. Thus the electric current promotes the metamagnetic transition, consistent with the reduction in the magnetic stiffness discussed above.

Because of the coexistence of diamagnetism and AFM-$b$, the magnetization signal contains both contributions. We attempt to isolate the diamagnetic contribution, which appears only under current, by evaluating $\Delta M=M(1\,\mathrm{mA})-M(0\,\mathrm{mA})$ as a function of $\mu_{\mathrm 0}H$ (Fig.~\ref{fig:MR-data}(b)). The actual diamagnetic contribution would be greater because the AFM contribution depends somewhat on $I$ due to weakening of the magnetic stiffness even at 6~K. We find $\Delta M \sim -2.3$~emu/$\mathrm{cm}^3$ up to 1~T, a value comparable to that in Ca$_{2}$RuO$_4$\ under current and larger than those in the strong diamagnets bismuth or graphite \cite{sow2017current}.

In the rest of this Letter, we discuss the possible origin of the current-induced diamagnetism in Ca$_{3}$(Ru$_{1-x}$Ti$_x$)$_{2}$O$_7$\/. In its sister compound Ca$_{2}$RuO$_4$\/, similar current-induced diamagnetism has been observed and attributed to the Landau diamagnetism of light-mass quasiparticles generated by the partial Mott gap closing \cite{sow2017current}. Ca$_{2}$RuO$_4$\ is a Mott insulator where $d_{{xy}}$ is fully occupied due to the flattening of the RuO$_6$ octahedra, and the $d_{{yz}}$ and $d_{{zx}}$ are half filled and exhibit the opening of a Mott-gap. Under DC current, it is experimentally known that the Mott gap is reduced, possibly originating from the reduction of the effective electron correlation $U$ \cite{okazaki_current-induced_2013}. When $U$ is reduced, it is proposed that the Mott gap first closes where the electronic state has a strong 2D character due to $d_{{yz}}$/$d_{{zx}}$ hybridization. This strongly $k$-dependent gap closing results in highly dispersive (light mass) quasiparticles, which leads to strong diamagnetism.

The ground state of pure Ca$_{3}$Ru$_{2}$O$_7$\ has a metallic character with a possible small gap in some parts of the Fermi surface \cite{kikugawa2010ca3ru2o7, yoshida2004quasi}. ARPES results indicate that the Fermi surface originates from quasi-1D $d_{{yz}}$ and $d_{{xz}}$ bands \cite{baumberger2006nested}. By Ti substitution, Ca$_{3}$(Ru$_{1-x}$Ti$_x$)$_{2}$O$_7$\ becomes Mott insulating with gap opening in the whole Fermi surface \cite{tsuda2013mott}. Upon application of a DC current we expect the effective correlation to be weakened and hence the Mott gap to be reduced similarly to Ca$_{2}$RuO$_4$\ \cite{okazaki_current-induced_2013}. This picture is corroborated by the observed reduction of the resistivity shown in Fig.~\ref{fig:main-data}(a). In this picture, the strong diamagnetism in Ca$_{3}$(Ru$_{1-x}$Ti$_x$)$_{2}$O$_7$\ is ascribable to thermally excited quasiparticles with very light effective mass emerging when the Mott gap is about to close.

Important novel aspects are found in the diamagnetic state of Ca$_{3}$(Ru$_{1-x}$Ti$_x$)$_{2}$O$_7$\/. Firstly, the current-induced diamagnetism occurs in a insulating state with much lower resistivity compared to Ca$_{2}$RuO$_4$\/. Secondly, diamagnetism in Ca$_{3}$(Ru$_{1-x}$Ti$_x$)$_{2}$O$_7$\ coexists with the AFM ordering. This important difference may originate from the difference in the nearest-neighbor spin alignment: it is ferromagnetic within a Ru-O bilayer in Ca$_{3}$(Ru$_{1-x}$Ti$_x$)$_{2}$O$_7$\ whereas it is G-type AFM in Ca$_{2}$RuO$_4$\ \cite{yoshida2005crystal}. These differences should be taken into account when constructing a realistic model to describe the diamagnetism in Ca$_{3}$(Ru$_{1-x}$Ti$_x$)$_{2}$O$_7$\/. In particular, possible interaction between the itinerant diamagnetism and localized AFM is an interesting issue to be explored. It is intriguing that current-induced diamagnetism is discovered in systems with different transport and magnetic characteristics, implying that it may arise in other SCES under the NESS condition.

To summarize, we demonstrated that the insulating state of Ca$_{3}$(Ru$_{1-x}$Ti$_x$)$_{2}$O$_7$\ system can be electronically controlled by flowing electric current. Most importantly, the antiferromagnetic insulating state can be reversibly \rev{changed} \textit{in situ} to a diamagnetic state with a lower resistivity. The basic origin of the diamagnetism in Ca$_{3}$(Ru$_{1-x}$Ti$_x$)$_{2}$O$_7$\ is believed to be caused by the light-mass carriers generated in the NESS. The present work reveals that DC current is a new control tool to \rev{control} the physical properties of a variety of SCES, with the possibility of inducing novel states which are not accessible by conventional control parameters.

 This work was supported by JSPS Grant-in-Aids KAKENHI Nos. JP26247060, JP15H05852, JP15K21717, and JP17H06136, as well as JSPS Core-to-Core program). N. K. and S. U. acknowledge the support from JST-Mirai Program (No. JPMJMI18A3) in Japan. We acknowledge fruitful discussions with Takashi Oka, Sota Kitamura, Kazuhiko Kuroki, Teppei Yoshida and Masatoshi Imada.

\section*{Supplementary Figures}
\begin{figure}[b]
\begin{center}
\includegraphics[width=8cm]{crystal-structure.pdf}
\end{center}
\caption{
(a) Schematic crystal structure of Ca$_{3}$Ru$_{2}$O$_7$\  \cite{peng2013quasi}. The orthorhombic crystalline axes and the magnetic structure in the antiferromagnetic AFM-\textit{b} phase are indicated. (b) Laue photograph of the Ca$_{3}$(Ru$_{1-x}$Ti$_x$)$_{2}$O$_7$\ crystal used for the transport and magnetic measurements of Fig.~\ref{fig:main-data} \rev{(Sample \#1)}.
\label{fig:crystal-structure}
}
\end{figure}
\begin{figure}[b]
\begin{center}
\includegraphics[width=8cm]{standard-data.pdf}
\end{center}
\caption{
 Temperature dependence of the resistance of Ca$_{3}$(Ru$_{1-x}$Ti$_x$)$_{2}$O$_7$\ ($x=0.5\%$)  \rev{(Sample \#1)} measured under various values of DC current at 1~T. The inset shows a zoom-in-view. These measurements were carried out with a constant positive current, the small current-dependent shift of the curves in the metallic phase is most likely due to uncompensated voltage offsets.
\label{fig:standard-data}
}
\end{figure}

\begin{figure*}
\begin{center}
\includegraphics[width=14cm]{thermometer-on-top.pdf}
\end{center}
\caption{
\rev{(a) Photograph of the resistive RuO$_x$ thick-film thermometer (KOA, RK73K1E) fixed with a low-temperature thermally-conducting glue (GE varnish) on top of a Ca$_{3}$(Ru$_{1-x}$Ti$_x$)$_{2}$O$_7$\ ($x=0.5 \%$) crystal (Sample \#2 of size $3\times 0.9 \times 0.5$ mm$^3$) equivalent to the one used in the main text. The scale in the bottom is 1 mm per division. (b) Additional two-probe resistance as a function of temperature measured at various currents on Sample \#2 which are consistent with the trend reported in Fig.~\ref{fig:main-data}(a) of the main text. Here the temperature is measured by the same thermometer used in the main text ($T_{\mathrm {sample}}$) which is 2-cm away from the sample. (c) temperature difference between readings of the additional thermometer ($T_{\mathrm {top}}$) and $T_{\mathrm {sample}}$. (d) Comparison of sample resistance plotted against $T_{\mathrm {top}}$ and $T_{\mathrm {sample}}$. (e) Total power dissipated by the sample measured at various currents. (f) Temperature difference $\Delta T=(T_{\mathrm {top}}-T_{\mathrm {sample}})$ measured as a function of current at various temperatures corresponding to \textit{T}$_{\mathrm{DM}}$\ in Fig.~\ref{fig:phase-diagram}. This $\Delta T$ gives the error bars in \textit{T}$_{\mathrm{DM}}$\ of Fig.~\ref{fig:phase-diagram}.
}
\label{fig:thermometer-on-top}
}
\end{figure*}
\begin{figure}
\begin{center}
\includegraphics[width=8cm]{I-V.pdf}
\end{center}
\caption{
\rev{$I$-$V$ characteristics measured at 20 K for Sample \#2.}
\label{fig:I-V}
}
\end{figure}

\begin{figure}
\begin{center}
\includegraphics[width=8cm]{resistance-comparison.pdf}
\end{center}
\caption{
Resistivity vs temperature  measured at 1 mA for pure Ca$_{3}$Ru$_{2}$O$_7$\ in comparison with that for a Ca$_{3}$(Ru$_{1-x}$Ti$_x$)$_{2}$O$_7$\ ($x=0.5\%$) \rev{sample \#1}.
\label{fig:resistance-comparison}
}
\end{figure}

\begin{figure}
\begin{center}
\includegraphics[width=8cm]{curr-dep.pdf}
\end{center}
\caption{
Current-density dependence of (a) resistivity and  (b) magnetization in Ca$_{3}$(Ru$_{1-x}$Ti$_x$)$_{2}$O$_7$\ ($x=0.5\%$) \rev{sample \#1}. The temperature increases from 3 K to 9 K due to Joule heating under high current.
\label{fig:curr-dep}
}
\end{figure}

\begin{figure*}
\begin{center}
\includegraphics[width=8cm]{MR-data.pdf}
\end{center}
\caption{
(a) Normalized resistance vs magnetic field at 6 K for Ca$_{3}$(Ru$_{1-x}$Ti$_x$)$_{2}$O$_7$\ with $x=0.5\%$ \rev{sample \#1}. The flattening above 6.5~T corresponds to the metamagnetic transition. (b) Change in the magnetization ($\Delta M$) due to the current flow as a function of applied magnetic field. The metamagnetic transition occurs at 6.5 T and the black line is a linear fit within $\pm1$~T yielding a diamagnetic susceptibility of $\chi= -2.5 \times$ 10$^4$ emu/Oe cm$^3$.
\label{fig:MR-data}
}
\end{figure*}
\begin{figure*}
\begin{center}
\includegraphics[width=16cm]{switching-hystogram.pdf}
\end{center}
\caption{
\rev{Current switching cycles on Ca$_{3}$(Ru$_{1-x}$Ti$_x$)$_{2}$O$_7$\ $x=0.5\%$ (sample \#2). (a) The current (gray line) was switched back and forth between the ON state (0.31 mA) and the OFF state (0.01 mA) every second for over $10^5$ sec., and (b) the voltage across the sample was recorded at each cycle (red and blue circles). (c) Histogram of the two-probe resistance in the ON and OFF states. The resistance spread in the OFF state is due to the temperature fluctuations}
\label{fig:switching-hystogram}
}
\end{figure*}
\begin{figure*}
\begin{center}
\includegraphics[width=12cm]{switching-details.pdf}
\end{center}
\caption{
\rev{Simultaneous resistance and magnetization control by DC current in Ca$_{3}$(Ru$_{1-x}$Ti$_x$)$_{2}$O$_7$\ $x=0.5\%$ sample \#1}. (a) In a negative field of $-1$~T (current changed between 10~nA and 300~$\mu$A), the magnetization is positive in the ON state, anti-parallel to the external field as it happens for positive field of 1 T. (b) For negative current between $-10$~nA and $-300$~$\mu$A (at +1~T), the magnetization in the ON state is negative, indicating that the magnetization direction does not depend on the current direction. The sample temperature shows a slight increase of 2 K in the ON state due to Joule heating, but its effect is negligible in the large observed changes of $R$ and $M$.
\label{fig:switching-details}
}
\end{figure*}
\clearpage


\section{Introduction}
\label{sec:intro}
The performance of speaker verification is significantly degraded when the speech contains background noise and/or is corrupted by interference speakers. Speaker diarization is usually applied on the non-overlapped multi-talker speech by speaker segmentation and clustering~\cite{anguera2012speaker}. It still works well by detecting and excluding the overlapped speech when the multi-talker speech is slightly overlapped \cite{charlet2013impact,yella2014overlapping}. However, such system fails when multi-talkers speak at the same time.

One possible solution is to separate the multi-talker speech into different speakers using a speech separation system, such as deep clustering \cite{hershey2016deep}, deep attractor network \cite{chen2017deep}, permutation invariant training \cite{kolbaek2017multitalker, xu2018single, xu2018shifted}, and so on. Although the performance of speech separation has been significantly improved by such approaches, the number of speakers has to be known in prior for these approaches. However, the number of speakers in the test speech is unknown in the real application of speaker verification.

To address the limitation that the number of speakers has to be know in prior, this paper proposes an overlapped multi-talker speaker verification framework by using the target speaker extraction. Given the target speaker information, the target speaker's speech is extracted from the overlapped multi-talker speech by a target speaker extraction module. Although the target speaker extraction system needs the target speaker information, it will not be a limitation. Because the target speaker information is provided as enroll speech in speaker verification system. Then, the extracted speech is passed to the speaker verification system (i.e. i-vector/PLDA speaker verification system \cite{Dehak&Kenny2011, Kenny10, Prince07, Garcia11} in this work) to verify whether the extracted speech is belonging to the target speaker.

Moreover, this paper also compares the effectiveness of two target speaker extraction networks for the overlapped multi-talker speaker verification. They are SBF-MTSAL \cite{Xu&Rao2019spkext} and SBF-MTSAL-Concat \cite{Xu&Rao2019spkext}. Experimental results demonstrate that the proposed method in this paper significantly improve the performance of speaker verification on overlapped multi-talker speech. In addition, SBF-MTSAL-Concat outperforms  SBF-MTSAL in the overlapped multi-talker speaker verification.

The remainder of the paper is organized as follows.  Section~\ref{sec:tse_sv} introduces our proposed overlapped multi-talker speaker verification framework in this paper. Section~\ref{sec:exp} and Section~\ref{sec:exp_res} report the experimental setup and results. Then, the conclusions and future works are presented in Section~\ref{sec:con}.

\vspace{-6pt}
\section{Multi-Talker Speaker Verification with Speaker Extraction}
\label{sec:tse_sv}
Since the target speaker information will be given in speaker verification, target speaker extraction is a good option to address the overlapped multi-talker speaker verification problem. Fig.~\ref{fig:overall_frm} illustrates the framework of the proposed overlapped multi-talker speaker verification system with target speaker extraction. The framework consists of a target speaker extraction module and a speaker verification system. Specifically, given a trial, the enrollment utterance $A$ and the overlapped multi-talker test utterance $Y$ are fed into the target speaker extraction network. Then, the target speaker's speech $\hat{X}$ is extracted from $Y$. The extracted speech $\hat{X}$ and enrollment speech $A$ are used as inputs of the standard i-vector/PLDA speaker verification system \cite{Dehak&Kenny2011,Kenny10,Prince07,Garcia11} to verify whether the extracted speech is belonging to the target speaker.

In this paper, we compare two target speaker extraction methods in the multi-talker speaker verification framework: (1) SpeakerBeam front-end with magnitude and temporal spectrum approximation loss (SBF-MTSAL) \cite{Xu&Rao2019spkext} and (2) SBF-MTSAL with concatenation framework (SBF-MTSAL-Concat) \cite{Xu&Rao2019spkext}. Both of these two methods are  the extended methods of SpeakerBeam front-end (SBF) \cite{delcroix2018single, delcroix2016context}.

\begin{figure}[t]
  \centering
  \centerline{\includegraphics[width=\linewidth]{TSE_FOR_SV_v2.pdf}}
  \caption{The flow chart of overlapped multi-talker speaker verification system with target speaker extraction. ``$Y$'' represents the mixture speech. ``$A$'' represents the enrollment speech. ``$\hat{X}$'' represents the extracted target speaker's speech from $Y$. The target speaker extraction network takes the enrollment utterance $A$ of the target speaker as the auxiliary information to extract the speech component $\hat{X}$ from $Y$ that belongs to the target speaker.}
  \label{fig:overall_frm}
  \vspace{-8pt}
\end{figure}

\begin{figure}[t]
  \centering
  \centerline{\includegraphics[width=7.0cm]{speakerbeam.pdf}}
  \caption{The architecture of SBF-MTSAL. ``AL'' in the trapezium box represents the adaptation layer. ``Sub'' represents the sub-layer. ``$\alpha$'' represents the weight obtained from the auxiliary network. ``$M$'' represents the number of sub-layers. ``$|Y|:$ Mixture'' represents the magnitude of the mixture speech. ``$|\hat{X}|:$ Extracted'' represents the output magnitude of the extracted target speaker's speech. ``$|X|:$ Reference'' represents the magnitude of clean speech, which is used to simulate the mixture. ``$|A|:$ Auxiliary'' represents the magnitude of the auxiliary speech. During the evaluation, the upper right dotted box is not necessary.}
  \label{fig:speakerbeam_frm}
\vspace{-8pt}
\end{figure}

\begin{figure}[t]
  \centering
  \centerline{\includegraphics[width=6.5cm]{josen.pdf}}
    \caption{The architecture of SBF-MTSAL-Concat. The meaning of ``$|Y|, |\hat{X}|, |X|, |A|$'' can be referred to the caption of Fig.~\ref{fig:speakerbeam_frm}. During the evaluation, the upper right dotted box is not necessary.}
    \label{fig:josen_frm}
\vspace{-8pt}
\end{figure}

\vspace{-10pt}
\subsection{SBF-MTSAL}
Fig.~\ref{fig:speakerbeam_frm} shows the architecture of SBF-MTSAL \cite{Xu&Rao2019spkext}. The SBF-MTSAL approach uses an auxiliary network to learn adaptation weights from the target speaker's voice, which is different from the utterance of the target speaker in the mixture. The adaptation weights contain speaker characteristics and are used to weight the sub-layers in the adaptation layer of the mask estimation network with a CADNN structure. Instead of computing objective loss between ideal binary mask and estimated mask in the original work \cite{delcroix2018single}, SBF-MTSAL computes a magnitude and temporal spectrum approximation loss to estimate a phase sensitive mask \cite{erdogan2015phase} due to its better performance. The magnitude and its dynamic information (i.e., delta and acceleration) are used in calculating the objective loss for temporal continuity.

\vspace{-10pt}
\subsection{SBF-MTSAL-Concat}
Fig.~\ref{fig:josen_frm} illustrates the architecture of SBF-MTSAL-Concat method \cite{Xu&Rao2019spkext}. The auxiliary network learns speaker embedding from a different utterance of target speaker, which contains speaker characteristics. Then, the speaker embedding is repeated concatenated with the activation of a BLSTM in the mask estimation network. The concatenated representations containing the mixture and target speaker information are used to estimate a phase sensitive mask with the same loss function as in SBF-MTSAL method.

\vspace{-6pt}
\section{Experimental Setup}
\label{sec:exp}
\vspace{-6pt}

\subsection{Speech Data}\label{sec:exp_data}
\vspace{-6pt}
The two-speaker mixed dataset\footnote{The database simulation code is available at: \url{https://github.com/xuchenglin28/speaker_extraction}} used to train the target speaker extraction network was simulated at a sampling rate of 8kHz based on the WSJ0 corpus \cite{garofolo1993csr}. In the simulation of two-speaker mixture, the first selected speaker was chosen as target speaker, the other one was interference speaker. The utterance of the target speaker from the original WSJ0 corpus was used as reference speech. Another utterance of this target speaker, which was different from the reference speech, was randomly selected to be used as input to the auxiliary network to obtain target speaker information.

The simulated dataset was divided into training set ($20,000$ utterances), development set ($5,000$ utterances), and test set ($3,000$ utterances). Specifically, the utterances from $50$ male and $51$ female speakers in the WSJ0 ``si\_tr\_s'' set were randomly selected to generate the training and development set. The SNR of each mixture was randomly selected between 0dB and 5dB. Similarly, the test set was created by randomly mixing the utterances from $10$ male and $8$ female speakers in the WSJ0 ``si\_dt\_05'' and ``si\_et\_05'' sets. Since the speakers in the test set were different from the training and development sets, the test set was used to evaluate the speaker verification performance.
\vspace{-6pt}

\subsection{Target Speaker Extraction Network Setup}\label{sec:exp_tse}
\vspace{-6pt}

A short-time Fourier transform (STFT) was used with a window length of $32$ms and a shift of $16$ms to obtain the magnitude features from both of the input mixture for mask estimation network and input target speech for auxiliary network. The normalized square root hamming window was applied.

The learning rate started from $0.0005$ and scaled down by $0.7$ when the training loss increased on the development set. The minibatch size was set to $16$. The network was trained with minimum $30$ epochs and stopped when the relative loss reduction was lower than $0.01$. The Adam algorithm \cite{kingma2014adam} was used to optimize the network.

The aforementioned magnitude extraction configuration and network training scheme were kept same in both SBF-MTSAL and SBF-MTSAL-Concat methods. The extracted magnitude were reconstructed into time-domain signal with phase of the mixture. Then the time-domain signal was used as input to the speaker verification system.

For SBF-MTSAL, the auxiliary network was composed of $2$ feed-forward relu hidden layers with $512$ hidden nodes and a linear layer with $30$ hidden nodes. The adaptation weights were obtained by averaging these $30$ dimensional outputs over all the frames. The mask estimation network used a BLSTM with $512$ cells in each forward and backward direction. The following adaptation layer had $30$ sub-layers. Each sub-layer had $512$ nodes with $1024$ dimensional inputs from the outputs of the previous BLSTM. The $30$ dimensional weights from the auxiliary network were used to weight these sub-layers, respectively. Then the activation of the adaptation layer was summed over all the sub-layers. Another $2$ feed-forward relu hidden layers with $512$ nodes were appended. The mask layer had $129$ nodes to predict the mask for the target speaker.

\begin{table*}[t]
    \centering
    \caption{Performance of SV system without and with target speaker extraction. ``Training'' represents the type of training data for SV system. ``Eval.'' represents the type of evaluation test data for SV system. ``TSE'' represents whether perform the target speaker extraction for SV. If the option of this column is No, it means that no speaker extraction is applied in the SV. If the option is SBF-MTSAL/SBF-MTSAL-Concat, it means that SBF-MTSAL/SBF-MTSAL-Concat speaker extraction is used for SV. ``Clean'' represents the speech from one speaker. ``Mixture'' means the overlapped multi-talker's speech. ``Ext'' represents the extracted target speaker's speech from the mixture by target speaker extraction network. ``Clean+Ext'' means pooling the clean speeches and extracted target speaker's speeches to train the SV system. ``Baseline'' represents the baseline performance of overlapped multi-talker speaker verification system. ``Upper Bound'' represents the upper bound performance of target speaker extraction for overlapped multi-talker speaker verification. ``DCF08'' represents the minimum detection cost with $P_{Target}=0.01$. ``DCF10'' represents the minimum detection cost with $P_{Target}=0.001$. The details of experimental setup can be referred to section~\ref{sec:exp_sv}.}
    \vspace{-6pt}
    \begin{tabular}{|l|l|l|l|c|c|c|} \hline
      System No. & Training & Eval. & TSE & EER $(\%)$ & DCF08 & DCF10 \\ \hline\hline
      1 (Baseline) & Clean & Mixture & No & 22.67 & 0.867 & 0.915 \\
      2 & Clean+Ext & Mixture & No & 21.67 & 0.850 & 0.898 \\ \hline\hline
      3 & Clean & Mixture & SBF-MTSAL & 11.17 & 0.760 & 0.844 \\
      4 & Clean & Mixture & SBF-MTSAL-Concat & 10.40 & 0.736 & 0.813 \\
      5 & Clean+Ext & Mixture & SBF-MTSAL-Concat & \textbf{7.77} & \textbf{0.631} & \textbf{0.747} \\ \hline\hline
      6 (Upper Bound) & Clean & Clean & No & 3.33 & 0.357 & 0.454 \\
      7 & Clean+Ext & Clean & No & 3.07 & 0.377 & 0.524 \\ \hline
    \end{tabular}
    \label{tab:SV_I}
    \vspace{-12pt}
\end{table*}

For SBF-MTSAL-Concat, the auxiliary had a BLSTM with $256$ cells in each forward and backward direction, a feed-forward relu hidden layer with $256$ nodes and a linear layer with $30$ nodes. The output of the linear layer was averaged over all frames to obtain a $30$ dimensional speaker embedding containing target speaker characteristics. The speaker embedding was repeatedly concatenated with the activation of the BLSTM layer in the mask estimation network. The BLSTM had $512$ cells in each forward and backward direction. Then the concatenated outputs were fed to a feed-forward relu hidden layer, a BLSTM layer and another feed-foreard relu hidden layer. The BLSTM had $512$ cells and the relu layers had $512$ nodes. The mask layer had $129$ nodes.
\vspace{-6pt}

\subsection{Speaker Verification (SV) System}\label{sec:exp_sv}
\vspace{-6pt}

According to the test set of simulated dataset, we generated 3000 target trials and 48,000 non-target trials for the SV evaluation. In the evaluation trials, each enrollment utterance contained a contiguous speech segment from a single speaker and test utterance contained the overlapped speech from multiple speakers. We called this evaluation set as \textit{mixture evaluation set}. Moreover, to show the upper bound of target speaker extraction on SV, we also generated another evaluation set with 51,000 trials from WSJ0 corpus according to the information of mixture set. This set was called as \textit{clean evaluation set}\footnote{The SV evaluation trials and keys for clean and mixture evaluation sets are available at: \url{https://github.com/xuchenglin28/speaker_extraction}}.

We selected 8,769 utterances from 101 speakers in WSJ0 corpus which were used to generate the training set of simulated database for training UBM, total variability matrix, LDA, and PLDA models. This set was named as \textit{clean training set}. Because this paper directly used the extracted target speaker's speech for SV, it would cause the mismatch between extracted speech and clean speech. To solve this mismatched problem, we pooled 5,000 extracted speech from the development set of the simulated 2-speaker mixed dataset and clean training set to train speaker verification system. We called this training set as \textit{clean+ext set}. Section~\ref{sec:exp_res} will show the performance by using different training and evaluation set.

The features of SV system were based on 19 MFCCs together with energy plus their 1st- and 2nd-derivatives extracted from the speech regions, followed by cepstral mean normalization \cite{Atal74} with a window size of 3 seconds. A 60-dimensional acoustic vector is extracted every 10ms, using a Hamming windowing of 25ms. An energy based voice activity detection method is used to remove silence frames. The system was based on a gender-independent UBM with 512 mixtures. The training set described in the previous paragraph was used for estimating the UBM and total variability matrix with 400 total factors. The same data set was used for estimating the LDA and Gaussian PLDA models with 150 latent variables.

\vspace{-6pt}
\section{Experimental Results}
\label{sec:exp_res}
\vspace{-6pt}

To investigate the effect of overlapped test speech on speaker verification, we perform the speaker verification experiments on both mixture and clean evaluation set described in section~\ref{sec:exp_sv}. System 1 of table~\ref{tab:SV_I} is the baseline system of SV with clean training data on mixture test set. System 6 of table \ref{tab:SV_I} shows the upper bound performance (also called as ideal performance) of target speaker extraction for overlapped multi-talker SV. Performance comparison between system 1 and system 6 of table~\ref{tab:SV_I} shows that the performance speaker verification system seriously degrades when the test speech is the fully overlapped multi-talker speech.

Table~\ref{tab:SV_I} also presents the performance of speaker verification system on the evaluation set without and with target speaker extraction. System 1 of table~\ref{tab:SV_I} is the baseline results of overlapped multi-talker speaker verification. System 3 to 5 of table~\ref{tab:SV_I} show the performance of speaker verification after using target speaker extraction. Results of system 1 to 5 demonstrate that applying target speaker extraction significantly improve the performance of overlapped multi-talker speaker verification. Specifically, System 5 of Table~\ref{tab:SV_I} can obtain around $65.7\%, 27.2\%, 18.4\%$ relative reduction over the baseline (system 1) on EER, DCF08, and DCF10, respectively.

This paper compares two target speaker extraction methods for overlapped multi-talker speaker verification: (1) SBF-MTSAL and (2) SBF-MTSAL-Concat. Performance comparison between system 3 and 4 of table~\ref{tab:SV_I} shows that SBF-MTSAL-Concat outperforms SBF-MTSAL on both EER and DCFs.

To alleviate the effect of the mismatch between extracted speech and clean speech for SV, we combine the clean training set and extracted speech data (Clean+Ext) to train speaker verification system. Both clean and mixture test sets are used to evaluation this speaker verification system. Additionally, because SBF-MTSAL-Concat achieves the better performance, we only apply this experiment on SBF-MTSAL-Concat. System 2, 5, and 7 in table~\ref{tab:SV_I} show the performance of SV system with clean+ext training data. System 2 of table~\ref{tab:SV_I} proves that most of improvement on the overlapped multi-talker speaker verification is done by target speaker extraction methods by comparing with system 1 and 3 to 5. Comparison between system 6 and 7 in table~\ref{tab:SV_I} shows that Clean+Ext training set will improve the performance of speaker verification on the clean test set in EER, but degrade the DCFs. And performance comparison between system 4 and 5 in table~\ref{tab:SV_I} demonstrates that Clean+Ext training set could advance the speaker verification performance with SBF-MTSAL-Concat on the mixture test set.

\vspace{-12pt}
\section{Conclusions and Future Works}
\label{sec:con}
\vspace{-6pt}

This paper applies the target speaker extraction to improve the performance of overlapped multi-talker speaker verification. Experimental results show that the proposed method could significantly improve the performance of overlapped multi-talker speaker verification. This paper also compares the performance of SBF-MTSAL and SBF-MTSAL-Concat on overlapped multi-talker speaker verification and finds that SBF-MTSAL-Concat achieves better performance than SBF-MTSAL.

This paper mainly focuses on the fully overlapped test speech from multiple speakers.
In the future, we will investigate the effectiveness of proposed method when the enrollment is also a multi-talker speech, apply the proposed method on the public available speaker verification database, and explore the joint training of target speaker extraction and speaker verification.

\vspace{-6pt}

\bibliographystyle{IEEEbib}