\section{Introduction}
Quantum entanglement is widely recognised as a resource ``as real as energy''~\cite{RevModPhys.81.865}.
Yet, limits on establishing entanglement between remote particles were systematically studied only recently and with surprising results.
Protocols in which the distant particles get entangled by exchanging ancillary particles can establish remote entanglement without ever communicating it, i.e., no entanglement with the ancillaries~\cite{cubitt,exp0,exp1,exp2,exp3}.
It is now understood that entanglement gain in these schemes is not bounded by the communicated entanglement, but rather by communicated quantum discord~\cite{bounds1,bounds2,Streltsov14,Streltsov15,Zuppardo16,Streltsov-review},
a form of quantum correlation that persists in many disentangled states~\cite{henderson2001classical,discord,discord-review}.

In another route to producing remote entanglement, the exchange of quantum particles is replaced by continuous interactions of distant systems (probes) with a third object, a mediator.
In this scenario the theory predicts that not only the probes can get entangled without ever entangling the mediator~\cite{cubitt},
but also that they can even get entangled in the absence of any quantum discord between the probes and the mediator~\cite{krisnanda2017}.
This lack of discord is a strong notion of classicality which means that the mediator can be measured at any time without disturbing the whole multipartite system.
It is the same notion as ``classical communication'' in the framework of local operations and classical communication at the core of entanglement theory~\cite{locc}, but generalised to continuous in time interactions.
In practical terms, the probes get entangled even if the mediator is continuously monitored or dephased.

It is an observation of this effect, for a discrete number of measurements on the mediator, that is reported here together with theoretical characterisation of maximal amount of entanglement that can be established in this way. Moreover, in our experiments the monitoring measurement is the same at all times, which reinforces classicality of the mediator being at all times in one of two distinguishable states (correlated to the probes). Additionally to observing exotic effect of multipartite entanglement our results have practical implications.
The scenario where two objects are coupled via a mediator is common in science. For example, entanglement gain between the probes has been proposed as a witness of quantum mediator in various scenarios~\cite{TanKris},
such as spin chains in solid state magnetic compounds~\cite{Kon}, mechanical membranes inside optical cavities~\cite{krisnanda2017}, detection of initial quantum correlations in open-system dynamics~\cite{krisnanda2017},
quantum gravity \cite{krisnanda2017,Bose2017,MV2017} or quantum properties of parts of living organisms coupled to a multi-mode cavity field~\cite{krisnanda_bio}.
Present results emphasise that these methods must verify features of the initial state in order to validate their implications, i.e., non-classicality of the mediating system.

\section{Results}
\subsection{Theoretical example}

The simplest example of the discussed phenomenon involves three qubits (spin-$\frac{1}{2}$ systems) in the following initial state~\cite{krisnanda2017}:
\begin{equation}
\rho_0 = \frac{1}{2} | \psi_+ \rangle \langle \psi_+ | \otimes | + \rangle \langle + | \,  +  \frac{1}{2} | \phi_+ \rangle \langle \phi_+ | \otimes | - \rangle \langle - |,
\label{rho_in}
\end{equation}
where the first two qubits are the probes $A$ and $B$, and the third qubit is the mediator $M$.
Kets $| \pm \rangle$ denote the eigenstates of $\sigma_x^M$ Pauli matrix,
whereas $| \psi^+ \rangle = \frac{1}{\sqrt{2}}(| 01 \rangle + | 10 \rangle)$ and
$| \phi^+ \rangle = \frac{1}{\sqrt{2}}(| 00 \rangle + | 11 \rangle)$ are the two Bell states.
Since one could dephase the mediator in the $\sigma_x^M$ basis without perturbing the total state, the mediator is said to be in a classical form.
Note also that initially the probes are not entangled as their state is an even mixture of two Bell states~\cite{VP1998}, but the whole tripartite state is initially entangled across partitions $A:MB$ and $AM:B$. This system evolves under Hamiltonian ($\hbar = 1$ throughout the paper):
\begin{equation}
\mathcal{H} = \omega(\sigma_x^A + \sigma_x^B) \otimes \sigma_x^M,
\label{EQ_H_EX}
\end{equation}
where each probe individually interacts with the mediator via a coupling constant $\omega $, but not directly with each other.
It is easy to see that the state of the mediator is stationary and hence it remains classical at all times.
Furthermore, at all times, it is one and the same measurement i.e., dephasing along $\sigma_x^M$ basis, that keeps the total state invariant.
Yet entanglement between the probes increases and they become even maximally entangled~\cite{krisnanda2017}.

At first sight this example might be surprising as it seems that entanglement between the probes is increased by local interactions with a classical mediator, in contradiction to the very definition of entanglement~\cite{RevModPhys.81.865,RMP.91.025001}.
We stress that there is no contradiction as already in the initial state each individual probe is entangled with the rest of the system. One can show that the corresponding entanglement, as quantified by negativity \cite{neg}, is given by $E_{A:MB} = E_{AM:B} = 1/2$, which is the amount of entanglement in maximally entangled state of two qubits.
The subsequent evolution localises this entanglement to the probes. One could also think about this process as continuous in time entanglement distillation, where the local interactions ``read'' the state of the mediator and suitably adapt the dynamics to arrive at entanglement between the probes.
It is the essence of our demonstration that entanglement localisation can be done via the classical mediator even if it is measured or dephased.

\subsection{Optimality}

We show in the Methods section that
a resource behind entanglement localisation via a classical mediator is the amount of initial correlations with the mediator.
The amount of entanglement that can be localised is bounded as follows:
\begin{equation}
\mathcal{E}_{A:B}(t) - \mathcal{E}_{A:B}(0) \le I_{AB:M}(0),
\label{EQ_I_BOUND}
\end{equation}
where $\mathcal{E}_{A:B}$ denotes the relative entropy of entanglement~\cite{REE} and $I_{AB:M}$ is the mutual information~\cite{NC-book}. Let us put this result in perspective and repeat, this time with quantitative statements, what has been known about the resources behind distribution of various correlations.
Consider first two laboratories (operated by Alice and Bob) that exchange a quantum particle $M$.
Initially, Alice has systems $A$ and $M$, then $M$ is communicated to Bob, who finally stores systems $M$ and $B$ in his laboratory.
How much can information between the laboratories increase in this process?
As expected, the bound is given by the communicated information and is characterised by the inequality
$I_{A:MB} - I_{AM:B} \le I_{AB:M}$~\cite{bounds2}.
Surprisingly, a similar inequality does not extend to quantum entanglement.
Instead, we have $\mathcal{E}_{A:MB} - \mathcal{E}_{AM:B} \le D_{AB|M}$, i.e., entanglement gain between the laboratories is bounded by the communicated quantum discord~\cite{bounds1,bounds2,Streltsov14,Streltsov15,Zuppardo16,Streltsov-review}. This allows for entanglement distribution via exchange of unentangled particles~\cite{cubitt} since discord can be nonzero in separable states.
Now, if the communication is continuous in time (not exchange of a particle) and we study entanglement between the probes only (not entanglement between the laboratories) Eq.~\ref{EQ_I_BOUND} shows that the initial information with  mediator is the relevant bound.
This empowers entanglement localisation via classical mediator because there exist correlated states without discord.

Summing up, the necessary conditions for an initial state to give rise to entanglement localisation via classical mediator include: (i) correlations to mediator, i.e. $I_{AB:M} > 0$, and (ii) entanglement between a probe and the rest of the system, e.g., $E_{A:MB} > 0$.
The latter is formally recognised in Eq.~(\ref{EQ_REVEALING}) of the Methods section, which proves $E_{A:B}(t) \le E_{A:MB}(0)$.
This holds for any convex entanglement monotone.
It is straightforward to verify that

the theoretical example presented above achieves both the upper bound in Eq.~\ref{EQ_I_BOUND} and the bound given by the initial entanglement with a probe particle.

\subsection{NMR setup}

We use liquid-state NMR spectroscopy of $^{13}$C, $^1$H and $^{19}$F in dibromofluoromethane dissolved in acetone with linear topology, H - C - F (see Fig.~\ref{FIG_INITIAL}(a)).
Nuclei of hydrogen and fluorine are identified as probes $A$ and $B$, respectively, whereas carbon nucleus is naturally the mediator $M$. Experiments were performed in $500$ MHz Bruker NMR spectrometer at room temperature.
The sample consists of identical and fairly isolated dibromofluoromethane molecules and all the dynamics of the three-qubit system is completed before any significant environmental influences~\cite{NMR1,NMR2,NMR3}.
The longitudinal and transverse relaxation time constants are longer than $2$ s and $0.2$ s, respectively.
The internal Hamiltonian of the three spin system in a frame rotating about the Zeeman field with individual Larmor frequencies reads:
\begin{equation}
\begin{split}
H_0 = \frac{\pi}{2} \Big ( J_{AM} \, \sigma_z^A \otimes \sigma_z^M +  J_{BM} \, \sigma_z^B \otimes \sigma_z^M +\\
 J_{AB} \, \sigma_z^A \otimes \sigma_z^B \Big ),
\end{split}
\label{Mol_ham}
\end{equation}
with $J_{AM}=224.5$ Hz, $J_{BM}=-310.9$ Hz, and $J_{AB}=49.7 ~\mathrm{Hz}$ being the corresponding coupling constants between the nuclei. The qubits in the molecular system have internal dynamics that directly couples the probes $A$ and $B$. The effects of this coupling must be canceled if we are to claim generation of entanglement between the probes via classical mediator only. Thus, during experiments, to evolve the system under the interaction Hamiltonian $\mathcal{H}$ in Eq.~\ref{EQ_H_EX} we switch off the internal interaction between spins $A$ and $B$ by applying suitable refocusing pulses as will be explained later.

In general, the quantum state of our three-qubit NMR system is of the form  \noindent $(1 - \epsilon)\frac{1}{8} \mathbb{1} + \epsilon \, \rho$,
where $\frac{1}{8} \mathbb{1}$ describes the background population, $\rho$ is the so-called deviation density matrix and $\epsilon$ is the purity factor, which is in the order of $10^{-5}$. Nevertheless the NMR experiments are sensitive to the deviation density matrix and  from now on whenever we refer to the ``state'' of the system we mean the pseudopure state characterized by the deviation density matrix. Despite the low purity of the NMR qubits, it has been shown that such  systems can still efficiently implement certain quantum information tasks, for which no known efficient classical algorithms exist \cite{morimae2017power}

Starting from a three qubit thermal equilibrium state of longitudinal magnetization at room temperature, we prepare the state corresponding to $| 00 \rangle \langle 00| \otimes \mathbb{1}/2$, written in the order $ABM$, using a similar pulse sequence as given in \cite{SciDir,Mahesh2013}.
The initial state $\rho_0$, Eq.~\ref{rho_in}, is then obtained by the succession of gates given in Fig.~\ref{FIG_INITIAL}(b).
All gates are implemented using radio frequency pulses resonant with the nuclei. The open CNOT gate is realised with the help of Krotov optimisation technique~\cite{krotov2008quantum} with fidelity exceeding 0.99 using push-pull optimization of quantum controls \cite{batra2020push}. The fidelity of the produced initial state to the ideal one is more than $97\%$.

Fig.~\ref{FIG_INITIAL}(c) shows the pulses used to realize the interaction Hamiltonian, $\mathcal{H}$ in Eq.~\ref{EQ_H_EX}. In our experiments we have set the strength of the coupling constant $\omega = 1$ rad/s. The solid bars and empty bars represent $\pi/2$ and $\pi$ pulses, respectively. The first half of the pulse sequence evolves the system under the coupling between $M$ and $A$.
Since we have $\sigma_z \otimes \sigma_z$ coupling in our system to start with (see Eq.~\ref{Mol_ham}),
the $(\pi/2)_y$ pulses transform the $ z $-basis to $ x $-basis which then evolves under $\sigma_z \otimes \sigma_z$ coupling.
Since the Hamiltonian in Eq.~\ref{EQ_H_EX} is a sum of two commuting terms, we first evolve the entire system under $\sigma_x^A \otimes \sigma_x^M$
followed by the evolution under $\sigma_x^B \otimes \sigma_x^M$ for the same amount of time,
i.e, the physical time rescaled by the coupling strengths $J_{AM}$ and $J_{BM}$.
As all the three qubits are coupled to each other, we decouple $B$ during the first half of the evolution by refocusing it using a $\pi$ pulse, as shown in Fig.~\ref{FIG_INITIAL}(c). The net effect is that the system only evolves under $AM$ coupling, whereas $B$ remains unaltered. The same is repeated in the second half of the evolution with $A$ being refocused and the system evolving under $BM$ coupling.
We repeat the experiment with the same initial conditions and different duration of dynamics in order to illustrate how entanglement accumulates between the probes.
The probes in principle gain maximal entanglement at $\omega t = \pi/8$.
Finally, we obtain the deviation density matrices via full state tomography using eleven detection experiments~\cite{NMR-tomo}.

To make the claim that the mediator $M$ is classical even stronger, we introduce another set of experiments in which we dephase (measure) the mediator qubit in between and at the end of the evolution.
The pulse sequence implementing the dephasing of $M$
is depicted in the orange box of Fig.~\ref{FIG_INITIAL}(c).
In contrast to the previous set of experiments, just after the realization of $\sigma_x^A \otimes \sigma_x^M$ the mediator qubit $M$ is dephased in $x$-basis.
The selective dephasing of the mediator is achieved by a pair of opposite pulsed-field-gradients (PFGs)  separated by a $\pi$-pulse on the mediator.  The PFGs cancel each other for the probes $A$ and $B$, whereas they add-up for $M$.  A $\pi$-pulse on $M$ is applied to undo the spin-flip caused by the previous $\pi$-pulse.  Finally, measurement along $x$-basis is realized by simply rotating the basis using $(\pi/2)_y$ and $(\pi/2)_{-y}$ pulses as shown.

\begin{figure}[!t]
	\centering
	\includegraphics[trim=4.5cm 0cm 6cm 0cm,clip=true,width=8.5cm]{circuit_pulse5.pdf}
	\caption{(a) Molecular structure of dibromofluoromethane. We identify $^1$H and $^{19}$F nuclei as probe qubits and $^{13}$C as the mediator qubit. (b) Preparing the initial state as in Eq.~\ref{rho_in} using CNOT and Hadamard gates as shown.
		(c) Pulse sequences used to evolve the system under the coupling Hamiltonian of Eq.~\ref{EQ_H_EX}. The solid and empty bars represent $\pi/2$ and $\pi$ pulses with phases shown above them.  The blue pulses cancel each other for the no-dephasing case.  Dephasing of $M$ is realized by introducing pulses shown in the orange box in the positions marked by the dashed lines.  Here PFG represents the pulsed-field gradient along $\pm z$ axis.  The delays $\tau_{AM} = 1/(4J_{AM})$ and $\tau_{BM} = 1/(4J_{BM})$.}
	\label{FIG_INITIAL}
\end{figure}

\section{Discussion}

\begin{figure}
	\centering

	\includegraphics[trim=0.5cm 6.7cm 1.7cm 7.5cm,clip=true,width=9cm]{final_plot_ts1.pdf}
	\caption{Theoretical (lines) and experimental (markers) correlations: (a) without dephasing and (b) with dephasing the mediator qubit.
		Solid lines show noiseless theoretical predictions: blue for $E_{A:B}$ and purple for $E_{AM:B}$.
		The corresponding experimental data is marked with yellow and purple markers, respectively.
		Green markers show measured discord $D_{AB|M}$, all within experimentally established region of vanishing discord (grey, see main text). The error bars represent the random error in the experiments, obtained from the signal to noise ratio for each experiment. The dotted red lines represent $E_{A:B}$ within a model including inhomogeneity of radio frequency pulses with the distributions $\mathrm{P}(\omega_{\rm RF})$ as shown in the insets.}
	\label{FIG_RESULTS}
\end{figure}

From the experimentally measured three-qubit deviation density matrices we compute various quantum correlations such as discord between the two probes and mediator, $D_{AB|M}$, quantum entanglement between the probes, as measured by the negativity $E_{A:B}$, as well as the negativity $E_{B:AM}$. The quantum discord is calculated following the definition of Ollivier and Zurek~\cite{discord}.
Recall that discord is not a symmetric quantity and $D_{AB|M}$ denotes discord as measured on the mediator.
It should also be stressed that due to small admixture of the deviation density matrix, the ensemble averaged NMR signals mask genuine entanglement~\cite{Braunstein1999}.
From this perspective one can think of our experiment as NMR simulation of entanglement localisation via classical mediator.

The measured discord and entanglement  are presented in Fig.~\ref{FIG_RESULTS} for datasets without and with dephasing the mediator.
The gray-shaded region represents measurability threshold of discord owing to experimental errors.
This threshold is obtained from measurements of discord for experimental thermal equilibrium state.
Ideally this state has vanishing discord but experimental imperfections in state tomography give rise to residual values.
The amount of discord $D_{AB|M}$ calculated for evolved deviation density matrices (green data points) all lie well within this experimental precision limit of discord.
We thus conclude that the mediator was classical at all times during the evolution. Yet, negativity of quantum entanglement between the probes $E_{A:B}$ consistently grows
as shown by experimentally estimated values depicted with yellow markers.
The error-bars represent random errors and do not exactly match the noiseless prediction based on Hamiltonian in Eq.~\ref{EQ_H_EX}, presented with a solid blue line.
This is the effect of systematic errors which include the spatial RF inhomogeneity (RFI) over the sample volume, fluctuations in the resonance frequencies due to variations in ambient temperature, as well as finite precision involved in quantum state tomography.
We have simulated the effect of RFI in the evolution as well as tomography pulses, see the dotted red curves,
and indeed observe better agreement with the experimental data.
It is interesting to note that the entanglement between each probe with the rest of the system, $E_{B:AM}$ or $E_{A:BM}$, remains invariant throughout the evolution.
We demonstrated that an increment of quantum entanglement between two probes coupled via a mediator in general does not signify a non-classical mediator.

We note that the process demonstrated here is different from, e.g., entangling two spins via dipole-dipole interaction.
The usual Hamiltonian of the latter directly couples magnetic moments of the two particles and hence it is not surprising that entanglement grows.
In contradistinction, we study tripartite system with an explicit mediator.
Even if the dipole-dipole interaction is rewritten in the form where the mediating virtual photons are clearly distinguished, they typically get entangled with the particles and hence the mediator is not classical.

Perhaps the most interesting application for revealing non-classicality of mediators is witnessing quantum gravity through entanglement between nearby masses~\cite{krisnanda2017,Bose2017,MV2017}. Assuming that the whole mass-gravity-mass system can be described by the standard tripartite Hilbert space formalism, entanglement $E_{A:MB}$ or $E_{AM:B}$ cannot grow via classical mediator, i.e., when the discord $D_{AB|M} = 0$ at all times~\cite{krisnanda2017}. However, the masses alone can become entangled, i.e., $E_{A:B}$ may grow even via classical mediator --- analogously to the phenomenon demonstrated in the present work.
As shown here, this requires initial entanglement $E_{A:MB}$ or $E_{AM:B}$ and correlations to the mediator.
Since in practice one would only measure the masses and not the field, it is highly desirable to provide the bound on possible entanglement gain via classical mediator solely in terms of quantities measured on the masses.
Ref.~\cite{krisnanda2017} proves that the relevant bound is given by the sum of initial entropies of the masses.
Estimations with concrete experimental arrangements show that in order to observe gravitational entanglement the masses need to be cooled down near the ground state of their traps~\cite{Bose2017, Mann2018,arXiv:1812.09776, arXiv1906.08808}.
In such a case the masses are close to a pure state and hence they are initially almost uncorrelated from the rest of the world, i.e., their entropies are small, but not zero. It is this latter bound on entanglement that has to be experimentally violated in order to witness quantum gravity within this framework.

\section{Methods}

We prove here Eq.~\ref{EQ_I_BOUND} of the main text.
Let us begin with a useful lemma.

\begin{lemma}
	For a tripartite system with classically correlated mediator, i.e, in a state $\rho = \sum_m p_m \: \: \rho_{AB|m} \otimes | m \rangle \langle m |$ with orthonormal basis $\{ | m \rangle \}$, the relative entropy of entanglement follows the bound
	\begin{equation}
	\mathcal{E}_{A:BM} - \mathcal{E}_{A:B} \le I_{AB:M},
	\end{equation}
	where $I_{AB:M}$ is the mutual information between the mediator and remaining systems.
\end{lemma}

\begin{proof}
	From the definition of relative entropy of entanglement, we have $\mathcal{E}_{A:BM} = - \mathrm{tr}( \rho \log \sigma) - S_{ABM}$, where $\sigma$ is the closest separable state to $\rho$ and $S_{ABM}$ stands for von Neumann entropy of state $\rho$~\cite{REE}.
	The flags condition~\cite{flags} applied to $\rho$ gives $\mathcal{E}_{A:BM} = \sum_m p_m \: \mathcal{E}_{A:B}(\rho_{AB|m})$.
	Furthermore, the same reference shows that the corresponding closest separable states satisfy
	$\sigma = \sum_m p_m \sigma_{AB|m} \otimes | m \rangle \langle m |$, where $\sigma_{AB|m}$ is the closest separable state to $\rho_{AB|m}$.
	Using expressions for $\sigma$ and $\rho$, we find
	\begin{eqnarray}
		\mathcal{E}_{A:BM} & = & S_M - S_{ABM} - \nonumber \\
		&&\sum_m p_m \mathrm{tr}(\rho_{AB|m} \log \sigma_{AB|m}) \\
		\mathcal{E}_{A:B} & = & - \sum_m p_m \mathrm{tr}(\rho_{AB|m} \log \sigma_{AB}) - S_{AB},
	\end{eqnarray}
	where $\sigma_{AB}$ is the closest separable state to the marginal $\rho_{AB} = \sum_m p_m \rho_{AB|m}$.
	Using the definition of mutual information, $I_{AB:M} = S_M + S_{AB} - S_{ABM}$, we have
	\begin{eqnarray}
		\mathcal{E}_{A:BM} - \mathcal{E}_{A:B} & = & I_{AB:M} + \nonumber \\
		&&\sum_m p_m [ -\mathrm{tr}(\rho_{AB|m} \log \sigma_{AB|m}) ] \nonumber \\
		& - & \sum_m p_m [ - \mathrm{tr}(\rho_{AB|m} \log \sigma_{AB})].
	\end{eqnarray}
	The lemma follows by noting that the difference between the last two sums is non-positive because each $\sigma_{AB|m}$ minimises the relative entropy of entanglement of the corresponding $\rho_{AB|m}$.
\end{proof}

Eq.~\ref{EQ_I_BOUND} is the result of the following theorem.
\begin{theorem}
	In a tripartite system with classically correlated mediator at all times (each subsystem can be open to its local environment) we have
	\begin{equation}
	\mathcal{E}_{A:B}(t) - \mathcal{E}_{A:B}(0) \le I_{AB:M}(0),
	\end{equation}
	with notation as in the lemma.
\end{theorem}

\begin{proof}
	Consider the following chain of inequalities:
	\begin{eqnarray}
	\mathcal{E}_{A:B}(t) - \mathcal{E}_{A:B}(0) & \le & \mathcal{E}_{A:BM}(t) - \mathcal{E}_{A:B}(0) \\
	& \le & \mathcal{E}_{A:BM}(0) - \mathcal{E}_{A:B}(0) \label{EQ_REVEALING} \\
	& \le & I_{AB:M}(0).
	\end{eqnarray}
	In the first line we used monotonicity of entanglement under tracing out the mediator $M$,
	i.e, $\mathcal{E}_{A:B}(t) \le \mathcal{E}_{A:BM}(t)$.
	Inequality in Eq.~\ref{EQ_REVEALING} is proven in Ref.~\cite{krisnanda2017} and states that entanglement in partition $A:BM$ (or $B:AM$) cannot grow via classical mediator,
	i.e, $\mathcal{E}_{A:BM}(t) \le \mathcal{E}_{A:BM}(0)$.
	Lemma 1 confirms the last line.
\end{proof}

\section{Acknowledgments}
TP acknowledges discussions with Hermann Kampermann. We thank the organisers of QIPA 2018 where this collaboration was initiated. This work is supported by the Singapore Ministry of Education Academic Research Fund Tier 2, Project No. MOE2015-T2-2-034 and Polish National Agency for Academic Exchange NAWA Project No. PPN/PPO/2018/1/00007/U/00001. TK thanks Timothy C. H. Liew for hospitality at Nanyang Technological University. SP and PR acknowledge discussions with V. R. Krithika. TSM acknowledges the support from the Department of Science and Technology, India (Grant Number DST/SJF/PSA-03/2012-13) and the Council of Scientific and Industrial Research, India (Grant Number CSIR-03(1345)/16/EMR-II).

\bibliographystyle{plainnat}


\section{Introduction}
\thispagestyle{fancy}
\definecolor{spana}{RGB}{215,47,16}
\definecolor{spanb}{RGB}{0,116,63}
\begin{table}[ht]
    \centering
    \small
        \begin{tabular}{p{7.2cm}}
        \hline
          \textbf{Article}: marseille , \textcolor{spanb}{france the french prosecutor leading an investigation into the crash of germanwings flight} 9525 insisted wednesday that \textcolor{spana}{he was not aware of any video footage} from on board the plane . marseille prosecutor brice robin told cnn that so far no videos were used in the crash investigation ...... of a cell phone video showing the harrowing final seconds from on board germanwings flight 9525 \textcolor{spanb}{as it crashed into the french alps .} ......paris match and bild reported \textcolor{spanb}{that the video was recovered from a phone at the wreckage site .} ...... cnn 's frederik pleitgen , pamela boykoff , antonia mortensen , sandrine amiel and anna-maja rappard contributed to this report . \\ \hline
          \textbf{CNN Seq2Seq}: \textcolor{spana}{french prosecutor UNK robin says he was not aware of any video .} \\ \hline
          \textbf{DivCNN Seq2Seq with Micro DPPs}: \textcolor{spanb}{new french prosecutor leading an investigation into the crash of UNK wings flight UNK 25 which crashed into french alps . the video was recovered from a phone at the wreckage site .} \\ \hline
          \textbf{DivCNN Seq2Seq with Macro DPPs}: \textcolor{spana}{french prosecutor says he was not aware of any video footage} \textcolor{spanb}{from on board UNK wings flight UNK 25 as it crashed into french alps .} \\ \hline
        \end{tabular}
    \caption{Article-summary sample from CNN-DM dataset. Colored spans are attentive parts. Micro DPPs model puts wider attention on article than vanilla does and Macro DPPs puts the widest attention, including former two models' attentive parts.}
    \label{table:sample}
\end{table}

Given an article, abstractive summarization aims at generating one or several short sentences that cover the main idea of original article, which is a combination of Natural Language Understanding(NLU) and Natural Language Generation(NLG).

Abstractive summarization uses Seq2Seq models~\cite{sutskever2014sequence} which consist of an encoder, a decoder and attention mechanism~\cite{mnih2014recurrent}. With attention mechanism the decoder can choose a weighted context representation at each generation step so it can focus on different parts of encoded information. Seq2Seq with attention achieved remarkable results on machine translation~\cite{bahdanau2014neural} and other text generation tasks such as abstractive summarizaiton~\cite{rush2015neural}.

Unlike machine translation that emphasizes attention mechanism as a method of learning word level alignments between source text and target text, attention in summarization should be soft and diverse. Many works noticed that attention may be over concentrated for summarization and hence cause problems like generating duplicate words or duplicate sentences. Researchers try to solve these problems by introducing various attention structures, including local attention~\cite{luong2015effective}, hierarchical attention~\cite{nallapati2016abstractive}, distraction attention~\cite{chen2016distraction} and coverage mechanism~\cite{see2017get} etc. But all these works ignore another repeat problem, as we call it, "Original Text Repetition". We define and explain this problem in section 3.

In this paper we propose a novel Diverse Convolutional Seq2Seq Model(DivCNN Seq2Seq) based on Micro Determinantal Point Processes(Micro DPPs) and Macro Determinantal Point Processes(Macro DPPs). Our contributions are as follows:
\begin{itemize}
    \item We define and describe the Original Text Repetition problem in abstractive summarization and identify the cause behind it, which are degenerated attention distributions. We have also introduced three article-related metrics for the Original Text Repetition estimation and applied them in our experiments.
    \item We suggest a solution to this problem in the form of introducing DPPs into deep neural network (DNN) attention adjustment and propose DivCNN Seq2Seq. In order to adapt DPPs to large scale computing, we propose two kinds of methods: Micro DPPs and Macro DPPs. To the best of our knowledge, this is the first attempt to adjust attention distributions considering both quality and diversity.
    \item We evaluate our models on six open datasets and show its superiority on improving the comprehensiveness of generated summaries without losing much training and inference speed.
\end{itemize}

\section{Convolutional Seq2Seq Learning}
\thispagestyle{plain}
Usually encoder and decoder in Seq2Seq architecture are recurrent neural network(RNN) or its variants like Long Short Term Memory (LSTM)~\cite{hochreiter1997long} and Gated Recurrent Unit (GRU)~\cite{chung2014empirical} network. Recently, a Seq2Seq architecture based entirely on convolutional neural networks (CNN Seq2Seq)~\cite{gehring2017convolutional} was proposed. It has better hierarchical representation of natural language and can be computed in parallel. In this paper we choose CNN Seq2Seq as our baseline system because it performs better on capturing long-term dependency, which is important for summarization.

Both encoder and decoder in CNN Seq2Seq consist of convolutional blocks. Each block contains a one dimensional convolution (Conv1d), a gated linear unit (GLU)~\cite{dauphin2017language} and several fully connected layers for dimension transformation. Residual connection~\cite{he2016deep} and batch normalization~\cite{ioffe2015batch} are used in each block. Each block receives an input $I$ of size $\mathbb{R}^{B*T*C}$, where $B$, $T$, and $C$ are respectively batch size, length of text and number of channels (the same as embedding size). Conv1d pads the sentence first and then generates a tensor $[O_1,O_2]$ of size $\mathbb{R}^{B*T*2C}$, doubling the channel. The extra channels are used in a simple non-linearity gated mechanism:
\begin{gather}
    O_1,O_2 = Conv1d(I) \\
    GLU([O_1,O_2]) = O_1 \otimes \sigma(O_2)
\end{gather}

Multi-step attention~\cite{gehring2017convolutional} are used in CNN Seq2Seq. Each convolutional block in decoder has its own attentive context. Followed on query-key-value definition of attention, queries $Q \in \mathbb{R}^{B*T_g*C}$ are different decoder block outputs, where $T_g$ stands for summary length; keys $K \in \mathbb{R}^{B*T_s*C}$ are encoder last block outputs, where $T_s$ stands for article length; values are sum of encoder input embeddings $E \in \mathbb{R}^{B*T_s*C}$ and $K$. Because of the parallel architecture, attention for all decoder time steps can be calculated at once. Such architecture can speed up training and give convenience for our DPPs calculation. Using the simplest dot product attention, all the calculations can be done with an efficient batch matrix multiplication (BMM).
\begin{gather}
    score_{attn} = BMM(Q,K) \\
    weight_{attn} = Softmax(score_{attn}) \\
    context = BMM(weight_{attn},K+E)
\end{gather}

\section{Original Text Repetition}
Original Text Repetition(OTR) problem means that each sentence in generated summaries are repetitions of article sentences.  The abstractive summarization hence degenerates to extractive summarization. The ROUGE metric can not detect this problem since it only measures the n-grams concurrence between generated summaries and gold summaries without taking articles into consideration. The word repeat problem~\cite{see2017get} or the lack of abstraction problem~\cite{kryscinski2018improving} can be seen as extreme condition or alternative description of OTR. Behind this phenomenon is the degenerated attention distribution learned by model which we define as:
\begin{itemize}
    \item \textbf{Narrow Word Attention} for each summary word, the attention distribution narrows to one word position in article.
    \item \textbf{Adjacent Sentence Attention} for all words in each summary sentence, their positions of attention peaks are adjacent or semantically adjacent, which means that attended article parts have similar features.
\end{itemize}

\begin{figure}[t]
  \includegraphics[width=\linewidth]{attention.png}
  \caption{Degenerated attention distribution behind OTR problem. The generated summary repeats the first sentence in article. We select the first 16 words of summary and show their attention over first 50 words of article.}
  \label{fig:attention}
\end{figure}

As shown in the Figure \ref{fig:attention}, sentence attention degenerates to several adjacent peaks on the repeated article positions. Usually each sentence in gold summaries considers multiple article sentences and induct to one, not simply copying one article sentence. The gap between generated summaries(copy) and gold summaries(induce) means that model just learned to find article sentences that has the maximum similarity to gold summaries not the relation between article facts and summaries. Degenerated attention mechanism misleads the model.
\section{Diverse Convolutional Seq2Seq Model}
\thispagestyle{plain}
To prevent Seq2Seq model from attention degeneration, we introduce DPPs as a method of regularization in CNN Seq2Seq and propose DivCNN Seq2Seq.
\subsection{Quality and Diversity Decomposition of Determinantal Point Processes}
DPPs have been widely used in recommender systems, information retrieval and extractive summarization systems. It can generate subsets with both high quality and high diversity~\cite{kulesza2011k}.

Given a discrete, finite point process $P$ and a ground set $D$, if for every $A \in D$ and a random subset $\boldsymbol{Y}$ drawn according to P, there is:
\begin{gather}
    P(A \in \boldsymbol{Y}) = det(K_A)
\end{gather}
where K is a real symmetric matrix that indexed by the elements of D, then P is a determinantal point process and K is the marginal kernel of DPPs. Marginal kernel merely gives marginal probability of one certain item to be selected in one particular sampling process, hence we use L-ensemble~\cite{kulesza2011k} to model atomic probabilities for every possible instantiation of Y:
\begin{gather}
    K = L(L+I)^{-1} = I - (L+I)^{-1} \\
    P_L(\boldsymbol{Y} = Y) \propto det(L_Y) \\
    P_L(\boldsymbol{Y} = Y) = \frac{det(L_Y)}{det(L+I)} \label{equ:qdscore}
\end{gather}
L-ensemble is also one kind of DPPs and can be constructed directly using the quality($q$) and similarity($sim$) of point set:
\begin{gather}
    L_{i,j} = q(i)*sim(i,j)*q(j)
\end{gather}
Equation \ref{equ:qdscore} is a probability that subset $Y$ being chosen, which is actually a quantitative indicator for the score of the subset considering both its quality and diversity(QD-score). Summarization follows the same principle: a good summary should consider both information significance and redundancy. In extractive summarization set of sentences with high score (quality) and diversity is chosen to a summary, using DPPs sampling algorithm~\cite{li2017uids}.

In Figure \ref{fig:dpps} we show the difference between quality-only sampling and DPPs sampling. We first generate a simulated attention distribution for testing. Then we use word position distance as similarity measure and attention as quality to construct the $L$ matrix($L$-ensemble) for DPPs. Point subset is sampled based on quality (green) or DPPs (blue), then a gaussian mixture distribution was generated around these points to soften and reweight the attention. Both samplings approximate the distribution of original attention distribution (orange), but DPPs approximate it better and have more scattering peaks. Sampling only considering attention weight (quality) generates less peaks, which means many adjacent points with low diversity are sampled.

\begin{figure}[ht]
  \includegraphics[width=\linewidth]{dpps.png}
  \caption{Comparison of different reweighting methods on a simulated distribution. DPPs sampling reweighting approximates original distribution better since it catches the high attention area around position 160. It also samples less adjacent points around position 110.}
  \label{fig:dpps}
\end{figure}

In actual experiments we choose attention weight as quality. The model learns attention distribution to score different parts of article and obviously higher attention means higher quality. In original CNN Seq2Seq the sum of encoder output and encoder input embeddings are encoded feature vectors. We follow this setting and use the feature vectors to calculate cosine similarity. Specifically, the encoder output are tree-like semantic features extracted by CNN encoder while the encoder input embedding provides point information about a specific input element before encoding~\cite{gehring2017convolutional}. Hence feature vectors contain both highly abstract semantic features and specific grammatical features when calculating diversity. Compared to extractive summarization, DPPs in abstractive summarization use status of DNN as quality and diversity which can be optimized dynamically during training.

\begin{figure}[t]
  \includegraphics[width=\linewidth]{matrix_l.pdf}
  \caption{Construction of $L$ matrix}
  \label{fig:matrix_l}
\end{figure}

The computation of $L$ matrix is shown in Figure \ref{fig:matrix_l}. For each sample in a batch(128 in our experiments), the encoder input embeddings $E \in  \mathbb{R}^{T_s*C}$ multiply its transpose to produce similarity matrix $S \in \mathbb{R}^{T_s * T_s}$. The weight vectors of Multi-step Attention average over decoder layers and summary length, then do the same operation to generate quality matrix $Q \in R^{T_s*T_s}$. Then we use the hadamard product of $Q$ and $S$ as $L \in R^{T_s*T_s}$.

\subsection{Macro DPPs}
\thispagestyle{plain}
\begin{figure}[ht]
  \includegraphics[width=\linewidth]{macro_dpp.pdf}
  \caption{Conditional sampling in Macro DPPs}
  \label{fig:dpp_macro}
\end{figure}

The idea of Macro DPPs is to pick subsets under some restriction and evaluate QD-score of subset using equation \ref{equ:qdscore}. The ideal attention distribution should have subsets with high QD-score.

We do not use DPPs sampling since the purpose of Macro DPPs is to evaluate subsets not to sample subsets with high QD-score. The attention distribute over the ground set so we introduce conditional sampling to sample a subset that has high quality or high diversity, then improve the other metric as follows:
\begin{itemize}
    \item \textbf{Improve Diversity in High Quality Subset} Select points with high attention weight to construct subset and require no gradient for quality matrix, just optimize diversity.
    \item \textbf{Improve Quality in High Diversity Subset} Sampling point subset with high diversity is hard to realize, so we just make equidistant(equidistant on word positions) sampling to approximate it. Contrary to the previous method, we require no gradient for similarity matrix and just optimize quality.
\end{itemize}

We randomly choose one condition in each batch. After the point subset was chosen, the submatrix $L_Y$ can be built by selecting elements in $L$ indexed by point subset. Then we calculate the QD-score of the submatrix and add it into model loss as a regularization. We calculate the logarithmic summation of eigenvalues to prevent numerical underflow.
\begin{gather}
    loss_{QD} = \sum \log \lambda _{L + I} - \sum \log \lambda _{L_Y}   \\
    \propto \frac{det(L_Y)}{det(L+I)} \\
    loss_{model} = \gamma loss_{MLE} + (1 - \gamma)loss_{QD}
\end{gather}

\subsection{Micro DPPs}
\thispagestyle{plain}
The idea of Micro DPPs is to sample a subset $Y$ with large QD-score from all article positions and to use these sampled points as adjusted attention focus points. Then a Gaussian Mixture (GM) distribution around these points is generated as ideal attention distribution($weight_{ideal})$. The whole process can be seen as a selection and softening on attention. The Kullback-Leibler divergence of the ideal distribution and attention distribution ($weight$) then is added into the loss function as regularization.
\begin{gather}
    P = BFGMInference(L,t) \\
    weight_{ideal} = GM_{\mu \in P}(\mu, \sigma, \pi) \\
    loss_{KL} = KLdiv(weight_{ideal},weight_{attn}) \\
    loss_{model} = \gamma loss_{MLE} + (1 - \gamma)loss_{KL}
\end{gather}

Classic sampling algorithm for DPPs~\cite{kulesza2011k} runs slow when the size of $L$ matrix is large and it can not be computed in batch. In the DivCNN Seq2Seq model we need to construct an $L$ matrix for every sample and every layer in the decoder, which is ultimately large. To optimize DPPs runtime for this large-scale computation, we introduce a Batch computation version of Fast Greedy Maximum A Posteriori Inference~\cite{chen2018fast}(BFGMInference) to sample a subset with high QD-score.
\begin{algorithm}
\caption{BFGMInference}
\label{alg:BFGMInference}
\begin{algorithmic}[1]
\fontsize{10}{10}\selectfont
\REQUIRE matrix $L \in \mathbb{R}^{B*T_s*T_s}$, size of sampled subset $t$
\ENSURE Sampled subset $Y \in \mathbb{R}^{B*t}$
\STATE Initialize $D_i = L_{ii}$; $mask = 1^{B*T_s}$; $J = \mathop{argmax}(\log (D*mask))$; $C \in 0^{B*T_s*1}$
\STATE $mask_{j \in J} = 0$
\STATE $count = 1$
\WHILE{$count<t$}
    \STATE $candidate = \{ i|mask_i = 1 \}$
    \STATE $ctemp = 0^{B*T_s*1}$, $dtemp = 0^{B*T_s}$
    \FOR{$idx=0;idx<Ts-count;idx++$}
        \STATE $i = candidate[:,idx]$, $j = J$
        \STATE $e_i = (L_{j,i} - \langle c_j,c_i \rangle )/d_j$
        \STATE $ctemp_i=e_i$, $dtemp_i = {e_i}^2 $
    \ENDFOR
    \STATE $C = [C,ctemp]$, $D = D - dtemp$
    \STATE $J = \mathop{argmax}(\log (D*mask))$
    \STATE $mask_{j \in J} = 0$
    \STATE $count =  count + 1$
\ENDWHILE
\STATE $Y = \{ i|mask_i = 0 \}$
\RETURN $Y$
\end{algorithmic}
\end{algorithm}

BFGMInference uses a greedy method to approximate the MAP result $Y_{map} = \mathop{argmax}_{Y \in D}det(L_Y)$: each time we select $j$ that has maximum QD-score improvements and add it to $Y$.
\begin{gather}
    f(Y) = \log det(L_Y) \\
    j = \mathop{argmax}_{i \in D \backslash Y} f(Y \cup \{i\}) - f(Y) \label{equ:argmax_j}
\end{gather}
By using Cholesky decomposition we have:
\begin{gather}
    L_Y = VV^T \\
    L_{Y \cup \{i\}} = \left[ \begin{matrix} V & 0 \\ c_i & d_i \end{matrix} \right] \left[ \begin{matrix} V & 0 \\ c_i & d_i \end{matrix} \right] ^T \\
    Vc_i^T = L_{Y,i} \label{equ:c_update} \\
    d_i^2 = L_{ii} - ||c_i||_2^2 \label{equ:d_update}
\end{gather}
Then we can transform equation \ref{equ:argmax_j} into:
\begin{gather}
    j = \mathop{argmax}_{i \in D \backslash Y} \log(d_i^2)
\end{gather}
The $c$ and $d$ can be updated incrementally according to equation \ref{equ:c_update} and \ref{equ:d_update}. The complete algorithm is described in Algorithm \ref{alg:BFGMInference}. BFGMInference algorithm gains significant speed improvements when the size of $L$ matrix is large as shown in Figure \ref{fig:speedup}.
\begin{figure}
  \includegraphics[width=\linewidth]{speedup.png}
  \caption{Speed comparison of classical DPPs sampling (blue), FGMInference (red) and BFGMInference (gray) with a batch size of 100.}
  \label{fig:speedup}
\end{figure}

\begin{table*}[ht]
\centering
\footnotesize
\begin{tabular}{ccccc}
\hline
\textbf{Dataset}  & \textbf{\# Docs} & \textbf{Type} & \textbf{\begin{tabular}[c]{@{}c@{}}Average Document \\ Words\end{tabular}} & \textbf{\begin{tabular}[c]{@{}c@{}}Average Summary \\ Words\end{tabular}} \\ \hline
\textbf{CNN-DM}   & 287227/13368/11490          & News      & 789/777/768                                                                                & 55/59/62                                                                                \\
\textbf{NEWSROOM} & 995041/108862/108837        & News       & 659/654/652                                                                                & 26/26/26                                                                                \\
\textbf{REDDIT} & 34000/4000/4000               & Social Media & 418/445/451                                                                                & 20/23/25                                                                                \\
\textbf{BIGPATENT} & 1207222/67068/67072        & Documentation       & 699/699/699                                                                                & 116/116/116                                                                                \\
\textbf{TLDR} & 960000/20000/20000              & Social Media & 197/195/204                                                                                & 19/19/19                                                                                \\
\textbf{WIKIHOW} & 180000/10000/20000           & Knowledge Base    & 475/488/418                                                                                & 62/60/74                                                                                \\ \hline
\end{tabular}
\caption{Dataset overview(train/valid/test).}
\label{table:dataset}
\end{table*}

\section{Experimental Setup}

\thispagestyle{plain}

\textbf{Datasets} We test DivCNN Seq2Seq model on the widely used CNN-DM dataset ~\cite{hermann2015teaching} and give detailed analysis on diversity and quality. Also we tried our model on other five abstractive summarization datasets which are NEWSROOM corpus~\cite{grusky2018newsroom}, TLDR~\cite{volske2017tl}, BIGPATENT~\cite{sharma2019bigpatent}, WIKIHOW~\cite{koupaee2018wikihow} and REDDIT~\cite{kim2018abstractive}. For CNN-DM corpus we truncate articles to 600 words and summaries to 70 words. For other corpus we only keep articles and summaries that have length around its average length. Specially we only use the TIFU-long version of REDDIT and non-anonymized version of CNN-DM dataset. If the raw datasets were not divided into train/dev/test then we divide the shuffled datasets manually. Details of all six datasets are shown in Table \ref{table:dataset}.

\thispagestyle{plain}

\textbf{Hyperparameters and Optimization} All the CNN models use a 50000 words article dictionary and 20000 words summary dictionary with byte pair encoding (BPE)~\cite{sennrich2015neural}. Word embeddings are pretrained on training corpus using Fasttext~\cite{bojanowski2017enriching,joulin2016fasttext}. We do not train models with large parameters to increase ROUGE results since what we try to improve is the comprehensiveness of each sentence in summary. The total parameters of whole CNN seq2seq Model are about 3800w and the DivCNN Seq2Seq does not change the parameters amount. All the models set embedding dimensionality and CNN channels to 256. The encoder has 20 blocks with kernel size 5 and the decoder has 4 blocks with kernel size 3. Such scale of model parameters are enough for the model to generate fluent summaries. The $\gamma$ is 0.6 for Macro DPPs and 0.7 for Micro DPPs. In Macro DPPs we choose top 30 points when optimizing diversity and a stride of 20 for equidistant. In Micro DPPs for each summary we sample 20 points to generate gaussian mixture distributions. We train the model with Nesterov's accelerated gradient method using a momentum of 0.99 and renormalized gradients when the norm exceeded 0.1~\cite{sutskever2013importance}. The beam search size is 5 and we apply a dropout of 0.1 to the embeddings and linear transform layers. We did not fix training epoches. The model was trained until the average epoch loss can not be lower anymore. The DPPs regularization only works when training and does not bring extra parameters into model. During test the model has learned proper attention so the generation is the same as vanilla CNN Seq2Seq Model.

\thispagestyle{plain}

\textbf{Article-Related Metrics} It is hard to evaluate a summary since summarization itself is very subjective. ROUGE compares generated summaries and gold summaries in checking concurrence of n-grams that results in a very limited evaluation in a word level. We set three article-related metrics to evaluate the comprehensiveness of summaries:

\thispagestyle{plain}

\begin{itemize}
    \item \textbf{Jaccard Similarity Upper Bound (JS)} For each summary sentence, we compute its jaccard similarity with every article sentence. The largest jaccard similarity for each summary sentence is selected as JS. It measures the extent to which summaries copy articles. The worst situation is 1.
    \item \textbf{Sentence Coverage (SC)} We define those article sentences that have a jaccard similarity higher than the gold summaries’ JS value as covered sentence. Then the average counts of covered article sentences for each summary sentence is a ratio that can be used to measure the coverage of the article by the summaries. The worst situation is less than or equal to 1.
    \item \textbf{Novel Bigram Proportion (NOVEL)} The percentage of bigrams in summaries that did not appear in the article. It reflects the abstraction of summaries. The worst situation is 0.
\end{itemize}

\textbf{Strong Baseline} We chose four strong baselines which reported high ROUGE scores. BottomUp~\cite{gehrmann2018bottom}, sumGAN~\cite{liu2018generative} and RL Rerank~\cite{chen2018rewriting} are complicate systems that have additional modules or post-processings and partially relieved the OTR problem. The Pointer Generator~\cite{see2017get} reaches best ROUGE result in single end to end model but suffers greatly from repetition problem.

\textbf{Various Possible Causes of the OTR problem} We had supposed several other reasons for the repetition problems besides attention degeneration including overfitting, bad usage of translation-style attention mechanism, lack of decoding ability and high variance attention distribution. Respectively, we designed comparative experiments as follows:
\begin{itemize}
    \item \textbf{Overfitting} We set dropout ratio to 0.1 (SMALL) and 0.5 (LARGE) for testing overfitting.
    \item \textbf{Direct Attention} Remove the encoder input embedding in attention value so the decoder looks at highly abstract features directly (DIRECT).
    \item \textbf{Lack of Decoding Ability} We double (DOUBLE) or half (HALF) the vanilla (VANILLA) decoder layers to adjust the decoding ability.
    \item \textbf{High Variance Attention} We scale down the attention distribution manually when trainning (SCALE), lowering the variance of the distribution.
\end{itemize}

\begin{table*}[ht]
\centering
\begin{tabular}{clcccccc}
\hline
\multicolumn{2}{c}{\textbf{Model}} & \textbf{\begin{tabular}[c]{@{}c@{}}JS\end{tabular}} & \textbf{\begin{tabular}[c]{@{}c@{}}SC\end{tabular}} & \textbf{\begin{tabular}[c]{@{}c@{}}NOVEL\end{tabular}} & \textbf{\begin{tabular}[c]{@{}c@{}}R1\end{tabular}} & \textbf{\begin{tabular}[c]{@{}c@{}}R2\end{tabular}} & \textbf{\begin{tabular}[c]{@{}c@{}}RL\end{tabular}} \\ \hline
\multicolumn{2}{c}{gold} & 0.326 & -- & 0.575 & -- & -- & -- \\ \hline
\multicolumn{2}{c}{\cite{liu2018generative} sumGAN} & 0.709 & 1.136 & 0.118 & 39.92 & 17.65 & 27.25 \\
\multicolumn{2}{c}{\cite{gehrmann2018bottom} BottomUp} & \textbf{0.541} & 1.015 & 0.098 & \textbf{41.53} & \textbf{18.76} & \textbf{27.92} \\
\multicolumn{2}{c}{\cite{chen2018rewriting} RL Rerank} & 0.585 & 1.181 & 0.105 & 39.38 & 16.03 & 24.95 \\
\multicolumn{2}{c}{\cite{see2017get} Pointer Generator} & 0.774 & \textbf{1.317} & 0.079 & 39.53 & 17.28 & 26.89 \\ \hline
\multicolumn{2}{c}{CNN Seq2Seq Vanilla} & 0.616 & 1.137 & 0.167 & 30.4 & 11.7 & 23.09 \\
\multicolumn{2}{c}{DivCNN Seq2Seq with Micro DPPs} & 0.568  & 1.214  & \textbf{0.183} & 30.61 & 11.82 & 23.19 \\
\multicolumn{2}{c}{DivCNN Seq2Seq with Macro DPPs} & 0.587 & 1.265  & 0.177 & 32.28 & 12.75 & 24.32 \\ \hline
\multicolumn{2}{c}{Extract with Attention} & --  & -- & -- & 35.48 & 13.67 & 22.85 \\
\multicolumn{2}{c}{Extract with DPPs Diversified Attention} & -- & -- & -- & 35.35 & 13.69 & 23.07 \\ \hline
\end{tabular}
\caption{Results on CNN-DM datasets.}\label{table:cnndm_result}
\end{table*}

\section{Results}
\thispagestyle{plain}
\begin{table}[ht]
\centering
\footnotesize
\begin{tabular}{ccccc}
\hline
\multicolumn{2}{c}{\textbf{Model}} & \textbf{\begin{tabular}[c]{@{}c@{}}JS\end{tabular}} & \textbf{\begin{tabular}[c]{@{}c@{}}SC\end{tabular}} & \textbf{\begin{tabular}[c]{@{}c@{}}NOVEL\end{tabular}} \\ \hline
\multicolumn{2}{c}{SCALE} & \textbf{0.382} & 0.79 & 0.199 \\
\multicolumn{2}{c}{DIRECT} & 0.567 & 1.14 & \textbf{0.207} \\ \hline
\multicolumn{1}{c|}{\multirow{3}{*}{LARGE}} & DOUBLE & 0.591 & 1.201 & 0.192 \\
\multicolumn{1}{c|}{} & VANILLA & 0.639 & 1.261 & 0.162 \\
\multicolumn{1}{c|}{} & HALF & 0.639 & 1.281 & 0.153 \\ \hline
\multicolumn{1}{c|}{\multirow{3}{*}{SMALL}} & DOUBLE & 0.631 & 1.259 & 0.167 \\
\multicolumn{1}{c|}{} & VANILLA & 0.616 & 1.137 & 0.167 \\
\multicolumn{1}{c|}{} & HALF & 0.625 & \textbf{1.29} & 0.161 \\ \hline
\end{tabular}
\caption{Explore various possible causes of OTR.}\label{table:explore_result}
\end{table}

\textbf{Various Possibilities} As shown in Table \ref{table:explore_result}, scaled attention has the lowest jaccard similarity upper bound, which confirms our idea that over-concentrated attention makes model to copy article sentences. As for sentence coverage, small decoder with large drop out ratio performs the best, proving that large and overfitted models may have degenerated attention. Although the scaled attention has the best JS score, its SC score is the worst (SC less than 1.0 means duplicate sentences are generated). So, we may conclude that directly scaling down attention breaks the value of attention. The ideal attention is not about erasing the peak or the variance of attention but to have multiple peaks in sentence attention and have high diversity at the same time. Neither aggregative nor scattering attention distributions do good to summary generation. Direct attention model has the maximum NOVEL score which means point information about a specific input element makes model prefer copying article words instead of generating new words.

\textbf{CNN-DM Results} With large model parameters and dictionaries, four models in strong baselines reach nearly 40 points in ROUGE-1 but they perform poorly on article-related metrics. Single end to end systems like Pointer Generator performs poorly on JS value and NOVEL proportion which means most of its summaries are copied from articles. As for three models with multiple modules or post-processing, the BottomUp model has relatively good jaccard similarity upper bound and the best ROUGE result but its article-related metrics are still far away from gold summaries level. RL Rerank model has better score on JS and sumGAN has better NOVEL score but none of these model reached a balanced good performance on three article-related metrics. Compared to vanilla CNN Seq2Seq, DivCNN Seq2Seq improves the JS and NOVEL points and raises the ROUGE score at the same time, proving that proper attention distribution can help reaching a better local optima. Compared with strong baselines, DivCNN Seq2Seq achieves the best in NOVEL, second and third in JS and SC, respectively. Empirically we suggest that $\gamma$ for both Micro and Macro DPPs should be set to make average loss change less than $10\%$ compared to vanilla models. We also observed that Micro DPPs is more sensitive to $\gamma$ compared to Macro DPPs and is easier to converge but may degenerate to vanilla CNN Seq2Seq. Macro DPPs usually can reach better results but it needs more time to train since eigenvalue calculation is expensive and can not be accelerated through fp16 tensor computing.

\textbf{Novel Bigram} NOVEL is a tricky metric which is used in many researches about abstractive summarization. There are many possibilities on explaining a high NOVEL score: first, the summary get a high Novel Bigram ratio because it has many Novel unigrams, which may be good or bad; second, the model may be underfitting and can not generate fluent sentences; third, the generated summary use novel bigrams to conclude the original text and generate readable sentences, which is the best condition. From the table \ref{table:cnndm_result} we can see that Bottom Up has the best ROUGE and JS results but worst NOVEL score. Our DivCNN Models have just the opposite metric scores. These three metrics shouldn't be ambivalent since gold summaries can reach high NOVEL and low JS at the same time. Base on these facts we make the following conjectures:
\begin{itemize}
    \item Good summaries model learned have styles differ from human-write summaries. Model tend to copy bigrams from original article and reorganize them into short summary sentences. Human tend to use brand new bigrams to paraphrase facts contained in original article. The model use a rewrite(compress and extract) way while human-write is overwrite style.
    \item Though we use human-write summaries as gold summaries for model to learn and the MLE loss is steadily descending during training but it learned summaries with different style. It implicates that model may not have a "NLU+NLG" process like human do but be restrained in a sentence-level rewrite framework. For Bottom Up and RL Rerank it is not a problem because these two systems are designed to rewrite. They only send parts of article into Seq2Seq. Such design can gain high ROUGE score but it is not the way in which human write gold summaries.
\end{itemize}

\begin{table*}[t!]
\centering
\footnotesize
\begin{tabular}{ccccc}
\hline
\textbf{Datasets} & \textbf{Baseline} & \textbf{Baseline} & \textbf{Micro DPPs} & \textbf{Macro DPPs} \\ \hline
NEWSROOM  & LEAD-3 Baseline & 30.63/21.41/28.57 & 38.33/25.01/35.3 & \textbf{40.10}/\textbf{26.71}/\textbf{37.24} \\
TLDR  & - & - & 65.94/56.97/64.65 & \textbf{66.93}/\textbf{57.84}/\textbf{65.71} \\
BIGPATENT & \cite{chen2018rewriting} & \textbf{37.12}/\textbf{11.87}/\textbf{32.45} & 33.21/10.47/24.86 & 34.55/11.65/25.96 \\
WIKIHOW  & \cite{see2017get} & \textbf{28.53}/\textbf{9.23}/\textbf{26.54} & 24.52/6.49/20.56 & 27.58/8.01/22.61 \\
REDDIT  & \cite{kim2018abstractive} & 19/3.7/15.1 & 21.39/4.24/17.11 & \textbf{21.57}/\textbf{4.48}/\textbf{17.29} \\ \hline
\end{tabular}
\caption{ROUGE F1 results(R1/R2/RL) on different datasets} \label{table:rouge_others}
\end{table*}

\begin{figure}[ht]
  \includegraphics[width=\linewidth]{learned_attention.png}
  \caption{Actual attention distribution learned by vanilla model and DPPs models.}
  \label{fig:attn_actual}
\end{figure}
\thispagestyle{plain}
\textbf {Extractive Summarization based on Learned Attention} We also extract article sentences based on sentence attention learned by DPPs models to generate summaries. The attention of a sentence is the sum of the attention weights of the words in the sentence. Table \ref{table:cnndm_result} shows that extractive summarization reaches better ROUGE values, implicating that both vanilla and DivCNN models learned appropriate sentence attention. Extractive summarization uses accumulated sentence attention instead of specific distribution, so the results of vanilla models are almost the same as DivCNN.

\textbf{Sample Visualization} We randomly choose one sample in the test set of CNN-DM corpus to visualize and analyze. As shown in Table \ref{table:sample} we highlighted attentive parts in the article for different models. The vanilla model just generates one sentence which only focuses on one part of article. The Micro DPPs model generates two sentences considering three parts of the article. Macro DPPs considered article spans that both vanilla model  and Micro DPPs model paid attention to. We also checked the attention distribution of this sample. As shown in Figure \ref{fig:attn_actual}, vanilla model (red) learned only several peaks over article position 70 to 90, which suggests that it only focuses on one sentence and repeats this sentence in a summary. Attention learned by Micro DPPs model (green) still narrows to several peaks but explores more positions compared to vanilla. Macro DPPs (blue) has more natural design of loss function and it optimize quality and diversity directly so it has a more scattering attention distribution.

\textbf{More Datasets} We test our model on other five newly-released abstractive summarization datasets which have various compression ratio, different professional field and more flexible human-write summaries. Only ROUGE results are collected since no baseline generated summaries are provided for us to calculate article-related metrics. Table \ref{table:rouge_others} shows that DivCNN performs better than best baselines on NEWSROOM, REDDIT and reaches incredible ROUGE scores more than 60 (but no baseline is reported in the dataset paper so the result is not comparable). The compression ratio and article length have little impact on the performance of DivCNN. The results show that DivCNN prefers short summaries.

\begin{figure}[t!]
  \includegraphics[width=\linewidth]{presentation_degeneration.png}
  \caption{Presentation Degeneration Problem in NLG. We use tSNE~\cite{maaten2008visualizing} to reduce the dimension of word embeddings learned in the model.}
  \label{fig:presentation_degeneration}
\end{figure}
\thispagestyle{plain}
\textbf{Attention \& Representation Degeneration} In order to solve attention degeneration we introduce DPPs to improve the diversity of features where model paid high attention to. This solution is consistent with the Presentation Degeneration Problem in NLG~\cite{gao2018representation}. As shown in Figure \ref{fig:presentation_degeneration}, Macro DPPs have more diverse embedding presentation compared to vanilla model.~\cite{gao2018representation} directly add a regularization loss of diversity to increase the representation power of word embeddings while we aim at generating attention distribution considering both quality and diversity, resulting in learning word embeddings with rich representation power.

\section{Conclusions and Future Works}
We have defined the "OTR" problem that leads to incomplete summaries and revealed the cause behind it, which is attention degeneration. We also introduce three article-related metrics to evaluate this problem. DPPs are applied directly on attention generation and we propose Macro and Micro DPPs versions of DivCNN Seq2Seq model to adjust attention considering both quality and diversity. Results on CNN-DM and other five open datasets show that DivCNN Seq2Seq can improve the comprehensiveness of summaries.

Due to the hardware limitation we only train a small-parameters version of DivCNN. Also we lost some precision when approximating L matrix and accelerating sampling. These drawbacks lead to limited performance improvements. In the future we hope to explore further on following directions: Quantifiable and controllable quality/diversity in DPPs; better approximation in conditional sampling, such as dynamic sampling stride adjustment; try to apply DPPs-optimized attention on another student model to improve generation.
\clearpage
\thispagestyle{plain}