\section{{\normalsize{}{}{}{}{}{}{}{}{}{}Introduction}}

Dain \cite{Dain} constructed a geometric invariant that measures
the \textit{non-stationary} energy for an asymptotically flat hypersurface
in 3+1 dimensions for the case of time-symmetric initial data which,
for vacuum, is an invariant that quantifies the total energy of the
gravitational radiation. So this invariant is a component of the total
ADM energy \cite{ADM} assigned to an asymptotically flat hypersurface.
That construction was extended to the time-non symmetric case recently
in \cite{Kroon}. To give an example of how useful such a geometric
invariant can be when constructing initial data for the gravitational
field, let us recall the first observation of the merger of two black
holes \cite{merger}. According to this observation, two initial black
holes with masses (approximately) 36$\textup{M}_{\odot}$ and 29$\textup{M}_{\odot}$
merged to produce a single stationary black hole of mass 62$\textup{M}_{\odot}$
plus gravitational radiation of total energy equivalent to 3$\textup{M}_{\odot}$.
Assuming this system to be isolated in an asymptotically flat spacetime,
the total initial ADM energy of 65$\textup{M}_{\odot}$ is certainly
conserved. But this total ADM energy of the initial data needs a refinement
as it clearly has a non-stationary part equal to 3$\textup{M}_{\odot}$.
The important question is to identify this non-stationary energy in
the initial data.

Dain's construction and its extension to the non-time symmetric case
by Kroon and Williams \cite{Kroon} are based on several earlier crucial
works one of which is the Killing initial data (KID) concept of Moncrief
\cite{Moncrief} and Beig-Chru\'{s}ciel \cite{Beig}; and a fourth
order operator defined by Bartnik \cite{Bartnik}. Of course all of
the discussion is related to the Cauchy problem in General Relativity
and the related issue of constructing initial data for the time evolution
equations. Here by using the time-evolution equations, in the form
given by Fischer and Marsden \cite{FisMar72}, we construct a new
representation of the non-stationary energy in generic $D$ dimensional
spacetimes with or without a cosmological constant.

The outline of the paper is as follows: in section II we briefly summarize
Dain's construction using the constraints and present a new approach
using the evolution equations. In section III we give the details
of the relevant computations in $D$ dimensions . The Appendix is
devoted to the ADM decomposition.

\section{{\normalsize{}{}{}Dain's invariant in brief and a new formulation}}

Leaving the details of the construction to the next section, let us
first briefly summarize the ingredients needed to define Dain's invariant
on a spacelike hypersurface $\Sigma$ of the spacetime $\mathscr{M}=\mathbb{R}\times\Sigma$.
Then we shall discuss our new formulation via the evolution equations.

The initial data on the hypersurface is defined by the Riemannian
metric $\gamma_{ij}$ and the extrinsic curvature $K_{ij}$ in local
coordinates. Denoting $D_{i}$ to be the covariant derivative compatible
with $\gamma_{ij}$ and assuming the usual ADM decomposition of the
spacetime metric $g_{\mu\nu}$, the line element reads
\begin{equation}
ds^{2}=(N_{i}N^{i}-N^{2})dt^{2}+2N_{i}dtdx^{i}+\gamma_{ij}dx^{i}dx^{j},\label{ADMdecompositionofmetric}
\end{equation}
while the extrinsic curvature becomes \footnote{Our definition of the extrinsic curvature is as follows: given $(X,Y)$
two vectors on the tangent space $T_{p}\Sigma$ and $n$ be the unit
normal to $\Sigma$, then $K(X,Y):=g(\nabla_{X}n,Y)$ with $\nabla$
being the covariant derivative compatible with the spacetime metric
$g$. }
\begin{equation}
K_{ij}=\frac{1}{2N}\left(\dot{\gamma}_{ij}-D_{i}N_{j}-D_{j}N_{i}\right),
\end{equation}
with the lapse function $N=N(t,x^{i})$ and the shift vector $N^{i}=N^{i}(t,x^{i})$.
The spatial indices can be raised and lowered with the $D-1$ dimensional
spatial metric $\gamma$; over dot denotes the derivative with respect
to $t$, and the Latin letters are used for the spatial dimensions,
$i,j,k,...=1,2,3,...D-1$, whereas the Greek letters are used to denote
the spacetime dimensions, $\mu,\nu,\rho,...=0,1,2,3,...D-1$. All
the relevant details of the ADM decomposition are given in the Appendix.

Under the above decomposition of spacetime, the $D$-dimensional Einstein
equations
\begin{equation}
R_{\mu\nu}-\frac{1}{2}Rg_{\mu\nu}+\Lambda g_{\mu\nu}=\kappa T_{\mu\nu}
\end{equation}
yield the Hamiltonian and momentum constraints on the hypersurface
$\Sigma$ as
\begin{eqnarray}
 &  & \Phi_{0}(\gamma,K):=-{}^{\Sigma}R-K^{2}+K_{ij}^{2}+2\Lambda-2\kappa T_{nn}=0,\nonumber \\
 &  & \Phi_{i}(\gamma,K):=-2D_{k}K_{i}^{k}+2D_{i}K-2\kappa T_{ni}=0,~~~~~~~~~~~~~~\label{Einstein_c}
\end{eqnarray}
where $K:=\gamma^{ij}K_{ij}$ and $K_{ij}^{2}:=K^{ij}K_{ij}$. From
now on we shall work in vacuum, hence $T_{\mu\nu}=0$. Denoting $\Phi(\gamma,K)$
to be the constraint covector with components ($\Phi_{0},\Phi_{i}$)
and $D\Phi(\gamma,K)$ to be its linearization about a given solution
$(\gamma,K)$ to the constraints and $D\Phi^{*}(\gamma,K)$ to be
the formal adjoint map, then following Bartnik \cite{Bartnik}, one
defines another operator ${\cal {P}}$:
\begin{equation}
{\cal {P}}:=D\Phi(\gamma,K)\circ\begin{pmatrix}1 & 0\\
0 & -D^{m}
\end{pmatrix}.
\end{equation}
The reason why we need this operator will be clear below. Using the
formal adjoint ${\cal {P}}^{*}$ of Bartnik's operator, Dain \cite{Dain}
defines the following integral over the hypersurface
\begin{equation}
\mathscr{I}(N,N^{i}):=\intop_{\Sigma}dV~{\cal {P}}^{*}\begin{pmatrix}N\\
N^{k}
\end{pmatrix}\cdot{\cal {P}}^{*}\begin{pmatrix}N\\
N^{k}
\end{pmatrix},\label{integralinvariant}
\end{equation}
where the multiplication is defined as
\begin{equation}
\begin{pmatrix}N\\
N^{i}
\end{pmatrix}\cdot\begin{pmatrix}A\\
B_{i}
\end{pmatrix}:=NA+N^{i}B_{i}.
\end{equation}
The integral (\ref{integralinvariant}) is to be evaluated for specific
vectors $\xi:=(N,N^{i})$ that satisfy the fourth-order equation
\begin{equation}
{\cal {P}}\circ{\cal {P}}^{*}\left(\xi\right)=0,\label{approx_KID}
\end{equation}
which Dain called the \textit{approximate Killing initial data} (KID)
equation. It is clear that if $\xi$ satisfies the lower derivative
equation ${\cal {P}}^{*}\left(\xi\right)=0$, then it also satisfies
(\ref{approx_KID}). Moreover, these particular solutions, together
with an assumption on their decay at infinity, also solve the KID
equations which are simply $D\Phi^{*}(\gamma,K)\left(\xi\right)=0$.
In fact this point is crucial but well-established: Moncrief \cite{Moncrief}
proved that $\xi$ is a spacetime Killing vector satisfying $\nabla_{\mu}\xi_{\nu}+\nabla_{\nu}\xi_{\mu}=0$
if and only if it satisfies the KID equations. Namely one has
\begin{equation}
\nabla_{\mu}\xi_{\nu}+\nabla_{\nu}\xi_{\mu}=0\Leftrightarrow D\Phi^{*}(\gamma,K)\left(\xi\right)=0,
\end{equation}
with $(N,N^{i})$ being the projections off and onto the hypersurface
of the Killing vector field $\xi$. The physical picture is clear:
initial data on the hypersurface clearly encode the spacetime symmetries.
There have been rigorous works on the KIDs in \cite{BeiChr96,Chrusciel:2005pb,Beig}
which we shall employ in what follows.

Observe that for any Killing vector field $\text{\ensuremath{\mathscr{I}}}(N,N^{i})$
vanishes identically. So by design, Dain's invariant identically vanishes
for initial data with exact symmetries. Then Dain goes on to show
that for asymptotically flat spaces, for the case of approximate translational
KID's $\text{\ensuremath{\mathscr{I}}}(N,N^{i})$ can measure the
\textit{non-stationary energy } contained in the hypersurface $\Sigma$.
To simplify his calculations Dain considered the time symmetric initial
data ($K_{ij}=0$) in three spatial dimensions.There are two crucial
points to note about Dain's construction: firstly, one can show that
for any asymptotically flat three manifold, the approximate KID equation
has non-trivial solutions which are not KIDs; secondly, using integration
by parts, one can convert the volume integral (\ref{integralinvariant})
to a surface integral. We shall discuss these in the next section,
but let us first give another formulation of this invariant.

\subsection{Non-Stationary Energy via Time-evolution Equations}

In Dain's construction, as is clear from the above summary, time evolution
of the initial data has not played a role: in fact one only works
with the constraints on the hypersurface. This fact somewhat obscures
the interpretation of the proposed invariant as the non-stationary
energy contained in the initial data. In what follows, we propose
another formulation of this invariant with the help of the time evolution
equations which makes the interpretation clearer. For this purpose
let us consider the phase space variables to be the spatial metric
$\gamma_{ij}$ and the canonical momenta $\pi^{ij}$; the latter can
be found from the Einstein-Hilbert Lagrangian
\begin{equation}
\text{\ensuremath{\mathscr{L}}}_{EH}=\frac{1}{\kappa}\sqrt{-g}(R-2\Lambda)=\frac{1}{\kappa}\sqrt{\gamma}N({}^{\Sigma}R+K_{ij}^{2}-K^{2}+\Lambda)+boundary~terms
\end{equation}
which are
\begin{equation}
\pi^{ij}:=\frac{\delta\text{\ensuremath{\mathscr{L}}}_{EH}}{\delta\dot{\gamma_{ij}}}=\frac{1}{\kappa}\sqrt{\gamma}(K^{ij}-\gamma^{ij}K).
\end{equation}
Using the canonical momenta, it pays to recast the densitized versions
of the constraints (\ref{Einstein_c}) for $T_{\mu\nu}=0$ and setting
$\kappa=1$ as
\begin{eqnarray}
 &  & \Phi_{0}(\gamma,\pi):=\sqrt{\gamma}\left(-^{\Sigma}R+2\Lambda\right)+G_{ijkl}\pi^{ij}\pi^{kl}=0,\nonumber \\
 &  & \Phi_{i}(\gamma,\pi):=-2\gamma_{ik}D_{j}\pi^{kj}=0,\label{einstein_c2}
\end{eqnarray}
where the \textit{DeWitt metric } \cite{DeWitt} $G_{ijkl}$ in $D$
dimensions reads
\begin{equation}
G_{ijkl}=\frac{1}{2\sqrt{\gamma}}\left(\gamma_{ik}\gamma_{jl}+\gamma_{il}\gamma_{jk}-\frac{2}{D-2}\gamma_{ij}\gamma_{kl}\right).
\end{equation}
Ignoring the possible boundary terms, the ADM Hamiltonian density
turns out to be a sum of the constraints as
\begin{equation}
{\cal {H}}=\int_{\Sigma}d^{D-1}x\thinspace\left\langle {\cal {N}},\Phi(\gamma,\pi)\right\rangle ,\label{hamilton}
\end{equation}
with ${\cal {N}}$ being the lapse-shift vector with components $(N,N^{i})$
which play the role of the Lagrange multipliers; and the angle-brackets
denote the usual contraction. Given an ${\cal {N}}$, the remaining
evolution equations can be written in a compact form (the Fischer-Marsden
form \cite{Fischer-Marsden}) as
\begin{equation}
\frac{d}{dt}\begin{pmatrix}\gamma\\
\pi
\end{pmatrix}=J\circ D\Phi^{*}(\gamma,\pi)({\cal {N}}),\label{evolution}
\end{equation}
where the $J$ matrix reads
\begin{equation}
J=\begin{pmatrix}0 & 1\\
-1 & 0
\end{pmatrix}.
\end{equation}
The reason why the formal adjoint of the linearized constraint map
$D\Phi^{*}(\gamma,\pi)$ appears in the time evolution can be seen
as follows: the Hamiltonian form of the Einstein-Hilbert action
\begin{equation}
{\cal {S}}_{EH}\left[\gamma,\pi\right]=\intop dt\intop d^{D-1}x\left(\pi^{ij}\dot{\gamma}_{ij}-\left\langle {\cal {N}},\Phi(\gamma,\pi)\right\rangle \right),
\end{equation}
when varied about a background $\left(\gamma,\pi\right)$ gives
\begin{equation}
D{\cal {S}}_{EH}\left[\gamma,\pi\right]=\intop dt\intop d^{D-1}x\left(\delta\pi^{ij}\dot{\gamma}_{ij}+\pi^{ij}\delta\dot{\gamma}_{ij}-\left\langle {\cal {N}},D\Phi(\gamma,\pi)\cdot\left(\delta\gamma,\delta\pi\right)\right\rangle \right).\label{actionvariation}
\end{equation}
Here the linearized form of the constraint map can be computed to
be
\begin{equation}
D\Phi\begin{pmatrix}h_{ij}\\
p^{ij}
\end{pmatrix}=\begin{pmatrix}\sqrt{\gamma}\left(^{\varSigma}R{}^{ij}h_{ij}-D^{i}D^{j}h_{ij}+\triangle h\right)\\
-hG_{ijkl}\pi^{ij}\pi^{kl}+2G_{ijkl}p^{ij}\pi^{kl}+2G_{njkl}h_{im}\gamma^{mn}\pi^{ij}\pi^{kl}\\
\\
-2\gamma_{ik}D_{j}p^{kj}-\pi^{jk}\left(2D_{k}h_{ij}-D_{i}h_{jk}\right)
\end{pmatrix},\label{pppmatrix}
\end{equation}
where $\delta\gamma_{ij}:=h_{ij}$, $h:=\gamma^{ij}h_{ij}$, $\delta\pi^{ij}:=p^{ij}$
and $\triangle:=D_{k}D^{k}$. We have used the vanishing of the constraints
to simplify the expression. In (\ref{actionvariation}) using integration
by parts when necessary and dropping the boundary terms one arrives
at the desired result
\begin{equation}
D{\cal {S}}_{EH}\left[\gamma,\pi\right]=\intop dt\intop d^{D-1}x\left(\delta\pi^{ij}\dot{\gamma}_{ij}-\dot{\pi}^{ij}\delta\gamma_{ij}-\left\langle \left(\delta\gamma,\delta\pi\right),D\Phi^{*}(\gamma,\pi)\cdot{\cal {N}}\right\rangle \right),\label{actionvariation3}
\end{equation}
where the adjoint constraint map appears in the process which reads
\begin{equation}
D\Phi^{*}\begin{pmatrix}N\\
N^{i}
\end{pmatrix}=\begin{pmatrix}\sqrt{\gamma}\left(^{\varSigma}R{}^{ij}-D^{i}D^{j}+\gamma^{ij}\triangle\right)N\\
-N\gamma^{ij}G_{klmn}\pi^{kl}\pi^{mn}+2NG_{klmn}\gamma^{ik}\pi^{jl}\pi^{mn}\\
+2\pi^{k(i}D_{k}N^{j)}-D_{k}(N^{k}\pi^{ij})\\
\\
2NG_{ijkl}\pi^{kl}+2D_{(i}N_{j)}
\end{pmatrix}.\label{formaladjointoflinearizedmap_pi}
\end{equation}
Setting the variation (\ref{actionvariation3}) to zero one obtains
the evolution equations (\ref{evolution}) or in more explicit form
one has
\begin{equation}
\frac{d\gamma_{ij}}{dt}=2NG_{ijkl}\pi^{kl}+2D_{(i}N_{j)},\label{ev1}
\end{equation}
and
\begin{eqnarray}
 &  & \frac{d\pi^{ij}}{dt}=\sqrt{\gamma}\left(-^{\varSigma}R{}^{ij}+D^{i}D^{j}-\gamma^{ij}\triangle\right)N+N\gamma^{ij}G_{klmn}\pi^{kl}\pi^{mn}-2NG_{klmn}\gamma^{ik}\pi^{jl}\pi^{mn}\label{timederivativeofpi}\\
 &  & ~~~~~~~~-2\pi^{k(i}D_{k}N^{j)}+D_{k}(N^{k}\pi^{ij}).\nonumber
\end{eqnarray}
Together with the constraints (\ref{einstein_c2}) these two tensor
equations constitute a set of constrained dynamical system for a \textit{given}
lapse-shift vector an $(N,N^{i})$. The constraints have a dual role:
they determine the viable initial data and also generate time evolution
of the initial data once the lapse-shift vector is chosen. As noted
above, if $D\Phi^{*}(\gamma,\pi)({\cal {N}})=0$, namely ${\cal {N}}=\xi$
is a Killing vector field then the time evolution is trivial. In particular
this would be the case for a stationary Killing vector.

Consider now an ${\cal {N}}$ which is \textit{not} a Killing vector,
which means $D\Phi^{*}(\gamma,\pi)({\cal {N}})\neq0$; and in particular
directly from the evolution equations we can find how much $D\Phi^{*}(\gamma,\pi)({\cal {N}})$
differs from zero (or how much a given ${\cal {N}}$ fails to be a
Killing vector) as
\begin{equation}
D\Phi^{*}(\gamma,\pi)({\cal {N}})=J^{-1}\circ\frac{d}{dt}\begin{pmatrix}\gamma\\
\pi
\end{pmatrix}.
\end{equation}
To get a number from this matrix, first one should note that the units
of $\gamma$ and $\pi$ are different by a factor of $1/L$ and so
a naive approach of taking the "square" of this matrix does not
work. At this stage to remedy this, one needs the (adjoint) operator
of Bartnik that we have introduced above: so one has
\begin{equation}
{\cal {P}}^{*}({\cal {N}}):=\begin{pmatrix}1 & 0\\
0 & D_{m}
\end{pmatrix}\circ D\Phi^{*}(\gamma,\pi)({\cal {N}})=\begin{pmatrix}1 & 0\\
0 & D_{m}
\end{pmatrix}\circ J^{-1}\circ\frac{d}{dt}\begin{pmatrix}\gamma\\
\pi
\end{pmatrix},
\end{equation}
which yields ${\cal {P}}^{*}({\cal {N}})=(-\dot{\pi},D_{m}\dot{\gamma})$.
Since $\pi$ is a tensor density to get a number out of this vector,
we further define
\begin{equation}
\widetilde{{\cal {P}}}^{*}({\cal {N}}):=\begin{pmatrix}\gamma^{-1/2} & 0\\
0 & 1
\end{pmatrix}\circ{\cal {P}}^{*}({\cal {N}}).\label{formaladjoint_ptilde}
\end{equation}
Then the integral of $\widetilde{{\cal {P}}}^{*}({\cal {N}})\cdot\widetilde{{\cal {P}}}^{*}({\cal {N}})$
over the hypersurface yields
\begin{equation}
\text{\ensuremath{\mathscr{I}}}({\cal {N}})=\intop_{\Sigma}dV\thinspace\widetilde{{\cal {P}}}^{*}({\cal {N}})\cdot\widetilde{{\cal {P}}}^{*}({\cal {N}})=\intop_{\Sigma}dV\thinspace\left(|D_{m}\dot{\gamma}_{ij}|^{2}+\frac{1}{\gamma}|\dot{\pi}^{ij}|^{2}\right),\label{bizimdenk}
\end{equation}
where $|D_{m}\dot{\gamma}_{ij}|^{2}:=\gamma^{mn}{\gamma}^{ij}{\gamma}^{kl}D_{m}\dot{\gamma}_{ik}D_{n}\dot{\gamma}_{jl}$
and $|\dot{\pi}^{ij}|^{2}:={\gamma}_{ij}{\gamma}_{kl}\dot{\pi}^{ik}\dot{\pi}^{jl}$.
This is another representation of Dain's invariant which explicitly
involves the time derivatives of the canonical fields. We have also
not assumed that the cosmological constant vanishes, hence our result
is valid for generic spacetimes. Note that this expression is valid
for any ${\cal {N}}$ which is not necessarily an approximate KID,
hence given a solution to the constraint equations and a choice of
the lapse-shift vector, one can compute this integral. But the volume
integral becomes a surface integral when ${\cal {N}}$ is an approximate
KID which is the case considered by Dain. Observe that by construction,
$\text{\ensuremath{\mathscr{I}}}({\cal {N}})$ is a non-negative number.
To get the explicit expression as a volume integral in terms of the
canonical fields and not their time derivatives, one should plug the
two evolution equations (\ref{ev1}) and (\ref{timederivativeofpi})
to (\ref{bizimdenk}). The resulting expression is
\begin{eqnarray}
 &  & \text{\ensuremath{\mathscr{I}}}({\cal {N}})=\intop_{\Sigma}dV\Bigg\{|D_{m}V^{ij}|^{2}+{}^{\varSigma}R_{ij}^{2}N^{2}+(D_{i}D_{j}N)^{2}-2{}^{\varSigma}R^{ij}ND_{i}D_{j}N+2{}^{\varSigma}RN\triangle N\nonumber \\
 &  & ~~~~~~~~~~~~+(D-3)\triangle N\triangle N+2Q\triangle N+Q_{ij}^{2}+2{}^{\varSigma}R_{ij}NQ^{ij}-2Q^{ij}D_{i}D_{j}N\nonumber \\
 &  & ~~~~~~~~~~~~~+4D_{m}D_{(i}N_{j)}D^{m}D^{(i}N^{j)}+4D_{m}D_{i}N_{j}D^{m}V^{ij}\Bigg\},\label{esasintegral}
\end{eqnarray}
where
\begin{equation}
V^{ij}:=\frac{2N}{\sqrt{\gamma}}\left(\pi^{ij}-\frac{1}{D-2}\pi\gamma^{ij}\right),\label{V_ij}
\end{equation}
and
\begin{eqnarray}
Q^{ij}: & = & \frac{2N}{\gamma}\left(\pi_{k}^{i}\pi^{kj}-\frac{\pi\pi^{ij}}{D-2}\right)-\frac{N}{\gamma}\gamma^{ij}\left(\pi_{kl}^{2}-\frac{\pi^{2}}{D-2}\right)\nonumber \\
 &  & -\frac{1}{\sqrt{\gamma}}D_{k}(N^{k}\pi^{ij})+\frac{2}{\sqrt{\gamma}}\pi^{k(i}D_{k}N^{j)},\label{Q_ij}
\end{eqnarray}
and $Q:=\gamma_{ij}Q^{ij}$. Equation (\ref{esasintegral}) is our
main result: given a solution, that is an initial data, one an compute
this integral which measures the deviation from stationarity. We can
also write (\ref{esasintegral}) in terms of $\gamma_{ij}$ and the
extrinsic curvature $K_{ij}$ . For this purpose all one needs to
do is to rewrite $V^{ij}$ and $Q^{ij}$ in terms of these variables.
They are given as
\begin{equation}
V^{ij}=2NK^{ij},
\end{equation}
and
\begin{eqnarray}
Q^{ij}: & = & 2N\left(K_{k}^{i}K^{kj}-KK^{ij}\right)-N\gamma^{ij}\left(K_{kl}^{2}-K^{2}\right)\nonumber \\
 &  & -D_{k}(N^{k}K^{ij})+\gamma^{ij}D_{k}(N^{k}K)+2K^{k(i}D_{k}N^{j)}-2KD^{(i}N^{j)}.
\end{eqnarray}
Up to now we have not made a choice of gauge or coordinates. Let us
now choose the Gaussian normal coordinates ( $N=1$, $N^{i}=0$) on
$\Sigma$ for which the integral reads
\begin{eqnarray}
\text{\ensuremath{\mathscr{I}}}({\cal {N}})= &  & \intop_{\Sigma}dV\Bigg\{\frac{4}{\gamma}\left(|D_{m}\pi^{ij}|^{2}-\frac{D-3}{\left(D-2\right)^{2}}|D_{m}{Tr}(\pi)|^{2}\right)\nonumber \\
 &  & +^{\varSigma}R_{ij}^{2}+\frac{4}{\gamma}{}^{\varSigma}R_{ij}\pi^{ik}\pi_{k}^{j}-\frac{4}{(D-2)\gamma}{}^{\varSigma}R_{ij}\pi^{ij}{Tr}(\pi)\nonumber \\
 &  & -\frac{4}{\gamma}\Lambda\left({Tr}(\pi^{2})-\frac{1}{(D-2)}({Tr}(\pi))^{2})\right)+\frac{D-7}{\gamma^{2}}\left({Tr}(\pi^{2})-\frac{1}{D-2}({Tr}(\pi))^{2}\right)^{2}\nonumber \\
 &  & +\frac{4}{\gamma^{2}}\left({Tr}(\pi^{4})-\frac{2}{D-2}{Tr}(\pi){Tr}(\pi^{3})+\frac{1}{(D-2)^{2}}({Tr}(\pi))^{2}{Tr}(\pi^{2})\right)\Bigg\},
\end{eqnarray}
where ${Tr}(\pi):=\gamma_{ij}\pi^{ij}$ and ${Tr}(\pi^{2}):=\pi^{ij}\pi_{ij}$
and so on. In terms of the extrinsic curvature, in the Gaussian normal
coordinates, one has
\begin{eqnarray}
\text{\ensuremath{\mathscr{I}}}({\cal {N}}) & = & \intop_{\Sigma}dV\Bigg\{4|D_{m}K_{ij}|^{2}+{}^{\varSigma}R_{ij}^{2}+4{}^{\varSigma}R_{ij}(K^{ik}K_{k}^{j}-KK^{ij})\nonumber \\
 &  & +4\Lambda\left(K^{2}-K_{ij}^{2}\right)+4K_{ij}K^{jl}K_{lm}K^{mi}-8KK_{ij}K^{jl}K_{l}^{i}\\
 &  & -2(D-9)K^{2}K_{ij}^{2}+(D-7)\left((K_{ij}^{2})^{2}+K^{4}\right)\Bigg\}.\nonumber
\end{eqnarray}
For a physically meaningful solution whose ADM mass and angular momenta
are finite for the asymptotically flat case, or in the case of $\Lambda\neq0$
whose Abbott-Deser \cite{AD} charges are finite, this quantity is
expected to be finite and represents the non-stationary part of the
total energy by construction. Observe that while the ADM momentum
($P_{i}=\ointctrclockwiseop_{\partial\Sigma}K_{ij}dS^{j}$) and angular
momenta ($J^{jk}=\ointctrclockwiseop_{\partial\Sigma}(x^{j}K^{km}-x^{k}K^{jm})dS_{m}$)
are linear in the extrinsic curvature given as integrals over the
boundary, $\text{\ensuremath{\mathscr{I}}}({\cal {N}})$ has quadratic,
cubic and quartic terms in the extrinsic curvature in the bulk integral.

Before we lay out the details of the above discussion, let us note
that our final formula (\ref{esasintegral}) can be reduced in various
ways depending on the physical problem or the numerical integration
scheme: for example, one can choose the maximal slicing gauge for
which ${Tr}(\pi)=K=0$. If the problem permits time-symmetric initial
data $\pi^{ij}=K^{ij}=0$, then in this restricted case, $V^{ij}=Q^{ij}=0$,
and the integral (\ref{esasintegral}) reduces to
\begin{eqnarray*}
 &  & \text{\ensuremath{\mathscr{I}}}({\cal {N}})=\intop_{\Sigma}dV\Bigl({}^{\varSigma}R_{ij}^{2}N^{2}+(D_{i}D_{j}N)^{2}-2{}^{\varSigma}R^{ij}ND_{i}D_{j}N+2{}^{\varSigma}RN\triangle N\\
 &  & ~~~~~~~~~~~~~~+4D_{m}D_{i}N_{j}D^{m}D^{(i}N^{j)}+(D-3)\triangle N\triangle N\Bigr).
\end{eqnarray*}
Let us go back to (\ref{bizimdenk}) which was the defining relation
of the invariant and try to write it as a boundary integral over the
boundary of the hypersurface $\Sigma$. Then one has
\begin{equation}
\text{\ensuremath{\mathscr{I}}}({\cal {N}})=\intop_{\Sigma}dV\thinspace\widetilde{{\cal {P}}}^{*}({\cal {N}})\cdot\widetilde{{\cal {P}}}^{*}({\cal {N}})=\intop_{\Sigma}dV\thinspace{\cal {N}}\cdot\widetilde{{\cal {P}}}\circ\widetilde{{\cal {P}}}^{*}({\cal {N}})+\ointop_{\partial\Sigma}dS\thinspace n^{k}B_{k},
\end{equation}
which requires $\widetilde{{\cal {P}}}\circ\widetilde{{\cal {P}}}^{*}({\cal {N}})=0$.
This the approximate KID equation introduced by Dain \cite{Dain}
and $B_{k}$ is the boundary term to be found below. Note that our
bulk integral (\ref{esasintegral}) is more general and does not assume
the existence of approximate symmetries.

\section{{\normalsize{}DETAILS OF THE CONSTRUCTION IN $D$ DIMENSIONS}}

\subsection{Boundary Integral}

The importance of the Einstein constraints (\ref{Einstein_c}) cannot
be overstated: clearly the initial data is not arbitrary, one must
solve these equations to feed the evolution equations; but, as importantly,
the constraints also determine the evolution equations and they are
related to the symmetries of the spacetime in a rather intricate way
as we have seen above. One can consider the constraints (\ref{Einstein_c})
as the kernel of a map $\Phi$
\begin{equation}
\Phi:\mathscr{M}_{2}\times{\cal {S}}_{2}^{*}\rightarrow{\cal {C}}^{*}\times{\cal {X}}^{*},
\end{equation}
where $\text{\ensuremath{\mathscr{M}}}_{2}$ denotes the space of
the Riemannian metrics and ${\cal {S}}_{2}^{*}$ denotes the space
of symmetric rank-2 tensor densities, ${\cal {C}}^{*}$ denotes the
space of scalar function densities and ${\cal {X}}^{*}$ the space
of vector field densities on the hypersurface $\Sigma$. We can express
the constraint map explicitly as
\begin{equation}
\Phi\begin{pmatrix}\gamma_{ij}\\
\pi^{ij}
\end{pmatrix}=\begin{pmatrix}\sqrt{\gamma}\left(2\Lambda-^{\varSigma}R\right)+\gamma^{-1/2}\left(\pi_{ij}^{2}-\frac{\pi^{2}}{D-2}\right)\\
-2\gamma_{ki}D_{j}\pi^{kj}
\end{pmatrix},
\end{equation}
whose linearization can be found to be
\begin{equation}
D\Phi\begin{pmatrix}h_{ij}\\
p^{ij}
\end{pmatrix}=\begin{pmatrix}\sqrt{\gamma}\left(^{\varSigma}R{}^{ij}-D^{i}D^{j}+\gamma^{ij}\triangle\right)h_{ij}\\
\frac{1}{\sqrt{\gamma}}\left(\gamma^{ij}\left(\frac{\pi^{2}}{D-2}-\pi_{ij}^{2}\right)+2\left(\pi^{ik}\pi_{k}^{j}-\frac{\pi^{ij}\pi}{D-2}\right)\right)h_{ij}\\
+\frac{2}{\sqrt{\gamma}}\left(\pi_{ij}-\frac{\pi\gamma_{ij}}{D-2}\right)p^{ij}\\
\\
\left(\pi^{ij}D_{k}-2\delta_{k}^{(i}\pi^{j)l}D_{l}\right)h_{ij}-2\gamma_{k(i}D_{j)}p^{ij}
\end{pmatrix}.
\end{equation}
We can define a $2\times2$ matrix as
\begin{equation}
D\Phi:=\begin{pmatrix}\sqrt{\gamma}\left(^{\varSigma}R{}^{ij}-D^{i}D^{j}+\gamma^{ij}\triangle\right) & \frac{2}{\sqrt{\gamma}}\left(\pi_{ij}-\frac{\pi\gamma_{ij}}{D-2}\right)\\
+\frac{1}{\sqrt{\gamma}}\left(\gamma^{ij}\left(\frac{\pi^{2}}{D-2}-\pi_{ij}^{2}\right)+2\left(\pi^{ik}\pi_{k}^{j}-\frac{\pi^{ij}\pi}{D-2}\right)\right)\\
\\
\pi^{ij}D_{k}-2\delta_{k}^{(i}\pi^{j)l}D_{l} & -2\gamma_{k(i}D_{j)}
\end{pmatrix},
\end{equation}
such that
\begin{equation}
D\Phi\begin{pmatrix}h_{ij}\\
p^{ij}
\end{pmatrix}=D\Phi\circ\begin{pmatrix}h_{ij}\\
p^{ij}
\end{pmatrix}.
\end{equation}
Defining \cite{Bartnik}
\begin{equation}
\widetilde{{\cal {P}}}:=D\Phi\circ\begin{pmatrix}\gamma^{-1/2} & 0\\
0 & -D^{m}
\end{pmatrix},
\end{equation}
one finds
\begin{equation}
\widetilde{{\cal {P}}}:=\begin{pmatrix}^{\varSigma}R{}^{ij}-D^{i}D^{j}+\gamma^{ij}\triangle & \frac{2}{\sqrt{\gamma}}\left(\frac{\pi\gamma_{ij}}{D-2}-\pi_{ij}\right)D^{m}\\
+\frac{1}{\gamma}\left(\gamma^{ij}\left(\frac{\pi^{2}}{D-2}-\pi_{ij}^{2}\right)+2\left(\pi^{ik}\pi_{k}^{j}-\frac{\pi^{ij}\pi}{D-2}\right)\right)\\
\\
\frac{1}{\sqrt{\gamma}}\left(\pi^{ij}D_{k}-2\delta_{k}^{(i}\pi^{j)l}D_{l}\right) & 2\gamma_{k(i}D_{j)}D^{m}
\end{pmatrix},\label{ptildeoperator}
\end{equation}
which is a map as
\begin{equation}
\widetilde{{\cal {P}}}:{\cal {S}}_{2}\times{\cal {S}}_{1,2}\rightarrow{\cal {C}}\times{\cal {X}},
\end{equation}
where ${\cal {S}}_{2}$ denotes the space of covariant rank-2 tensors,
${\cal {S}}_{1,2}$ denotes the space of covariant rank-3 tensors
which are symmetric in last two indices, ${\cal {C}}$ denotes the
space of scalar function and ${\cal {X}}$ the space of vector fields
on the hypersurface $\Sigma$.

The formal adjoint of $\widetilde{{\cal {P}}}$-operator was defined
in (\ref{formaladjoint_ptilde}) via the (\ref{formaladjointoflinearizedmap_pi})
and it is a map of the form
\begin{equation}
\widetilde{{\cal {P}}}^{*}:{\cal {C}}\times{\cal {X}}\rightarrow{\cal {S}}_{2}\times{\cal {S}}_{1,2}.
\end{equation}
 Working out the details, one arrives at
\begin{equation}
\widetilde{{\cal {P}}}^{*}\begin{pmatrix}N\\
N^{k}
\end{pmatrix}=\begin{pmatrix}N^{\varSigma}R{}^{ij}-D^{i}D^{j}N+\gamma^{ij}\triangle N+Q^{ij}\\
D_{m}\left(2D_{(i}N_{j)}+V_{ij}\right)
\end{pmatrix},\label{ptildeadjoint}
\end{equation}
where $V^{ij}$ and $Q^{ij}$ were given (\ref{V_ij},\ref{Q_ij})
respectively. We have used this expression in the previous section
to find the bulk integral of the non-stationary energy. Now let us
use this operator and its adjoint to find an expression on the boundary.
For this purpose we need the following identity:

\begin{equation}
\intop_{\Sigma}dV\thinspace\begin{pmatrix}N\\
N^{k}
\end{pmatrix}\cdot{\cal \widetilde{{\cal {P}}}}\begin{pmatrix}s_{ij}\\
s_{kij}
\end{pmatrix}=\intop_{\Sigma}dV\begin{pmatrix}s_{ij}\\
s_{kij}
\end{pmatrix}\cdot\widetilde{{\cal {P}}}^{*}\begin{pmatrix}N\\
N^{k}
\end{pmatrix}+\ointop_{\partial\Sigma}dS\thinspace n^{k}{\cal {B}}{}_{k},\label{integrall}
\end{equation}
with generic $s_{ij}\in{\cal {S}}_{2}$ and $s_{kij}\in{\cal {S}}_{1,2}$.
After making use of (\ref{ptildeoperator}) and (\ref{ptildeadjoint}),
a slightly cumbersome computation yields the boundary term:
\begin{eqnarray}
 &  & {\cal {B}}{}_{k}=s_{kj}D^{j}N-ND^{j}s_{kj}+ND_{k}s-sD_{k}N+2N^{i}D^{j}s_{jki}-2s_{kij}D^{i}N^{j}\nonumber \\
 &  & ~~~~~~~~~~+\frac{2N}{\sqrt{\gamma}}\left(\frac{\pi}{D-2}s_{kj}\thinspace^{j}-s_{kij}\pi^{ij}\right)+\frac{1}{\sqrt{\gamma}}\left(\pi^{ij}s_{ij}N_{k}-2s_{ij}N^{i}\pi_{k}^{j}\right),\label{B_k}
\end{eqnarray}
where $s=\gamma^{ij}s_{ij}$. Let us now assume a particular $s_{ij}$
and a particular $s_{kij}$ such that
\begin{equation}
\begin{pmatrix}s_{ij}\\
s_{kij}
\end{pmatrix}:=\widetilde{{\cal {P}}}^{*}\begin{pmatrix}N\\
N^{k}
\end{pmatrix},
\end{equation}
which yields
\begin{equation}
{\cal \widetilde{{\cal {P}}}}\begin{pmatrix}s_{ij}\\
s_{kij}
\end{pmatrix}={\cal \widetilde{{\cal {P}}}}\circ\widetilde{{\cal {P}}}^{*}\begin{pmatrix}N\\
N^{k}
\end{pmatrix}.
\end{equation}
Then (\ref{integrall}) becomes
\begin{equation}
\intop_{\Sigma}dV\thinspace\begin{pmatrix}N\\
N^{k}
\end{pmatrix}\cdot{\cal \widetilde{{\cal {P}}}}\circ\widetilde{{\cal {P}}}^{*}\begin{pmatrix}N\\
N^{k}
\end{pmatrix}=\text{\ensuremath{\mathscr{I}}}({\cal {N}})+\ointop_{\partial\Sigma}dS\thinspace n^{k}{\cal {B}}{}_{k},\label{integrallreduced}
\end{equation}
where ${\cal {B}}{}_{k}$ given in (\ref{B_k}) must be evaluated
with
\begin{equation}
s_{ij}=N{}^{\varSigma}R_{ij}-D_{i}D_{j}N+\gamma_{ij}\triangle N+Q_{ij}
\end{equation}
and
\begin{equation}
s_{kij}=D_{k}\left(2D_{(i}N_{j)}+V_{ij}\right).
\end{equation}
Equation (\ref{integrallreduced}) shows that generically $\text{\ensuremath{\mathscr{I}}}({\cal {N}})$
cannot be written as an integral on the boundary of the hypersurface
unless ${\cal \widetilde{{\cal {P}}}}\circ\widetilde{{\cal {P}}}^{*}({\cal {N}})=0$.
In that case, the invariant reduces to
\begin{equation}
\text{\ensuremath{\mathscr{I}}}({\cal {N}})=-\ointop_{\partial\Sigma}dS\thinspace n^{k}{\cal {B}}{}_{k}.
\end{equation}
Explicit computation shows that one has
\begin{eqnarray}
 &  & {\cal {B}}{}_{k}=\frac{N^{2}}{2}D_{k}{}^{\Sigma}R+N{}^{\Sigma}R_{kj}D^{j}N-D_{k}D_{j}ND^{j}N-(D-3)D_{k}N\triangle N+(D-2)ND_{k}\triangle N\nonumber \\
 &  & ~~~~~~~~~~+4N^{i}\triangle D_{(k}N_{i)}-4D_{k}D_{(i}N_{j)}D^{(i}N^{j)}+b_{k},\label{B_k-1}
\end{eqnarray}
where
\begin{eqnarray}
 &  & b_{k}:=Q_{kj}D^{j}N-ND^{j}Q_{kj}+ND_{k}Q-QD_{k}N+2N^{i}\triangle V_{ki}-2D_{k}V_{ij}D^{i}N^{j}\nonumber \\
 &  & ~~~~~~~~~+\frac{1}{\sqrt{\gamma}}\frac{2N\pi}{D-2}\left(2D_{k}D_{i}N^{i}+D_{k}V\right)-\frac{2N\pi^{ij}}{\sqrt{\gamma}}\left(2D_{k}D_{i}N_{j}+D_{k}V_{ij}\right)\nonumber \\
 &  & ~~~~~~~~~+\frac{1}{\sqrt{\gamma}}\left(\pi^{ij}N_{k}-2N^{i}\pi_{k}^{j}\right)\left(N^{\varSigma}R_{ij}-D_{i}D_{j}N+\gamma_{ij}\triangle N+Q_{ij}\right).\label{b_k}
\end{eqnarray}
In the Gaussian normal coordinates the boundary integral reads
\begin{equation}
\text{\ensuremath{\mathscr{I}}}({\cal {N}})=\ointop_{\partial\Sigma}dS\thinspace n^{k}\left((D-\frac{5}{2})D_{k}K_{ij}^{2}+(\frac{7}{2}-D)D_{k}K^{2}+2K^{lj}D_{j}K_{lk}\right).
\end{equation}
Another physically relevant case is the time symmetric asymptotically
flat case for which the boundary integral reduces to
\begin{eqnarray*}
 &  & \text{\ensuremath{\mathscr{I}}}({\cal {N}})=\ointop_{\partial\Sigma}dS\thinspace n^{k}\Bigl(D_{k}D_{j}ND^{j}N+(D-3)D_{k}N\triangle N-(D-2)ND_{k}\triangle N\\
 &  & ~~~~~~~~~~~~-4N^{i}\triangle D_{(k}N_{i)}+4D_{k}D_{(i}N_{j)}D^{(i}N^{j)}\Bigr).
\end{eqnarray*}
In the most general form $N$ and $N^{i}$ should satisfy the fourth
order equations ${\cal \widetilde{{\cal {P}}}}\circ\widetilde{{\cal {P}}}^{*}({\cal {N}})=0$
which explicitly read
\begin{equation}
\widetilde{{\cal {P}}}\text{\textopenbullet}\widetilde{{\cal {P}}}^{*}\begin{pmatrix}N\\
N^{i}
\end{pmatrix}=\begin{pmatrix}(D-2)\triangle\triangle N-{}^{\Sigma}R{}_{ij}D^{i}D^{j}N+N\left(\frac{1}{2}\triangle{}^{\Sigma}R+{}^{\Sigma}R{}_{ij}^{2}\right)\\
+2\,{}^{\Sigma}R\triangle N+\frac{3}{2}D_{i}{}^{\Sigma}RD^{i}N+Y\\
\\
4D^{j}\triangle D_{(k}N_{j)}+Y_{k}
\end{pmatrix}=0,\label{PcompositionPadjointDdimensions}
\end{equation}
where
\begin{eqnarray}
 &  & Y:=^{\Sigma}R{}^{ij}Q_{ij}-D^{i}D^{j}Q_{ij}+\triangle Q+\frac{2}{\sqrt{\gamma}}\left(\frac{\pi\gamma^{ij}}{D-2}-\pi^{ij}\right)\triangle(2D_{i}N_{j}+V_{ij})\label{YY}\\
 &  & ~~~+\left(\frac{2}{\gamma}(\pi^{ik}\pi_{k}^{j}-\frac{\pi\pi^{ij}}{D-2})-\frac{\gamma^{ij}}{\gamma}(\pi_{kl}^{2}-\frac{\pi^{2}}{D-2})\right)\left(N^{\varSigma}R_{ij}-D_{i}D_{j}N+\gamma_{ij}\triangle N+Q_{ij}\right)\nonumber
\end{eqnarray}
and
\begin{equation}
Y_{k}:=\frac{1}{\sqrt{\gamma}}\left(\pi^{ij}D_{k}-2\delta_{k}^{i}\pi^{jl}D_{l}\right)\left(N^{\varSigma}R_{ij}-D_{i}D_{j}N+\gamma_{ij}\triangle N+Q_{ij}\right)+2D^{i}\triangle V_{ik}.
\end{equation}

\subsection{{\normalsize{}{}{}The Approximate KID equation in D dimensions}}

Following the $D=4$ discussion of Dain \cite{Dain} let us now study
the approximate KID equation (\ref{PcompositionPadjointDdimensions})
in $D$ dimensions. It is easy to see that it is a fourth order elliptic
operator for $D>2$. This follows by computing the leading symbol:
for this purpose let us consider the higher order derivative terms
and set $D_{i}=\zeta_{i}$ and $\left|\zeta\right|^{2}=\zeta^{i}\zeta_{i}$.
Using (\ref{PcompositionPadjointDdimensions}), the leading symbol
of operator reads
\begin{equation}
\sigma\left[{\cal \widetilde{{P}}}\text{\textopenbullet}\widetilde{{\cal {P}}}^{*}\right]\left(\zeta\right)\begin{pmatrix}N\\
N_{i}
\end{pmatrix}=\begin{pmatrix}(D-2)\left|\zeta\right|^{4}N\\
4\left|\zeta\right|^{2}\zeta^{j}\zeta_{(k}N_{j)}
\end{pmatrix}.
\end{equation}
For a non-zero covector $\zeta$, if $\sigma$ is an isomorphism (here
a vector bundle isomorphism), then the operator is elliptic. For the
first component, this requires $D\neq2$ and for the second component
contraction with $\zeta^{k}$ yields
\begin{equation}
\left|\zeta\right|^{4}\zeta^{k}N_{k}=0.
\end{equation}
Assuming $D\neq2$ one has $\zeta^{k}N_{k}=0$. Inserting it back
in the second component one obtains
\begin{equation}
\left|\zeta\right|^{4}N_{k}=0,
\end{equation}
so for $\left|\zeta\right|^{2}\neq0$, the leading symbol is injective
and the operator ${\cal \widetilde{{P}}}\text{\textopenbullet}\widetilde{{\cal {P}}}^{*}$
is elliptic for $D>2$.

\subsection{{\normalsize{}{}{}Asymptotically Flat Spaces}}

Consider the initial data set $\left(\Sigma,\gamma_{ij},\pi^{ij}\right)$
for the vacuum Einstein field equations with $n>1$ asymptotically
Euclidean ends: this is to avoid bulk simplicity and allow black holes.
There exists a compact set ${\cal {B}}$ such that ${\Sigma}\setminus{\cal {B}}=\sum_{k=1}^{n}{\Sigma}_{(k)}$
, where ${\Sigma}_{(k)}$, $k=1,...,n$ are open sets diffeomorphic
to the complement of a closed ball in $\mathbb{R}^{D-1}$. Each asymptotic
end ${\Sigma}_{(k)}$ admits asymptotically Cartesian coordinates.
We consider the following decay assumptions, for $D>3$, which are
consistent with finite ADM mass and momenta:

\begin{equation}
\gamma_{ij}=\delta_{ij}+o(\left|x\right|^{(3-D)/2}),\label{decayassumptionh}
\end{equation}
\begin{equation}
\pi^{ij}=o(\left|x\right|^{(1-D)/2}),\label{decayassumptionk}
\end{equation}
where $\delta_{ij}=(++...+)$. Note that $\delta_{ij}=\mathcal{O}(1)$
and beware of the small $o$and and the big $\mathcal{O}$ notation.
One can compute the the following decay behavior for the Christoffel
connection

\begin{equation}
^{\Sigma}\Gamma_{ij}^{k}=o(\left|x\right|^{(1-D)/2}),
\end{equation}
and the curvatures
\begin{equation}
^{\Sigma}R^{k}\thinspace_{lmn}=o(\left|x\right|^{-(1+D)/2}),~~~~~~~{}^{\Sigma}R{}_{ij}=o(\left|x\right|^{-(1+D)/2}),~~~~~~~~{}^{\Sigma}R=o(\left|x\right|^{-(1+D)/2}).
\end{equation}

\subsection{{\normalsize{}{}{}KIDs in $D$ Dimensions}}

Let $\left(\Sigma,\gamma_{ij},\pi^{ij}\right)$ denote a smooth vacuum
initial data set satisfying the decay assumptions (\ref{decayassumptionh},
\ref{decayassumptionk}). Let $N,N^{i}$ be a smooth scalar field
and a vector field on $\Sigma$ satisfying the KID equations. Then
generalizing the $D=4$ result of \cite{BeiChr96}, the behavior of
all the possible solutions were given in \cite{Chrusciel:2005pb}
which we quote here.
\begin{enumerate}
\item There exits an antisymmetric tensor field $\mathscr{\omega}_{\mu\nu}$,
such that
\begin{equation}
N-\omega_{0i}x^{i}=o(\left|x\right|^{(5-D)/2}),~~~~~~~~~~~~N^{i}-\omega^{i}\thinspace_{j}x^{j}=o(\left|x\right|^{(5-D)/2}).
\end{equation}
\item If $\omega_{\mu\nu}=0$, then there exists a vector field ${\cal {U}}^{\mu}$,
such that
\begin{equation}
N-{\cal {U}}^{0}=o(\left|x\right|^{(3-D)/2}),~~~~~~~~~~~~~~~N^{i}-{\cal {U}}^{i}=o(\left|x\right|^{(3-D)/2}).\label{stationaryKIDs}
\end{equation}
\item If $\omega_{\mu\nu}=0={\cal {U}}^{\mu}$ then one has the trivial
solution $N=0=N^{i}$ . Both $\mathscr{\omega}_{\mu\nu}$ and ${\cal {U}}^{\mu}$
are constants in the sense that they are $\mathcal{O}(1)$ whenever
they don't vanish.
\end{enumerate}
Case $1$ above corresponds to the rotational Killing vectors while
case $2$ corresponds to the translational ones we shall employ the
latter.

We explained in section $\text{\mbox{II}}$ that solutions of the
$D\Phi^{*}(N,N^{i})=0$ yield spacetime Killing vectors. It is not
difficult to see that the modified equation $\widetilde{{\cal {P}}}^{*}\left(N,N^{i}\right)=0$
yields only the Killing vectors for the case of translational KIDs
(\ref{decayassumptionh},\ref{decayassumptionk}). Here is the proof:
$\widetilde{{\cal {P}}}^{*}\left(N,N^{i}\right)=0$ implies
\begin{equation}
N^{\Sigma}R{}_{ij}-D_{i}D_{j}N+\gamma_{ij}\triangle N+Q_{ij}=0,
\end{equation}
\begin{equation}
D_{m}\left(2D_{(i}N_{j)}+V_{ij}\right)=0.
\end{equation}
If one assumes $\left(N,N^{i}\right)$ decay as in (\ref{stationaryKIDs})
we have $D_{(i}N_{j)}=o(\left|x\right|^{(1-D)/2})$; and $V_{ij}=o(\left|x\right|^{(1-D)/2})$,
then
\begin{equation}
2D_{(i}N_{j)}+V_{ij}=o(\left|x\right|^{(1-D)/2})
\end{equation}
vanishes at infinity; and since it is covariantly constant, it must
vanish identically
\begin{equation}
2D_{(i}N_{j)}+V_{ij}=0.
\end{equation}
Together with the first component of $\widetilde{{\cal {P}}}^{*}\left(N,N^{i}\right)=0$
we get the formal adjoint of the linearized constraint map, namely
$D\Phi^{*}(N,N^{i})=0$. We can conclude that if $\widetilde{{\cal {P}}}^{*}\left(N,N^{i}\right)=0$
then $\left(N,N^{i}\right)$ solve the KID equations.

\subsection{Approximate KIDs in $D$ dimensions}

Generalizing Dain's $D=4$ result, let us search for translational
solutions of the approximate Killing equation \footnote{We work in a given asymptotic end and not the clutter the notation
we do not denote the corresponding index referring to the asymptotic
end. }
\begin{equation}
\widetilde{{\cal {P}}}\circ\widetilde{{\cal {P}}}^{*}\begin{pmatrix}N\\
N^{i}
\end{pmatrix}=0\label{yenia}
\end{equation}
as a deformation of the KIDs $(X,N^{i})$ in the following form:
\begin{equation}
N=\lambda\varphi+X,~~~~~~~~~~N^{i}=N^{i},\label{ansatz}
\end{equation}
where the function $\varphi$ is to be found, $\lambda$ is a constant.
KIDs decay as
\begin{equation}
X-{\cal {U}}^{0}=o(\left|x\right|^{(3-D)/2}),
\end{equation}
\begin{equation}
N^{i}-{\cal {U}}^{i}=o(\left|x\right|^{(3-D)/2}).
\end{equation}
Inserting the ansatz (\ref{ansatz}) into the approximate KID equation
(\ref{yenia}), one gets
\begin{equation}
\widetilde{{\cal {P}}}\circ\widetilde{{\cal {P}}}^{*}\begin{pmatrix}\varphi\\
0
\end{pmatrix}=-\widetilde{{\cal {P}}}\circ\widetilde{{\cal {P}}}^{*}\begin{pmatrix}X\\
N^{i}
\end{pmatrix}=0,\label{important}
\end{equation}
or more explicitly
\begin{equation}
\widetilde{{\cal {P}}}\circ\widetilde{{\cal {P}}}^{*}\begin{pmatrix}\varphi\\
0
\end{pmatrix}=\begin{pmatrix}(D-2)\triangle\triangle\varphi-{}^{\Sigma}R{}_{ij}D^{i}D^{j}\varphi+\varphi\left(\frac{1}{2}\triangle{}^{\Sigma}R+{}^{\Sigma}R_{ij}^{2}\right)\\
+2^{\Sigma}R\triangle\varphi+\frac{3}{2}D_{i}{}^{\Sigma}RD^{i}\varphi+Y\\
\\
Y_{k}
\end{pmatrix}=0.\label{matrix}
\end{equation}
For such a $\varphi$, the bulk integral (\ref{esasintegral}) becomes

\begin{eqnarray}
 &  & \text{\ensuremath{\mathscr{I}}}({\cal {N}})=\lambda^{2}\intop_{\Sigma}dV\Bigg\{|D_{m}V^{ij}|^{2}+{}^{\varSigma}R_{ij}^{2}\varphi^{2}+(D_{i}D_{j}\varphi)^{2}-2{}^{\varSigma}R^{ij}ND_{i}D_{j}\varphi+2{}^{\varSigma}R\varphi\triangle\varphi\nonumber \\
 &  & ~~~~~~~~~~~~~~~~~+(D-3)\triangle\varphi\triangle\varphi+2Q\triangle\varphi+Q_{ij}^{2}+2{}^{\varSigma}R_{ij}\varphi Q^{ij}-2Q^{ij}D_{i}D_{j}\varphi\Bigr\},\label{eb5}
\end{eqnarray}
where
\begin{equation}
V^{ij}=2\varphi K^{ij},
\end{equation}
and
\begin{equation}
Q^{ij}=2\varphi\left(K_{k}^{i}K^{kj}-KK^{ij}\right)-\varphi\gamma^{ij}\left(K_{kl}^{2}-K^{2}\right).
\end{equation}
The boundary form for the asymptotically flat case follows similarly
\begin{eqnarray}
 &  & \text{\ensuremath{\mathscr{I}}}({\cal {N}})=-\lambda^{2}\ointop_{\partial\Sigma}dS\thinspace n^{k}\Bigl\{-D_{k}D_{j}\varphi D^{j}\varphi-(D-3)D_{k}\varphi\triangle\varphi+(D-2)\varphi D_{k}\triangle\varphi\nonumber \\
 &  & +Q_{kj}D^{j}\varphi-\varphi D^{j}Q_{kj}-2\varphi K^{ij}D_{k}V_{ij}\Bigr\},\label{asdfdsa}
\end{eqnarray}
where we used $K_{kl}^{2}-K^{2}={}^{\Sigma}R=0$ on the boundary.

\section{{\normalsize{}{}{}Conclusions}}

Using the Hamiltonian form of the Einstein evolution equations as
given by Fischer and Marsden \cite{FisMar72}, we constructed an integral
that measures the non-stationary energy contained in a spacelike hypersurface in $D$ dimensional General Relativity with or without a cosmological constant.
This integral was previously studied by Dain \cite{Dain} who used
the Einstein constraints but not the evolution equations. The crucial observation
is the following: the critical points of the first order Hamiltonian form of Einstein equations
correspond to the initial data which possess Killing symmetries, a
result first observed by Moncrief \cite{Moncrief}. Hence, our vantage
point is that the failure of an initial data to possess Killing symmetries
is given by the evolution equations, namely non-vanishing of the time derivatives of the spatial metric and the canonical momenta. Then manipulating the evolution equations,
one arrives at the integral (\ref{esasintegral}). Once an initial
data is given, one can compute this integral, which by construction,
vanishes for stationary spacetimes.

\section{{\normalsize{}{}{}Acknowledgments}}
This work is dedicated to the memory of Rahmi G\"{u}ven (1948-2019) who spent a life in gravity research in a region quite timid about science.

\section{{\normalsize{}{}{}Appendix: ADM Split of Einstein's Equations in $D$ Dimensions}}

For the sake of completeness let us give here the ADM split of Einstein's
equations and all the relevant tensors. Using the $\left(D-1\right)+1$
dimensional decomposition of the metric given as (\ref{ADMdecompositionofmetric})
we have:
\begin{equation}
g_{00}=-N^{2}+N_{i}N^{i},~\ ~~~g_{0i}=N_{i},~~~~\ g_{ij}=\gamma_{ij},
\end{equation}
and
\begin{equation}
g^{00}=-\frac{1}{N^{2}},~~~~g^{0i}=\frac{1}{N^{2}}N^{i},~~~~g^{ij}=\gamma^{ij}-\frac{1}{N^{2}}N^{i}N^{j}.
\end{equation}
Let $\Gamma_{\nu\rho}^{\mu}$ denote the Christoffel symbol of the
$D$ dimensional spacetime
\begin{equation}
\Gamma_{\nu\rho}^{\mu}=\frac{1}{2}g^{\mu\sigma}\left(\partial_{\nu}g_{\rho\sigma}+\partial_{\rho}g_{\nu\sigma}-\partial_{\sigma}g_{\nu\rho}\right)
\end{equation}
and let $^{\Sigma}\Gamma_{ij}^{k}$ denote the Christoffel symbol
of the $D-1$ dimensional hypersurface, which is compatible with the
spatial metric $\gamma_{ij}$ as
\begin{equation}
^{\Sigma}\Gamma_{ij}^{k}=\frac{1}{2}\gamma^{kp}\left(\partial_{i}\gamma_{jp}+\partial_{j}\gamma_{ip}-\partial_{p}\gamma_{ij}\right).\label{Christoffelofhypersurface}
\end{equation}
Then a simple computation shows that
\begin{equation}
\Gamma_{00}^{0}=\frac{1}{N}\left(\dot{N}+N^{k}(\partial_{k}N+N^{i}K_{ik})\right)
\end{equation}
and
\begin{equation}
\Gamma_{0i}^{0}=\frac{1}{N}\left(\partial_{i}N+N^{k}K_{ik}\right),~~~~\Gamma_{ij}^{0}=\frac{1}{N}K_{ij},~~~~\Gamma_{ij}^{k}=^{\Sigma}\Gamma_{ij}^{k}-\frac{N^{k}}{N}K_{ij}
\end{equation}
and
\begin{equation}
\Gamma_{0j}^{i}=-\frac{1}{N}N^{i}\left(\partial_{j}N+K_{kj}N^{k}\right)+NK_{j}\thinspace^{i}+D_{j}N^{i}
\end{equation}
and also
\begin{equation}
\Gamma_{00}^{i}=-\frac{N^{i}}{N}\left(\dot{N}+N^{k}\left(\partial_{k}N+N^{l}K_{kl}\right)\right)+N\left(\partial^{i}N+2N^{k}K_{k}\thinspace^{i}\right)+\dot{N}^{i}+N^{k}D_{k}N^{i}.
\end{equation}

Starting with the definition of the $D$ dimensional Ricci tensor
\begin{equation}
R_{\rho\sigma}=\partial_{\mu}\Gamma_{\rho\sigma}^{\mu}-\partial_{\rho}\Gamma_{\mu\sigma}^{\mu}+\Gamma_{\mu\nu}^{\mu}\Gamma_{\rho\sigma}^{\nu}-\Gamma_{\sigma\nu}^{\mu}\Gamma_{\mu\rho}^{\nu}
\end{equation}
one arrives at

\begin{equation}
R_{ij}={}^{\Sigma}R_{ij}+KK_{ij}-2K_{ik}K_{j}^{k}+\frac{1}{N}\left(\dot{K}_{ij}-N^{k}D_{k}K_{ij}-D_{i}D_{j}N-K_{ki}D_{j}N^{k}-K_{kj}D_{i}N^{k}\right),\label{eq:rij}
\end{equation}
where $^{\Sigma}R_{ij}$ denotes the Ricci tensor of the hypersurface.
The remaining components can also be found to be

\begin{equation}
R_{00}=N^{i}N^{j}R_{ij}-N^{2}K_{ij}K^{ij}+N\left(D_{k}D^{k}N-\dot{K}-N^{k}D_{k}K+2N^{k}D_{m}K_{k}^{m}\right)\label{r00}
\end{equation}
and
\begin{equation}
R_{0i}=N^{j}R_{ij}+N\left(D_{m}K_{i}^{m}-D_{i}K\right).\label{eq:ri0}
\end{equation}
The scalar curvature can be found as

\begin{equation}
R=^{\Sigma}R+K^{2}+K_{ij}K^{ij}+\frac{2}{N}\left(\dot{K}-D_{k}D^{k}N-N^{k}D_{k}K\right)\label{r}
\end{equation}
Under the above splitting the cosmological Einstein equations
\begin{equation}
R_{\mu\nu}-\frac{1}{2}g_{\mu\nu}R+\varLambda g_{\mu\nu}=\kappa T_{\mu\nu}
\end{equation}
split in to constraints and evolution equations in local coordinates.
The momentum constraints read

\begin{equation}
N\left(D_{k}K_{i}^{k}-D_{i}K\right)-\kappa\left(T_{0i}-N^{j}T_{ij}\right)=0,
\end{equation}
via the Hamiltonian constraint becomes

\begin{equation}
N^{2}\left(^{\Sigma}R+K^{2}-K_{ij}^{2}-2\Lambda\right)-2\kappa\left(T_{00}+N^{i}N^{j}T_{ij}-2N^{i}T_{0i}\right)=0.
\end{equation}
On the other hand the evolution equations for the metric and the extrinsic
curvature become
\begin{equation}
\frac{\partial}{\partial t}\gamma_{ij}=2NK_{ij}+D_{i}N_{j}+D_{j}N_{i},
\end{equation}
\begin{equation}
\frac{\partial}{\partial t}K_{ij}=N\left(R_{ij}-{}^{\Sigma}R_{ij}-KK_{ij}+2K_{ik}K_{j}^{k}\right)+\mathscr{L}_{\overrightarrow{N}}K_{ij}+D_{i}D_{j}N,
\end{equation}
where $\text{\ensuremath{\mathscr{L}}}_{\overrightarrow{N}}$ is the
Lie derivative along the shift vector.


\section{Introduction}
Video moment retrieval, a key topic in information retrieval and computer vision, has attracted more and more interests
in recent years~\cite{gao2017tall,hendricks2017localizing}.
As two examples in Figure~\ref{fig:example1} show,
according to a given natural language query, moment retrieval aims to locate the temporal boundary of
the most related moment in the video, which can help us quickly filter out useless contents in the video.
More accurate moment retrieval requires sufficient understanding of both the video and the query,
which makes it a challenging task.
Although recent works~\cite{chen2018temporally,zhang2019cross,zhang2019man} has achieved good results,
they are mostly trained in a fully-supervised setting, which requires the full annotations of temporal boundary
for each video.
However, manually labeling the ground truth temporal boundaries is time-consuming and expensive, requiring a large amount
of human labor.
Moreover, considering an untrimmed video contains multiple consecutive temporal activities, it can be difficult to mark
the boundaries accurately, which produces ambiguity and noise in training data.
Relatively, it is much easier to obtain coarse descriptions of a video without marking the temporal boundaries, such as the captions
of videos in YouTube.
This motivates us to develop a weakly-supervised method for moment retrieval that needs only coarse video-level
annotations for training.
Existing weakly-supervised method in ~\cite{mithun2019weakly} proposes to learn a joint visual-text
embedding, and utilizes the latent alignment produced by
intermediate Text-Guided Attention~(TGA) to localize the target moment.
However, the latent attention weights without extra supervision usually
focus on the most discriminative but small regions~\cite{singh2017hide}
instead of covering complete regions.
To deal with these issues, in this paper,
we devise a novel weakly-supervised Semantic Completion Network~(SCN)
including proposal generation and selection, semantic completion for semantic similarity estimation
and scoring refinement.

\begin{figure}[t]
\centering
\includegraphics[width=1.0\columnwidth]{example1}
\caption{Examples of video moment retrieval: search the temporal boundary
of the most relevant moment in video according to the given natural language query.}
\label{fig:example1}
\end{figure}

Firstly, rather than localizing the most relevant moment relying on the ambiguous attention weights,
we extract the semantically important proposals through a proposal generation module.
Further than method in ~\cite{gao2017tall} that treats each candidate proposal separately,
we leverage the cross-modal fusion representations of video and
query to score all the candidate proposals sampled at different scales
in a single pass, which makes full use of context information for scoring
other proposals.

With a large set of densely sampled proposals, we then devise an algorithm
that considers both exploitation and exploration to select top-K proposals.
Concretely, we first rank all candidate proposals
based on their corresponding confidence scores.
Further than just selecting the proposals with high confidence score based on Non Maximum Suppression~(NMS),
to encourage full exploration,
we select next proposal randomly with a decay possibility,
which is helpful for finding potentially good proposals
and giving more accurate confidence scores for proposals.
At the beginning of training, we tend to select next proposal randomly for exploration.
As the model converges gradually, proposals with high confidence score are chosen more often for exploitation.

To explicitly model the scoring of proposal generation module
rather than rely on attention weights without extra supervision~\cite{mithun2019weakly},
we are supposed to further measure the semantic similarity
between the selected proposals and query for scoring refinement.
Inspired by the success of recent works about masked language model~\cite{devlin2019bert,song2019mass,wang190409408},
we design a novel semantic completion module that predicts the important words~(e.g. noun, verb) that are masked
according to the given visual context.
In detail, by masking the important words in the decoder side, SCN forces the decoder rely on the visual context
to reconstruct the query and the most semantically matching proposal can provide enough information to
predict the key words.
Then with the evaluation results given by semantic completion module,
we compute reward for each proposal based on the reconstruction loss
and formulate a rank loss to encourage the proposal generation module
to give higher confidence score for those proposals with greater rewards.

In total, the main contributions of our work are listed as follows:
\begin{itemize}
    \item We propose a novel weakly-supervised moment retrieval framework
    requiring only coarse annotations for training,
    and experiments on two datasets: ActivityCaptions~\cite{caba2015activitynet}
    and Charades-STA~\cite{gao2017tall} demonstrate the effectiveness of our method.
    \item We build a proposal generation module to score all candidate proposals in a single pass
    and formulate a rank loss for scoring refinement.
    \item We devise an algorithm for top-K proposals selection that encourages both exploitation
    and exploration.
    \item We design a novel semantic completion module that predicts the important words that are masked
    according to the given visual context for semantic similarity estimation.
\end{itemize}

\begin{figure*}[t]
    \centering
    \includegraphics[width=2.0\columnwidth]{framework}
    \caption{
        The Framework of our Semantic Completion Network for Video Moment Retrieval.
        (a) The proposal generation module leverages the cross-modal fusion representations of video and
        query to score all the candidate proposals at each time step,
        and then select the top-K proposals considering both exploitation and exploration.
        (b) The semantic completion module reconstruct the query in which the important words are masked
        according to the visual representations of proposal, compute the rewards based on the reconstruction
        loss, and provide feedbacks to the proposal generation module for scoring refinement.

    }
    \label{fig:framework}
\end{figure*}

\section{Related Work}
In this section, we briefly review some related works on image/video retrieval,
temporal action detection and video moment retrieval.

\noindent\textbf{Image/Video Retrieval:} Image/Video Retrieval aims to select image/video that is most relevant to
the queries from a set of candidate images/videos.
The methods in ~\cite{karpathy2015deep,escorcia2016daps,xu2015jointly,otani2016learning} all propose to learn a joint
visual-semantic space for cross-modal representations. In such space, the similarity of cross-modal representations
reflects the closeness between their original inputs.
In moment retrieval, however, we focus on retrieving a target moment in video based on the given query,
rather than simply selecting a target image/video from pre-defined candidate sets.

\noindent\textbf{Temporal Action Detection:} Temporal Action Detection aims at identifying the temporal boundary
as well as the category for each action instance in untrimmed videos. The approaches of action detection can be also
summarized into supervised settings and weakly-supervised settings.
These methods in ~\cite{shou2016temporal,escorcia2016daps,buch2017sst,shou2017cdc,zhao2017temporal}
are trained in two-stage supervised learning manner, which first generate temporal action proposals
through a proposal network, and then predict the action category for each proposal through a classification network.
In the weakly-supervised settings, however, only the coarse video-level labels instead of
the exact temporal boundary is available.
The UntrimmedNet in ~\cite{wang2017untrimmednets} make use of the principle of multiple instance learning
and the generated attention weights to select proposals that most probably contain action instances.
The method presented in ~\cite{nguyen2018weakly} combines temporal class activation maps and class agnostic attentions
for localizing the boundary of action instances.
Further than action detection that is limited to a pre-defined set of categories,
moment retrieval according to natural language query is much more challenging but general.

\noindent\textbf{Video Moment Retrieval:} Video Moment Retrieval is to address the target moment that is semantically aligned with
the given natural language query.
Prior works~\cite{gao2017tall,hendricks2017localizing,hendricks2018localizing,liu2018cross,chen2018temporally,xu2019multilevel,zhang2019cross,zhang2019man,wang2019language}
mainly focus on localizing the most relevant moment in a fully-supervised settings.
Among them, methods proposed in~\cite{gao2017tall,hendricks2017localizing,hendricks2018localizing} sample candidate moments
by sliding windows with various length, and perform coarse fusion to estimate the correlation between
the queries and moments in a multi-modal space.
Further, the Temporal GroundNet~(TGN)~\cite{chen2018temporally} proposes an interactor to exploit the evolving
fine-grained frame-by-word interactions and simultaneously score a set of candidate moments in one single pass.
The Cross-Modal Interaction Network~(CMIN)~\cite{zhang2019cross} advises a multi-head self-attention mechanism to
capture the long-range dependencies in videos and a syntactic GCN to obtain the fine-grained queries representations.
The Semantic Matching Reinforcement Learning~(SM-RL)~\cite{wang2019language} proposes a recurrent neural network based
reinforcement learning model and introduce mid-level semantic concepts to bridge the semantic gap between visual and semantic
information.

Though those methods achieve good performance, they still suffer from collecting
a large amount of manually labelled temporal annotations.
Some works~\cite{bojanowski2015weakly,duan2018weakly,mithun2019weakly} also study this task
in a weakly-supervised setting.
The method proposed in ~\cite{bojanowski2015weakly} consider the task of aligning a video with a set of temporal ordered
sentences, in which temporal ordering can be seen as additional constraint and supervision.
The method proposed in ~\cite{duan2018weakly} decomposes the problem of
weakly-supervised dense event captioning in videos~(WS-DEC) into a cycle of dual problems:
caption generation and moment retrieval and explores the one-to-one correspondence
between the temporal segment and event caption,
and has a complex training pipeline such as pre-training and alternating training.
The Text-Guided Attention~(TGA)~\cite{mithun2019weakly} proposes to learn a joint visual-semantic representations
and utilizes the attention score as the alignment between video frames and query.

\section{Approach}

\subsection{Problem Formulation}
In this paper, we consider the task of Video Moment Retrieval in a weakly-supervised setting.
Given an untrimmed video ${\bf v} = \{{\bf v}_i\}_{i=1}^{n_v}$
where $n_v$ is the frame number of the video and ${\bf v}_i$ is the $i$-th feature vector,
and a corresponding query ${\bf q} = \{{\bf q}_i\}_{i=1}^{n_q}$
where $n_q$ is the word number of the query and ${\bf q}_i$ is the $i$-th feature vector,
aims to localize the most relevant moment ${\hat \tau} = ({\hat s},~{\hat e})$ during inference,
where ${\hat s},~{\hat e}$ are the indices of start frame and end frame respectively.

\subsection{Proposal Generation Module}
In this section, we introduce the proposal generation module.
As mentioned above, the attention weights usually focus on the most discriminative but small regions,
and thus fails to cover the entire temporal extent of target moment.
As Figure~\ref{fig:framework}(a) shows, instead,
this module scores the candidate proposals according to the cross-modal representations of video
and query.
Moreover, further than these methods in ~\cite{gao2017tall,hendricks2018localizing} that
handle different proposals separately in a sliding window fashion, our method scores all the
candidate moments in a single pass, which makes full use of the context information.

In detail, the feature vector ${\bf q}_i$ of each word
can be extracted using a pre-trained word2vec embedding.
Then we develop a textual encoder ${\bf Enc}_q$ to obtain
the textual representations for the query ${\bf q}$.
After that, we input the textual representations and
the video features ${\bf v}^i$ to the visual decoder ${\bf Dec}_v$ to obtain
the final cross-modal representations ${\bf c} = \{{\bf c}_i\}_{i=1}^{n_v}$ of video and query, given by
\begin{eqnarray}
{\bf c} =  {\bf Dec}_v({\bf v},~{\bf Enc}_q({\bf q})),
\end{eqnarray}

To generate confidence score in a single pass, we first pre-define a set of candidate proposals
at each time step,
denoted by $C_t = \{(t - r_k * n_v,~t)\}_{k=1}^{n_k}$,
where $t - r_k * n_v,~t$ are the start and end boundaries of the $k$-th candidate proposal
at the $t$-th time step, $r_k \in (0, 1)$ is the $k$-th ratio and $n_k$ is the number of candidate proposals.
Note that $r_k$ is a fix ratio for each time step.
Then based on the cross-modal representations ${\bf c}$,
we can simultaneously give the confidence scores for
these proposals at all time steps by a fully connected layer
with sigmoid nonlinearity, denoted by
\begin{eqnarray}
    SC_t = \sigma({\bf W}_s{\bf c}_t + {\bf b}_s),
\end{eqnarray}
where $SC_t \in \mathcal{R}^{n_k}$ represents the vector of confidence scores for the $n_k$ candidate proposals
at the $t$-th time step.

Given the candidate proposals $\{C_t\}_{t=1}^{n_v}$,
we apply the selection algorithm that considers both exploitation and exploration
to select the top-K proposals $G = \{G^{k}\}_{k=1}^{K}$
and give the corresponding confidence scores $S = \{S^{k}\}_{k=1}^{K}$,
where $G^{k} = (s_k,~e_k)$ represents the $k$-th proposal in top-K proposals and
$S^{k}$ is its confidence score.
Concretely, we rank the proposals according to their corresponding confidence scores.
At each step, we choose a proposal randomly with a possibility of $p$ or choose
the proposal with the highest score with a possibility of $1 - p$, and use Non Maximum Suppression~(NMS) to
remove those proposals that have high overlap with the chosen one.
We define the sampling possibility $p$ with a decay function dependent on
the times of parameter updates $n_{update}$, given by
\begin{eqnarray}
    p = \lambda_1 * {\rm exp}(-n_{update} / \lambda_2),
\end{eqnarray}
where $\lambda_1$, $\lambda_2$ are the hyper-parameters to control the decay rate.
As the training proceeds, the possibility of choosing next proposal randomly
decreases gradually.

\subsection{Semantic Completion Module}
In this section, we introduce the semantic completion module to
measure the semantic similarity between proposals and query,
compute rewards
and provide feedbacks to previous module for scoring refinement.
As shown in Figure~\ref{fig:framework}(b),
the important words~(e.g. noun, verb) are masked and predicted
according to the given visual context.
The most semantically matching proposal can
provide enough useful information to predict the key words
and also contains less noise.

First, we extract video features for the $k$-th proposal $G^{k} = (s_k,~e_k)$,
denoted by ${\bf \hat v}^k = \{{\bf v}_{i}\}_{i=s_k}^{e_k}$, and obtain
the visual representations
through the visual encoder ${\bf Enc}_v$.
We denote the original words sequence as ${\bf w} = \{{\bf w}_i\}_{i=1}^{n_q}$,
where ${\bf w}_i$ is the $i$-th word of the query.
Then given the words sequence ${\bf w} $ and a set of masked position $\mathcal{X}$,
we denote ${\bf \hat w}$ as a modified version of ${\bf w}$
where those words ${\bf w}_i$, $i \in \mathcal{X}$ are replaced by a special symbol.
We can extract word features for ${\bf \hat w}$, denoted as ${\bf \hat q} = \{{\bf \hat q}_i\}_{i=1}^{n_q}$.
Next, through a bi-directional textual decoder ${\bf Dec}_q$, we can obtain the final cross-modal semantic
representations ${\bf f}^{k} = \{{\bf f}^{k}_i\}_{i=1}^{n_q}$ for the proposal $G^{k}$, given by
\begin{eqnarray}

    {\bf f}^{k} = {\bf Dec}_q({\bf \hat q},~{\bf Enc}_v({\bf \hat v}^{k})),
\end{eqnarray}

To predict the masked words, we can compute the energy distribution ${\bf e}^{k} = \{{\bf e}^{k}_i\}_{i=1}^{n_q}$
on the vocabulary by a fully connected layer, denoted by
\begin{eqnarray}
    {\bf e}^{k}_i = {\bf W}_v{\bf f}^{k}_i + {\bf b}_v,
\end{eqnarray}
where ${\bf e}^{k}_i \in \mathcal{R}^{n_w}$ is the energy distribution at the $i$-th time step,
$n_w$ is the number of words in the vocabulary.

\subsection{Training of Semantic Completion Network}
In this section, we describe the loss function we optimize to train the Semantic Completion Network.

\noindent\textbf{Reconstruction Loss.}
With the energy distribution ${\bf e}^{k}$ for the proposal $G^k$,
we first adopt a reconstruction loss to train the semantic
completion module and make it able to extract key information from the visual context
to predict the masked words.
Formally, we then compute the negative log-likehood of each masked word and add them up,
denoted by
\begin{eqnarray}
    \mathcal{L}_{rec}^{k}

    = -\sum_{i=1}^{n_q-1}{\rm log}~p({\bf w}_{i+1}|{\bf \hat w}_{1:i}, {\bf \hat v}^{k}) \\
    = -\sum_{i=1}^{n_q-1}{\rm log}~p({\bf w}_{i+1}|{\bf e}^{k}_i),
\end{eqnarray}
where $\mathcal{L}_{rec}^{k}$ represents the reconstruction loss based on the visual context of
the proposal $G^{k}$.

\noindent\textbf{Rank Loss.} As Figure~\ref{fig:framework} shows,
in order to correct the confidence scores given by the proposal generation module,
we further apply a rank loss to train this module.
Note that we correct the confidence scores based on reward rather than one-hot label.
Specifically,
we define the reward $R^{k}$ for the proposal $G^{k}$
with a reward function to encourage
proposals with lower reconstruction loss.
The reward is reduced from one to zero in steps of $1~/~(K - 1)$.

Then the strategy of policy gradient is used to correct the scores.
Note that the confidence scores are normalized by a $softmax$ layer, which is
an extremely important operation to highlight the semantically matching proposals
and weaken the mismatched ones.
The rank loss $\mathcal{L}_{ran}^{k}$ for the proposal $G^{k}$ is computed by
\begin{eqnarray}

    \mathcal{L}_{ran}^{k} = -{R^{k}~{\rm log}~(
        \frac{{\rm exp}(S^{k})}{\sum_{i=1}^{K}{\rm exp}(S^{i})})},
\end{eqnarray}

\noindent\textbf{Multi-Task Loss.} With the reconstruction loss and the rank loss for each proposal, we average losses over all proposals and
compute a multi-task loss to train the semantic complete network in an end-to-end manner, denoted by
\begin{eqnarray}
    \mathcal{L} = \frac{1}{K}\sum_{k=1}^K({\mathcal{L}_{rec}^{k}} + \beta\mathcal{L}_{ran}^{k}),
\end{eqnarray}
where $\beta$ is a hyper-parameter to control the balance of two losses.

\subsection{Network Design}
In this section, we introduce the details of the semantic completion network, including
the components of visual/textual encoder and visual/textual decoder.

\noindent\textbf{Encoder and Decoder.} It has been indicated in ~\cite{tang2018self}
that Transformer~\cite{vaswani2017attention} is a strong feature extractor.
In this paper, we build our visual/textual encoder and visual/textual decoder based on
the bi-directional Transformer, as Figure~\ref{fig:framework}(c)(d) shows.
The encoder/decoder is composed of a stack of layer that contains
the multi-head attention sub-layer and the fully connected feed-forward network.

\noindent\textbf{Parameter Sharing.} We share the parameters between
the visual/textual encoder and visual/textual decoder.
As Figure~\ref{fig:framework}(c)(d) shows,
an encoder
can be also regarded as a decoder
without computing attention from another input of different modality.
Parameter sharing greatly reduces the number of parameters and save memory.
This is also a kind of model-level dual learning~\cite{xia2018model} sharing parameters across tasks,
which promotes knowledge sharing.

\section{Experiments}

\subsection{Datasets}
We perform experiments on two public datasets for video moment retrieval
to evaluate the effectiveness of our SCN method.

\noindent\textbf{ActivityCaptions.}
The ActivityCaptions~\cite{caba2015activitynet} dataset is originally developed for human activity understanding.
This dataset contains 20,000 various untrimmed videos
and each video includes multiple natural language descriptions with temporal annotations.
The released ActivityCaptions dataset comprise
17,031 description-moment pairs for training.
Since the caption annotations of test data of ActivityCaptions are not publically available,
we take the val\_1 as the validation set and val\_2 as test data.
The average length of the description, also regarded as query in moment retrieval,
is 13.16 words, and the average duration of the video is 117.74 seconds.

\noindent\textbf{Charades-STA.}
The Charades-STA dataset is released in ~\cite{gao2017tall} for moment retrieval,
and comprises 12,408 description-moment pairs for training, and 3,720 for testing.
The average length of the query is 8.6 words,
and the average duration of the video is 29.8 seconds.
The Charades dataset, originally introduced in ~\cite{sigurdssonhollywood},
contains only temporal activity annotation and multiple video-level descriptions for each video.
The authors of ~\cite{gao2017tall} design a semi-automatic way
to generate sentence temporal annotations.
First, the video-level descriptions from the original dataset
were split into sub-sentences.
Then, by matching keywords for activity categories,
these sub-sentences are aligned with moments in videos.
The rule-based annotations are ultimately verified by humans.

\subsection{Evaluation Metric}
To evaluate the performance of our SCN method and baselines,
we adopt the evaluation metric proposed by ~\cite{gao2017tall}
to compute ``R@n,~IoU=m''.
Specifically, we compute the percentage of at least one of the top-n
predicted moments having Intersection over Union~(IoU) larger than m,
denoted by $R(n,m) = \frac{1}{n_t}\sum_{i=1}^{n_t}{r(n,m,q_i)}$,
where $q_i$ is the $i$-th query, $n_t$ is the number of testing query,
${r(n,m,q_i)}$ is 1 only if the top-n returned moments about $q_i$
contains at least one that has a temporal IoU $>$ m
and $R(n,m)$ is the overall performance.

\subsection{Implementation Details}
\textbf{Data Preprocessing.} For each video, we pre-extract visual frame-based features
by a publicly available pre-trained 3D-ConvNet model
which has a temporal resolution of 16 frames.
This network was not fine-tuned on our data.
We reduce the dimensionality of the activations from the second fully-connected layer (fc7)
of the network from 4096 to 500 dimensions using PCA.
The C3D features were extracted every 8 frames.
The maximum number of frame is set to 200.

For each description, we split it into words using NLTK and extract word embeddings using
the pretrained Glove~\cite{pennington2014glove} word2vec for each word token.
The maximum description length is set to 20.
We also keep the most common $n_w$ words in training set, resulting in a vocabulary size of 8,000
for ActivityCaptions and 1,111 for Charades-STA.

\noindent\textbf{Model Settings.}
At each time step of video, we score $n_k$ candidate proposals of multiple scales.
We set $n_k$ to 6 with ratios of ~[0.167, 0.333, 0.500, 0.667, 0.834, 1.0] for ActivityCaptions,
and to 4 with ratios of ~[0.167, 0.250, 0.333, 0.500] for Charades-STA.
We then set the decay hyper-parameter $\lambda_1$ to 0.5,
$\lambda_2$ to 2000,
the number of selected proposals $K$ to 4,
the balance hyper-parameter $\beta$ to 0.1.
Also, we mask one-third of words in a sentence and replace with a special token
for semantic completion.
Note that noun and verb are more likely to be masked.
Moreover, for TransformerEncoder as well as TransformerDecoder,
the dimension of hidden state is set to 256
and the number of layers is set to 3.
During training,
we adopt the Adam optimizer with learning rate 0.0002 to
minimize the multi-task loss.
The learning rate increases linearly to the maximum with a warm-up step of 400
and then decreases itself based on
the number of updates~\cite{vaswani2017attention}.

\subsection{Compared Methods}
\textbf{Random.} We simply select a candidate moment randomly.

\noindent\textbf{VSA-RNN and VSA-STV.}~\cite{gao2017tall} This two methods both simply project
the visual feature of all candidate proposals and the textual feature of the query into a common space,
and computes the confidence scores based on cosine similarity.

\noindent\textbf{CTRL.}~\cite{gao2017tall} The CTRL method introduces a cross-modal temporal
localizer to estimate the alignment scores and uses clip location regression to further adjust the
the boundary.

\noindent\textbf{QSPN.}~\cite{xu2019multilevel} The QSPN method devises a multilevel approach
for integrating vision and language features using attention mechanisms, and also leverages
video captioning as an auxiliary task.

\noindent\textbf{WS-DEC.}~\cite{duan2018weakly} The WS-DEC method decomposes the problem of
weakly-supervised dense event captioning in videos into a cycle of dual problems:
caption generation and moment retrieval, and explores the one-to-one correspondence
between the temporal segment and event caption.

\noindent\textbf{TGA.}~\cite{mithun2019weakly} The TGA method proposes a weakly-supervised
joint visual-semantic embedding framework for moment retrieval, and utilizes the latent
alignment for localization during inference.

\subsection{Quantitative Results and Analysis}
The overall performance results of our SCN and baselines on ActivityCaptions and Charades-STA datasets
are presented in Table~\ref{table:activityres} and Table~\ref{table:charadesres} respectively.
We consider the evaluation metric ``R@n,~IoU=m'',
where $n \in \{1,5\}, m \in \{0.1,0.3,0.5\}$ for ActivityCaptions,
and $n \in \{1,5\}, m \in \{0.3,0.5,0.7\}$ for Charades-STA.
By observing the evaluation results, we can discover some facts:

\begin{itemize}
    \item Compared with Random method,
    the overall performance results of SCN have a huge improvements on
    both two datasets, which demonstrates that optimizing the multi-task loss
    instead of explicitly optimizing the localization loss can reach the goal
    of predicting the target moment and also indicates the feasibility of our SCN method.
    \item As the results show, the proposed SCN method outperforms
    the supervised visual-embedding approaches VSA-RNN and VSA-STV significantly,
    and obtains results comparable to the other fully-supervised methods
    on two datasets,
    indicating that even without the full annotations of temporal boundary,
    our SCN method can still
    effectively exploit the alignment relationship between video and query
    and find the most semantically relevant moment.
    \item The coarse methods VSA-RNN and VSA-STV achieve the worst performance on two datasets,
    even compared with the weakly-supervised SCN method,
    demonstrating the key role of visual and textual modeling in moment retrieval and indicating
    the limitation of learning a common visual-semantic space in high-quality retrieval.
    \item Also, compared with the weakly-supervised methods WS-DEC and TGA,
    our method achieves tremendous improvements on both ActivityCaptions and
    Charades-STA datasets.
    These results verify the effectiveness of the proposal generation module,
    the semantic completion module, the algorithm of proposals selection and
    the multi-task loss.
\end{itemize}

\begin{table}[t]
    \centering
    \caption{Performance Evaluation Results on the ActivityCaptions Dataset
    ($n \in \{1,5\}$ and  $m \in \{0.1,0.3,0.5\}$). }
    \label{table:activityres}
    \scalebox{0.7}{
        \begin{tabular}{c|ccc|ccc}
            \hline
            \hline
            \multirow{2}{*}{Method} & \multicolumn{3}{c|}{R@1} & \multicolumn{3}{c}{R@5} \\
                &  IoU=0.1&   IoU=0.3 &  IoU=0.5 & IoU=0.1&   IoU=0.3 &  IoU=0.5 \\
            \hline
                Random&38.23&18.64&7.63
                      &75.74&52.78&29.49\\
            \hline
                VSA-RNN&-&39.28&23.43
                       &-&70.84&55.52\\
                VSA-STV&-&41.71&24.01
                       &-&71.05&56.62\\
                CTRL&-&47.43&29.01
                    &-&75.32&59.17\\
                QSPN&-&52.12&33.26
                    &-&77.72&62.39\\
            \hline
                WS-DEC&62.71&41.98&23.34
                      &-&-&-\\
                SCN&{\bf 71.48}&{\bf 47.23}&{\bf 29.22}
                   &{\bf 90.88}&{\bf 71.45}&{\bf 55.69}\\
            \hline
        \end{tabular}
    }
\end{table}

\begin{table}[t]
    \centering
    \caption{Performance Evaluation Results on the Charades-STA Dataset
    ($n \in \{1,5\}$ and  $m \in \{0.3,0.5,0.7\}$). }
    \label{table:charadesres}
    \scalebox{0.7}{
        \begin{tabular}{c|ccc|ccc}
            \hline
            \hline
            \multirow{2}{*}{Method} & \multicolumn{3}{c|}{R@1} & \multicolumn{3}{c}{R@5} \\
                &  IoU=0.3&   IoU=0.5 &  IoU=0.7 & IoU=0.3&   IoU=0.5 &  IoU=0.7\\
            \hline
                Random&20.12&8.61&3.39
                      &68.42&37.57&14.98\\
            \hline
                VSA-RNN&-&10.50&4.32
                       &-&48.43&20.21\\
                VSA-STV&-&16.91&5.81
                       &-&53.89&23.58\\
                CTRL&-&23.63&8.89
                    &-&58.92&29.52\\
                QSPN&54.70&35.60&15.80
                    &95.60&79.40&45.40\\
            \hline
                TGA&32.14&19.94&8.84
                   &86.58&65.52&33.51\\
                SCN&{\bf 42.96}&{\bf 23.58}&{\bf 9.97}
                   &{\bf 95.56}&{\bf 71.80}&{\bf 38.87}\\
            \hline
        \end{tabular}
    }
\end{table}

\begin{figure}[t]
    \centering
    \subfigure[ActivityCaptions]{
        \includegraphics[width=0.45\columnwidth]{act_epoch-eps-converted-to}
    }
    \subfigure[Charades-STA]{
        \includegraphics[width=0.45\columnwidth]{cha_epoch-eps-converted-to}
    }

    \caption{Training Process of Different Models
    on ActivityCaptions and Charades-STA Datasets}
    \label{fig:ab_compare1}
\end{figure}

\begin{figure}[t]
    \centering
    \subfigure[ActivityCaptions]{
        \includegraphics[width=0.45\columnwidth]{act_ab-eps-converted-to}
    }
    \subfigure[Charades-STA]{
        \includegraphics[width=0.45\columnwidth]{./cha_ab-eps-converted-to}
    }

    \caption{Evaluation Results of Different Models
    on ActivityCaptions and Charades-STA Datasets}
    \label{fig:ab_compare2}
\end{figure}

\begin{figure*}[t]
    \centering
    \includegraphics[width=1.0\columnwidth]{example_act1}
    \includegraphics[width=1.0\columnwidth]{example_act2}
    \includegraphics[width=1.0\columnwidth]{example_act3}
    \includegraphics[width=1.0\columnwidth]{example_act4}
    \caption{Qualitative Examples on the ActivityCaptions dataset}
    \label{fig:example_act}
\end{figure*}

\begin{figure*}[!ht]
    \centering
    \includegraphics[width=1.0\columnwidth]{example_cha1}

    \includegraphics[width=1.0\columnwidth]{example_cha4}
    \caption{Qualitative Examples on the Charades-STA dataset}
    \label{fig:example_cha}
\end{figure*}

\subsection{Ablation Study}
To prove the validity of different parts of our method,
we simplify the algorithm to generate different ablation models as follows:
\begin{itemize}
    \item \textbf{SCN(w/o. rand).} During proposals selection,
    we assign the sample possibility $p$ to zero,
    which means we select next proposal
    completely based on the confidence scores without random selection
    at each step.
    \item \textbf{SCN(w/o. reward).} With feedbacks given by the semantic completion module,
    we modify the rank loss by using one-hot label instead of computing rewards for scoring refinement.
    Concretely, we simply assign a reward of one to the best proposal, and zero to the other ones.
    The rank loss is equivalent to the cross entropy loss.
    \item \textbf{SCN(w/o. mask).} To validate the effectiveness of the semantic completion module
    and the reconstruction loss, we replace this module with a ordinary captioning generator~\cite{duan2018weakly}
    without masking words.
    \item \textbf{SCN(w/o. share).} Instead of parameter sharing between the proposal generation module
    and the semantic completion module, we use two separate sets of parameters for this two modules.
\end{itemize}
The training process of different models on ActivityCaptions and Charades-STA
is presented in Figure~\ref{fig:ab_compare1}.
By analyzing the results, we can find some interesting points:
\begin{itemize}
    \item The simplified models SCN(w/o. rand) and SCN(w/o. reward) still
    achieve results comparable to the fully-supervised methods and outperform
    the existing weakly-supervised methods, which further demonstrates
    the effectiveness of our framework
    including proposal generation and selection,
    semantic completion for semantic similarity estimation and scoring refinement.
    \item The SCN(full) achieves better results than the SCN(w/o. rand) and
    the evaluation results of SCN(w/o. rand) grows more gently as the model converges,
    which proves the ability of the random selection to find potentially good proposals
    during proposals selection.
    When the model has not converged, selecting proposals randomly provide opportunities for
    those potentially good proposals and speed up training.
    \item The SCN(full) achieves the best results faster than SCN(w/o. reward), which demonstrates
    the effectiveness of employing rewards as feedbacks to train the proposal generation module.
    In the early of training stage, the semantic module can't provide accurate feedbacks
    but the one-hot label force the previous module to accept only one proposal
    and reject other proposals that are actually reasonable.
    \item As shown in ~\ref{fig:ab_compare2}, the SCN(full) achieves better
    results than the SCN(w/o. mask), which indicating the effectiveness of
    the semantic completion module and the masking operation.
    By masking the important words of the query, we forces the decoder to absorb the cross-modal
    visual information in the decoder side.
    \item The SCN(full) also perform a bit better than the SCN(w/o. share), demonstrating
    the effectiveness of parameter sharing to promote knowledge sharing.
    Also, The amount of parameters is greatly reduced by parameter sharing.
\end{itemize}

\subsection{Qualitative Results}
To qualitatively validate the performance of our SCN method,
several examples of video moment retrieval from ActivityCaptions and Charades-STA
are provided in Figure~\ref{fig:example_act} and Figure~\ref{fig:example_cha} respectively.
Each example provide the ground truth of temporal boundaries,
the first two proposals with the highest confidence score given
by the proposal generation module.
The bold words in the sentence are considered as the important words
associated with the video context, and are masked for semantic completion.
The corresponding reconstruction loss $\mathcal{L}_{rec}$ is also computed
and presented in each example.

It can be observed that both the first two proposals with the highest confidence score
cover the most discriminative video contents relevant to the query,
which qualitatively verify that the proposal generation module
can locate those semantically important proposals,
and the rank loss is helpful for scoring refinement during training.
Additionally, the proposal with higher IoU has lower reconstruction loss,
also indicating the proposal that is more semantically matching with the query
can be recognized by the semantic completion module.
Therefore, due to the effectiveness of two submodules and the training algorithm, our method is
successful in localizing the moment that has high IoU with the target moment.

\section{Conclusion}
In this paper, we study the task of video moment retrieval from the perspective
of weak-supervised learning
without manually-labelled temporal boundaries of start time and end time,
which makes this task more realistic but more challenging.
We propose a novel semantic completion network~(SCN)
including the proposal generation module to score all candidate proposals in a single pass,
an efficient algorithm for proposals selection considering both exploitation and exploration,
the semantic completion module for semantic similarity estimation
and a multi-task loss for training.
The experiments on the ActivityCaptions and Charades-STA datasets also demonstrate
the effectiveness of our method to exploit the alignment relationship between
video and query, and the efficiency of the proposal selection algorithm
and the rank loss.

\section{Acknowledgements}
This work is supported by the National Natural Science Foundation of China under
Grant No.61602405, No.U1611461, No.61751209 and No.61836002,
China Knowledge Centre for Engineering Sciences and Technology,
and Alibaba Innovative Research.