\section{Introduction}

The basic linear operator defined on real- or complex-valued functions on a non-Archimedean local field, for example the field $\Qp$ of $p$-adic numbers, is the Vladimirov pseudo-differential operator $\D$ of fractional differentiation; see \cite{K2001,VVZ}; for further development of this and related subjects see \cite{AKS,KKZ,ZG}.

It was found in \cite{K2014} that properties of $\D$ become much simpler on radial functions. For this case, it resembles the Caputo-Djrbashian fractional differentiation of real analysis. In particular, this operator possesses a right inverse, which can be seen as a $p$-adic counterpart of the Riemann-Liouville fractional integral or, for $\alpha =1$, the classical anti-derivative. As an application, in \cite{K2014} we considered linear equations for radial functions, in which the restriction of $\D$ plays the role of a derivative.

In this paper, we extend the above framework to nonlinear equations. We consider equations of the form
\begin{equation}
\label{1.1}
\left( \D u\right) (|t|_K) =f(|t|_K,u(|t|_K)),\quad 0\ne t\in K,
\end{equation}
with the initial condition
\begin{equation}
\label{1.2}
u(0)=u_0
\end{equation}
where $K$ is non-Archimedean local field with the normalized absolute value $|\cdot |_K$ (see section 2 for the definitions).

Under certain Lipschitz type condition, we prove the existence and uniqueness of a local solution of the Cauchy problem (\ref{1.1})-(\ref{1.2}) defined on a sufficiently small ``interval'' of $t$. While this property is similar to classical ones, our analytic technique is different, due to the different nature of our $p$-adic fractional integral. The analysis of the structure of (\ref{1.1}), that is of the properties of $\D$ on radial functions, then shows that under some assumptions regarding the function $f$, the local solution is extended to a global one defined on the whole field $K$. While for linear equations \cite{K2014} we used the theory of compact operators, for nonlinear equations studied in this paper, the above results are obtained by direct constructions based on iteration processes.

\section{Preliminaries}

{\bf 2.1. Local fields.} Let $K$ be a non-Archimedean local field,
that is a non-discrete totally disconnected locally compact
topological field. It is well known that $K$ is isomorphic either
to a finite extension of the field $\mathbb Q_p$ of $p$-adic
numbers (if $K$ has characteristic 0), or to the field of formal
Laurent series with coefficients from a finite field, if $K$ has
a positive characteristic. For a summary of main notions and results
regarding local fields see, for example, \cite{K2001}.

Any local field $K$ is endowed with an absolute value $|\cdot |_K$,
such that $|x|_K=0$ if and only if $x=0$, $|xy|_K=|x|_K\cdot |y|_K$,
$|x+y|_K\le \max (|x|_K,|y|_K)$. Denote $O=\{ x\in K:\ |x|_K\le 1\}$,
$P=\{ x\in K:\ |x|_K<1\}$. $O$ is a subring of $K$, and $P$ is an ideal
in $O$ containing such an element $\beta$ that
$P=\beta O$. The quotient ring $O/P$ is
actually a finite field; denote by $q$ its cardinality. We will
always assume that the absolute value is
normalized, that is $|\beta |_K=q^{-1}$. The normalized absolute
value takes the values $q^N$, $N\in \mathbb Z$. Note that for $K=\mathbb Q_p$
we have $\beta =p$ and $q=p$; the $p$-adic absolute value is normalized.

The additive group of any local field is self-dual, that is if
$\chi$ is a fixed non-constant complex-valued additive character of
$K$, then any other additive character can be written as
$\chi_a(x)=\chi (ax)$, $x\in K$, for some $a\in K$. Below we assume that $\chi$ is a rank
zero character, that is $\chi (x)\equiv 1$ for $x\in O$, while
there exists such an element $x_0\in K$ that $|x_0|_K=q$ and $\chi
(x_0)\ne 1$.

The above duality is used in the definition of the Fourier
transform over $K$. Denoting by $dx$ the Haar measure on the
additive group of $K$ (normalized in such a way that the measure
of $O$ equals 1) we write
$$
\widetilde{f}(\xi )=\int\limits_K\chi (x\xi )f(x)\,dx,\quad \xi
\in K,
$$
where $f$ is a complex-valued function from $L_1(K)$. As usual, the Fourier
transform $\mathcal F$ can be extended from $L_1(K)\cap L_2(K)$ to a
unitary operator on $L_2(K)$. If $\mathcal F f=\widetilde{f}\in L_1(K)$, we
have the inversion formula
$$
f(x)=\int\limits_K\chi (-x\xi )\widetilde{f}(\xi )\,d\xi .
$$

Working with functions on $K$ and operators upon them we often use standard integration formulas; see \cite{K2001,VVZ}. The simplest of them are as follows:

\begin{equation*}
\int\limits_{|x|_K\le q^n}dx=q^n;\quad \int\limits_{|x|_K=q^n}dx=\left( 1-\frac1q \right)q^n.
\end{equation*}

\begin{equation*}
\int\limits_{|x|_K\le q^n}|x|_K^{\alpha -1}\,dx=\frac{1-q^{-1}}{1-q^{-\alpha }}q^{\alpha n};\quad \text{here and above $n\in \mathbb Z,\alpha >0$}.
\end{equation*}

A function $f:\ K\to \mathbb C$ is said to be locally constant, if there exists such an integer $l$ that for any $x\in K$
$$
f(x+x')=f(x), \quad \text{whenever $|x'|\le q^{-l}$}.
$$
The smallest number $l$ with this property is called the exponent
of local constancy of the function $f$.

Let $\mathcal D(K)$ be the set of all locally constant
functions with compact supports; it is a vector space
over $\mathbb C$ with the topology of double inductive limit
$$
\mathcal D(K)=\varinjlim_{N\to \infty }\varinjlim_{l\to \infty }\mathcal D_N^l
$$
where $\mathcal D_N^l$ is the finite-dimensional space of functions supported in the ball
$B_N=\big\{ x\in K:$ $|x|\le q^N\big\}$ and having the exponents of local constancy $\le l$.
The strong conjugate space $\mathcal D'(K)$ is called the space of
Bruhat-Schwartz distributions.

The Fourier transform preserves the space $\mathcal D(K)$. Therefore the Fourier transform of a distribution defined
by duality acts continuously on $\mathcal D'(K)$. As in the case of $\mathbb R^n$, there exists a
well-developed theory of distributions over local fields; it includes
such topics as convolution, direct product, homogeneous distributions etc (see \cite{AKS,K2001,VVZ}).

\medskip
{\bf 2.2. Vladimirov's operator}. On a test function $\varphi \in \mathcal D(K)$, the fractional differentiation operator $D^\alpha$, $\alpha >0$, is defined as
\begin{equation}
\label{2.1}
\left( D^\alpha \varphi \right) (x)=\mathcal F^{-1}\left[ |\xi |_K^\alpha
(\mathcal F (\varphi ))(\xi )\right] (x).
\end{equation}
Note that $\D$ does not preserve $\mathcal D(K)$; see \cite{AKS} regarding the spaces of test functions and distributions preserved by this operator.

The operator $D^\alpha$ can also be represented as a hypersingular integral operator:
\begin{equation}
\label{2.2}
\left( D^\alpha \varphi \right) (x)=\frac{1-q^\alpha }{1-q^{-\alpha
-1}}\int\limits_K |y|_K^{-\alpha -1}[\varphi (x-y)-\varphi (x)]\,dy
\end{equation}
\cite{K2001,VVZ}.
In contrast to (\ref{2.1}), the expression in the right of (\ref{2.2}) makes sense for wider classes of functions. In particular, $\D$ is defined on constant functions and annihilates them. Denote for brevity $\theta_\alpha =\dfrac{1-q^\alpha }{1-q^{-\alpha -1}}$.

Below we consider the operator $\D$ on a radial function $u=u(|x|_K)$; here we identify the function $x\mapsto u(|x|_K)$ on $K$ with the function $|x|_K\mapsto u(|x|_K)$ on $q^{\mathbb Z}$. This abuse of notation does not lead to confusion.

The explicit expression of $\D u$ for a radial function $u$ satisfying some growth restrictions near the origin and infinity was found in \cite{K2014}. If $u=u(|x|_K)$ is such that
\begin{equation}
\label{2.3}
\sum\limits_{k=-\infty}^m q^k\left| u(q^k)\right| <\infty ,\quad \sum\limits_{l=m}^\infty q^{-\alpha l}\left| u(q^l)\right| <\infty,
\end{equation}
for some $m\in \mathbb Z$, then for each $n\in \mathbb Z$ the expression in the right-hand side of (\ref{2.2}) with $\varphi (x)=u(|x|_K)$ exists for $|x|_K=q^n$, depends only on $|x|_K$, and
\begin{multline}
\label{2.4}
(D^\alpha u)(q^n)=\theta_\alpha \left(1-\frac1q \right)q^{-(\alpha +1)n}\sum\limits_{k=-\infty}^{n-1} q^ku(q^k) +q^{-\alpha n-1}\frac{q^\alpha +q-2}{1-q^{-\alpha -1}}u(q^n)\\
+\theta_\alpha \left(1-\frac1q \right)\sum\limits_{l=n+1}^\infty q^{-\alpha l}u(q^l).
\end{multline}

Under the conditions (\ref{2.3}), the expression (\ref{2.4}) agrees also with the definition of $\D$ in terms of Bruhat-Schwartz distributions (see Chapter 2 of \cite{VVZ}).

\medskip
{\bf 2.3. The regularized integral}. The fractional integral mentioned in Introduction, was defined in \cite{K2014} initially for $\varphi \in \mathcal D(K)$ as follows:
\begin{equation}
\label{2.5}
(I^\alpha \varphi )(x)=\frac{1-q^{-\alpha}}{1-q^{\alpha -1}}\int\limits_{|y|_K\le |x|_K}\left( |x-y|_K^{\alpha -1}-|y|_K^{\alpha -1}\right) \varphi (y)\,dy,\quad \alpha \ne 1,
\end{equation}
and
\begin{equation}
\label{2.6}
(I^1\varphi )(x)=\frac{1-q}{q\log q}\int\limits_{|y|_K\le |x|_K}\left( \log |x-y|_K-\log |y|_K\right) \varphi (y)\,dy.
\end{equation}
Note that the integrals are taken, for each fixed $x\in K$, over bounded sets, and $(I^\alpha \varphi )(0)=0$. These properties are different from those of the anti-derivatives $D^{-\alpha }$ studied in \cite{VVZ}.

Let $u=u(|x|_K)$ be a radial function, such that
$$
\sum\limits_{k=-\infty}^m \max \left( q^k,q^{\alpha k}\right) \left| u(q^k)\right| <\infty ,\quad \text{if $\alpha \ne 1$},
$$
and
$$
\sum\limits_{k=-\infty}^m |k|q^k \left| u(q^k)\right| <\infty ,\quad \text{if $\alpha =1$},
$$
for some $m\in \mathbb Z$. Then \cite{K2014} $I^\alpha u$ exists, it is a radial function, and for any $x\ne 0$,
\begin{equation}
\label{2.7}
(I^\alpha u)(|x|_K)=q^{-\alpha}|x|_K^\alpha u(|x|_K)+\frac{1-q^{-\alpha}}{1-q^{\alpha -1}}\int\limits_{|y|_K< |x|_K}\left( |x|_K^{\alpha -1}-|y|_K^{\alpha -1}\right) u(|y|_K)\,dy,\quad \alpha \ne 1,
\end{equation}
and
\begin{equation}
\label{2.8}
(I^1 u)(|x|_K)=q^{-1}|x|_K u(|x|_K)+\frac{1-q}{q\log q}\int\limits_{|y|_K<|x|_K}\left( \log |x|_K-\log |y|_K\right) u(|y|_K)\,dy.
\end{equation}

\medskip
Another result from \cite{K2014} shows that $\II$ is indeed a right inverse of $\D$ on an appropriate class of radial functions. Namely,
suppose that for some $m\in \mathbb Z$,
\begin{equation}
\label{2.9}
\sum\limits_{k=-\infty}^m \max \left( q^k,q^{\alpha k}\right) \left| v(q^k)\right| <\infty ,\quad \sum\limits_{l=m}^\infty \left| v(q^l)\right| <\infty ,
\end{equation}
if $\alpha \ne 1$, and
\begin{equation}
\label{2.10}
\sum\limits_{k=-\infty}^m |k|q^k \left| v(q^k)\right| <\infty ,\quad
\sum\limits_{l=m}^\infty l\left| v(q^l)\right| <\infty ,
\end{equation}
if $\alpha =1$. Then there exists $\left( D^\alpha I^\alpha v\right) (|x|_K)=v(|x|_K)$ for any $x\ne 0$.

Note that the decay conditions at infinity in (\ref{2.9}) and (\ref{2.10}) cannot be dropped. This follows from the important identity
\begin{equation}
\label{2.11}
\II 1=0
\end{equation}
proved in \cite{K2014}.

Let us prove that under some decay conditions near the origin and infinity $\II$ is also a left inverse of $\D$. First we need an estimate for $\D u$. Below $C$ will denote various positive constants.

\medskip
\begin{prop}
Suppose that $u(0)=0$,
\begin{equation}
\label{2.12}
|u(q^n)|\le Cq^{dn},\quad n\le 0;
\end{equation}
\begin{equation}
\label{2.13}
|u(q^n)|\le Cq^{hn},\quad n\ge 0,
\end{equation}
where $d>\max (0,\alpha -1)$, $0\le h<\alpha$, and $h<\alpha -1$, if $\alpha >1$. Then the function $w=\D u$ satisfies, for any $m\in \mathbb Z$, the inequalities
\begin{equation}
\label{2.14}
\sum\limits_{k=-\infty}^m \max (q^k,q^{\alpha k})\left| w(q^k)\right| <\infty,\quad \sum\limits_{l=m}^\infty \left| w(q^l)\right| <\infty,
\end{equation}
if $\alpha \ne 1$, or
\begin{equation}
\label{2.15}
\sum\limits_{k=-\infty}^m |k|\cdot q^k\left| w(q^k)\right| <\infty,\quad \sum\limits_{l=m}^\infty |l|\cdot \left| w(q^l)\right| <\infty,
\end{equation}
if $\alpha =1$. Moreover,
\begin{equation}
\label{2.16}
\II \D u=u.
\end{equation}
\end{prop}

\medskip
{\it Proof}. Let $\alpha \ne 1$. Under (\ref{2.12}) and (\ref{2.13}), the conditions (\ref{2.3}) are satisfied, and we may use the expression (\ref{2.4}). Denote the three terms in (\ref{2.4}) by $S_1^{(n)}$, $S_2^{(n)}$ and $S_3^{(n)}$ respectively. We have for $n\ge 1$ that
$$
\sum\limits_{k=-\infty}^{n-1} q^k\left| u(q^k)\right|\le C\sum\limits_{k=-\infty}^0q^{(d+1)k}+C\sum\limits_{k=1}^{n-1}q^{(h+1)k}\le C_1q^{(h+1)n},
$$
so that $\left| S_1^{(n)}\right| \le Cq^{(h-\alpha )n}$ and $\sum\limits_{n=m}^\infty \left| S_1^{(n)}\right| < \infty$.

If $n\le 0$, then $\left| S_1^{(n)}\right| \le Cq^{(d-\alpha )n}$, whence
$$
\sum\limits_{k=-\infty}^m q^{\alpha k}\left| S_1^{(k)}\right| \le C\sum\limits_{k=-\infty}^m q^{dk}<\infty .
$$

Similar estimates for $S_2^{(k)}$ are obvious.

For $n\ge 1$,
$$
\left| S_3^{(n)}\right| \le \sum\limits_{l=n+1}^\infty q^{(h-\alpha )l}=C_2q^{(h-\alpha )n},
$$
so that $\sum\limits_{n=m}^\infty  \left| S_3^{(n)}\right|<\infty$. Next, let $0<\alpha <1$, so that $\max (q^k,q^{\alpha k})=q^{\alpha k}$ for $k\le 0$. We have
\begin{multline*}
\sum\limits_{n=-\infty}^{-2}q^{\alpha n}\sum\limits_{l=n+1}^\infty q^{-\alpha l}\left| u(q^l)\right| =\sum\limits_{n=-\infty}^{-2}q^{\alpha n}\left( \sum\limits_{l=n+1}^{-1}+\sum\limits_{l=0}^\infty \right) q^{-\alpha l}\left| u(q^l)\right| \\
\le C\sum\limits_{n=-\infty}^{-2}q^{\alpha n}\left( \sum\limits_{l=n+1}^{-1}q^{(d-\alpha )l}+\sum\limits_{l=0}^\infty q^{(h-\alpha )l}\right) =C\sum\limits_{l=-\infty}^{-1}q^{(d-\alpha )l}\sum\limits_{n=-\infty}^{l-1}q^{\alpha n}+C_3\sum\limits_{n=-\infty}^{-2}q^{\alpha n}\\
=C_4\sum\limits_{l=-\infty}^{-1}q^{dl}+C_3\sum\limits_{n=-\infty}^{-2}q^{\alpha n}<\infty.
\end{multline*}
Therefore
$$
\sum\limits_{k=-\infty}^m q^{\alpha k}\left| S_3^{(k)}\right| \le \infty,
$$
which implies the first inequality in (\ref{2.14}).

If $\alpha >1$, then $\max (q^k,q^{\alpha k})=q^k$ for $k\le 0$, and we perform similar calculations assuming that $d>\alpha -1$.

Finally, the proofs of the inequalities (\ref{2.15}) are similar to the above ones, since the factor $|k|$ does not influence the convergence of series with estimates exponential in $k$.

Let us prove (\ref{2.16}). Denote $v=\II \D u$. This is a legitimate object, since $\D u$ satisfies the conditions, under which $\II$ can be applied. Moreover, $\D v=\D \II (\D u)=\D u$, with a possible exception of the origin. Therefore $\D (v-u)$ is concentrated at the origin, which means (see e.g. Theorem 1.9 from \cite{K2001} or Section 6.3 in \cite{VVZ}) that $\D (v-u)=a\delta$, $a\in \mathbb R$. By Theorem 1, Section 9.3 of \cite{VVZ},
\begin{equation}
\label{2.17}
v-u=aD^{-\alpha}\delta +c
\end{equation}
where $c\in \RR$ and (\cite{AKS}, Example 9.2.1)
$$
\left( D^{-\alpha}\delta \right) (x)=\frac{|x|_K^{\alpha -1}}{\Gamma_K(\alpha )},\quad \Gamma_K(\alpha )=\frac{1-q^{\alpha -1}}{1-q^{-\alpha}},\text{ if $\alpha \ne 1$},
$$
$$
\left( D^{-1}\delta \right) (x)=\frac{q-1}{q\log q}\log |x|_K, \text{ if $\alpha =1$}.
$$
We need to prove that $a=c=0$.

Let $0<\alpha <1$. We saw above that $\left| S_1^{(n)}\right|, \left| S_2^{(n)}\right| \le Cq^{(d-\alpha )n}$, $n\le 0$. We also have that
$$
\left| S_3^{(n)}\right| \le C\sum\limits_{l=n+1}^\infty q^{-\alpha l}\left| u(q^l)\right| \le C_5+C_6\sum\limits_{l=n+1}^0q^{(d-\alpha )l}\le C_7q^{\min(0,d-\alpha )\cdot n},
$$
so that
\begin{equation}
\label{2.18}
\left| w(q^n)\right| \le Cq^{\min (0,d-\alpha )\cdot n},\quad n\le 0.
\end{equation}

Substituting (\ref{2.18}) into the expression (\ref{2.7}) for $\II w$ we find by a straightforward calculation that
\begin{equation}
\label{2.19}
\left| (\II w) (|x|_K)\right| \le C|x|_K^{\min (d,\alpha )} \to 0,\quad \text{as $x\to 0$}.
\end{equation}
Now it suffices to compare the assumption (\ref{2.12}), the estimate (\ref{2.19}) for $v=\II w$, and the relation (\ref{2.17}), to come to the conclusion that $a=c=0$, so that $v=u$.

The same argument works for $\alpha =1$. In this case the inequality (\ref{2.18}) remains valid, so that again $(I^1w)(q^n)\to 0$, as $n\to -\infty$, while $\log q^n\to -\infty$, as $n\to -\infty$.

Let $\alpha >1$. It follows from the above estimates of $S_1^{(n)},S_2^{(n)},S_3^{(n)}$ for $n\ge 1$ that
$$
\left| \left( \D u\right) (|x|_K)\right| \le C|x|_K^{h-\alpha},\quad |x|_K\ge 1.
$$
This implies the inequality
$$
|v(|x|_K)|\le C|x|_K^h, \quad |x|_K\ge 1
$$
(see the remark after Lemma 2 in \cite{K2014}). Comparing this with (\ref{2.17}) and our assumption (\ref{2.13}) we find that $a=0$. Then we consider as above the behavior of $u,v$ near the origin and find that $c=0$. $\qquad \blacksquare$

\medskip
\begin{cor}
Let $v=v_0+u$ where $v_0$ is a constant, $u$ satisfies the conditions of the above Proposition. Then $\II \D v=v-v_0$.
\end{cor}

\medskip
{\it Proof}. We have $\D v=\D u$, $\II \D v=u=v-v_0$. $\qquad \blacksquare$

\medskip
\section{The Cauchy problem}

{\bf 3.1. Local solvability.} Let us consider the problem (\ref{1.1})-(\ref{1.2}), that is
\begin{equation}
\label{3.1}
\left( \D u\right) (|t|_K) =f(|t|_K,u(|t|_K)),\quad 0\ne t\in K,
\end{equation}
with the initial condition
\begin{equation}
\label{3.2}
u(0)=u_0
\end{equation}
where the function $f:\ q^{\mathbb Z}\times \RR \to \RR$ satisfies the conditions
\begin{equation}
\label{3.3}
|f(|t|_K,x)|\le M;
\end{equation}
\begin{equation}
\label{3.4}
|f(|t|_K,x)-f(|t|_K,y)|\le F|x-y|,
\end{equation}
for all $t\in K,x,y\in \RR$.

With the problem (\ref{3.1})-(\ref{3.2}) we associate the integral equation
\begin{equation}
\label{3.5}
u(|t|_K)=u_0+\II f(|\cdot |_K,u(|\cdot |_K))(|t|_K).
\end{equation}
Note that, by the definition of $\II$, in order to compute $(\II \varphi )(|t|_K)$ for $|t|_K\le q^m$ ($m\in \mathbb Z$), one needs to know the function $\varphi$ in the same ball  $|t|_K\le q^m$. Therefore local solutions of the equation (\ref{3.5}) make sense, in contrast to solutions of (\ref{3.1}).

We will call a solution $u$ of (\ref{3.5}), if it exists, {\it a mild solution} of the Cauchy problem (\ref{3.1})-(\ref{3.2}). By the above Corollary, a solution $u$ of (\ref{3.1})-(\ref{3.2}), such that $u-u_0$ satisfies the conditions of Proposition, is a mild solution.

\medskip
\begin{teo}
Under the assumptions (\ref{3.3}),(\ref{3.4}), the problem (\ref{3.1})-(\ref{3.2}) has a unique local mild solution, that is the integral equation (\ref{3.5}) has a solution $u(|t|_K)$ defined for $|t|_K\le q^N$ where $N\in \mathbb Z$ is sufficiently small, and another solution $\overline{u}(|t|_K)$, if it exists, coincides with $u$ for $|t|_K\le q^{\overline{N}}$ where $\overline{N}\le N$.
\end{teo}

\medskip
{\it Proof}. We look for a solution of the equation (\ref{3.5}) as a limit of the sequence $\{ u_k\}$ where $u_0$ is the initial value from (\ref{3.5}),
\begin{equation}
\label{3.6}
u_k(|t|_K)=u_0+\II f(|\cdot |_K,u_{k-1}(|\cdot |_K))(|t|_K).
\end{equation}

We will use the following inequality proved in \cite{K2014}. Let
$$
I_{\alpha ,m}=\int\limits_{|y|_K<|t|_K}\left| |t|_K^{\alpha -1}-|y|_K^{\alpha -1}\right| |y|_K^{\alpha m}\,dy,\quad \alpha \ne 1,
$$
and
$$
I_{1,m}=\int\limits_{|y|_K<|t|_K}\left( \log |t|_K-\log |y|_K\right)|y|_K^m\,dy.
$$
Then
\begin{equation}
\label{3.7}
I_{\alpha ,m}=d_{\alpha ,m}|t|_K^{\alpha (m+1)},\quad 0<d_{\alpha,m}\le Aq^{-\alpha m},
\end{equation}
where $A>0$ does not depend on $m$.

Note in particular the case $m=0$, which implies the following inequality: if $\varphi$ is a bounded continuous function, then
\begin{equation}
\label{3.8}
\left| (\II \varphi )(|t|_K)\right| \le C\mu |t|_K^\alpha ,\quad \mu =\sup\limits_{t\in K}|\varphi (|t|_K)|.
\end{equation}
The meaning of this property is different from its classical counterparts, in view of the identity (\ref{2.11}).

Similarly, if $|\varphi (|s|_K)|\le \mu |s|_K^{\alpha m}$, then
\begin{equation}
\label{3.9}
\left| (\II \varphi )(|t|_K)\right| \le C\mu |t|_K^{\alpha (m+1)},\quad m=1,2,\ldots .
\end{equation}

Using (\ref{3.3}),(\ref{3.4}),(\ref{3.7}) and (\ref{3.8}),(\ref{3.9}) we find that
$$
|u_1(|t|_K)-u_0|\le CM|t|_K^\alpha ,
$$
$$
|u_2(|t|_K)-u_1(|t|_K)|=\left| \II [f(|\cdot|_K,u_1(|\cdot |_K))-f(|\cdot|_K,u_0)](|t|_K)\right|
$$
where
$$
|f(|s|_K,u_1(|s|_K))-f(|s|_K,u_0)|\le F|u_1(|s|_K)-u_0|\le CMF|s|_K^\alpha ,
$$
so that
$$
|u_2(|t|_K)-u_1(|t|_K)|\le C^2MF|t|_K^{2\alpha}.
$$

By induction, we find that
$$
|u_{k+1}(|t|_K)-u_k(|t|_K)|\le C^{k+1}MF^k|t|_K^{(k+1)\alpha}.
$$
This shows that the sequence $\{ u_k\}$ converges uniformly on the ball $\{ |t|_K\le T\}$, if $T$ is sufficiently small, to a limit $u(|t|_K)$. By (\ref{2.5}) and (\ref{2.6}), we obtain that
$$
\II f(|\cdot|_K,u_k(|\cdot |_K))(|t|_K)\longrightarrow f(|\cdot|_K,u(|\cdot |_K))(|t|_K),\quad |t|_K\le T,
$$
so that $u(|t|_K)$ is indeed a local solution of the equation (\ref{3.5}).

Suppose we have another local solution $\uu (|t|_K)$. We have by (\ref{3.8}) that
$$
|u(|t|_K)-\uu (|t|_K)|\le 2MC|t|_K^\alpha.
$$
On the other hand,
$$
u(|t|_K)-\uu (|t|_K)=\II f(|\cdot|_K,u(|\cdot |_K))(|t|_K)-\II f(|\cdot|_K,\uu (|\cdot |_K))(|t|_K),
$$
and by (\ref{3.4}) and (\ref{3.9}),
$$
|u(|t|_K)-\uu (|t|_K)|\le 2MC^2F|t|_K^{2\alpha}.
$$

Repeating these arguments we obtain by induction that
$$
|u(|t|_K)-\uu (|t|_K)|\le 2MC^m F^{m-1}|t|_K^{m\alpha}.
$$
For sufficiently small $|t|_K$, this iteration with $m\to \infty$ implies the equality $u(|t|_K)=\uu (|t|_K)$. $\qquad \blacksquare$

\bigskip
{\bf 3.2. Extension of solutions.} Let us study the possibility to continue the local solution constructed in Theorem 1 to a solution of the integral equation (\ref{3.5}) defined for all $t\in K$.

Suppose that the conditions of Theorem 1 are satisfied, and we obtained a local solution $u(|t|_K)$, $|t|_K\le q^N$, $N\in \mathbb Z$. Let $\alpha \ne 1$. In order to find a solution for $|t|_K=q^{N+1}$, we have, by virtue of (\ref{2.7}), to solve the equation
\begin{equation}
\label{3.10}
u(q^{N+1})=u_0+v_0^{(N)}+q^{\alpha N}f(q^{N+!},u(q^{N+1}))
\end{equation}
where
\begin{equation}
\label{3.11}
v_0^{(N)}=\frac{1-q^{-\alpha}}{1-q^{\alpha -1}}\int\limits_{|y|_K\le q^N}\left( q^{(N+1)(\alpha -1)}-|y|_K^{\alpha -1}\right) f(|y|_K,u(|y|_K))\,dy.
\end{equation}
is a known constant. A similar equation can be written for $\alpha =1$. Then the above procedure, if it is successful, is repeated for all $l>N$.

The following result is an immediate consequence of Banach's fixed point theorem.

\medskip
\begin{teo}
Suppose that the conditions of Theorem 1 are satisfied, as well as the following Lipschitz condition:
\begin{equation}
\label{3.12}
|f(q^l,x)-f(q^l,y)|\le F_l |x-y|,\quad x,y\in \RR ,\ l\in \mathbb Z,
\end{equation}
where $0<F_l<q^{-\alpha l}$ for each $l\in \mathbb Z$. Then a local solution of the equation (\ref{3.5}) admits a continuation to a global solution defined for all $t\in K$.
\end{teo}

\bigskip
{\bf 3.3. From an integral equation to a differential one.} Let us study conditions, under which the above continuation procedure leads to a solution of the problem (\ref{3.1})-(\ref{3.2}). As before, we assume the conditions (\ref{3.3}),(\ref{3.4}) and (\ref{3.12}). In addition, we will assume that
\begin{equation}
\label{3.13}
|f(q^l,x)|\le Cq^{-\beta l},\quad l\ge 1,\quad \text{for all $x\in \RR$,}
\end{equation}
where $\beta >\alpha$.

Let us estimate the mild solution $u(|t|_K)$ obtained by iterations for $|t|_K\le q^N$, and then extended like in (\ref{3.10}). This solution is automatically continuous at the origin. For each $l\ge N$, the solution satisfies the equation similar to (\ref{3.10}):
$$
u(q^{l+1})=u_0+v_0^{(l)}+q^{\alpha l}f(q^{l+1},u(q^{l+1})).
$$

For $l\ge 1$, let us write $v_0^{(l)}=v_{0,1}^{(l)}+v_{0,2}^{(l)}$ where the summands correspond to the integration over the sets $\{ |y|_K\le 1\}$ and $\{ |y|_K\ge q\}$ respectively. Then
$$
\left| v_{0,1}^{(l)}\right| \le C\left[ q^{(l+1)(\alpha -1)}+1\right] ;
$$
\begin{multline*}
\left| v_{0,2}^{(l)}\right| \le Cq^{(l+1)(\alpha -1)}\int\limits_{q\le |y|_K\le q^l}|y|_K^{-\beta}dy+C\int\limits_{q\le |y|_K\le q^l}|y|_K^{\alpha -1-\beta}dy\\
=C_1q^{(l+1)(\alpha -1)}\sum\limits_{j=1}^l q^{-\beta j+j}+C_1\sum\limits_{j=1}^l q^{j(\alpha -1-\beta)+j}\le C_2\left( 1+q^{(\alpha -\beta)l}\right) .
\end{multline*}

It follows from (\ref{3.10}) and (\ref{3.13}) that the conditions (\ref{2.3}) of the existence of $\D u$ are satisfied. In addition, it follows from (\ref{3.13}) that $f(|t|_K,u(|t|_K))$ satisfies the condition (\ref{2.9}) (or (\ref{2.10}), for $\alpha =1$), under which $\D \II f=f$. We come to the following result.

\medskip
\begin{teo}
Under the assumptions (\ref{3.3}),(\ref{3.4}), (\ref{3.12}) and (\ref{3.13}), the mild solution obtained by the iteration process with subsequent continuation, satisfies the equation (\ref{3.1}).
\end{teo}

\section*{Acknowledgments}
This work was funded in part under the budget program of Ukraine No. 6541230 ``Support to the development of priority research trends'' and under the research work "Markov evolutions in real and p-adic spaces" of the Dragomanov National Pedagogical  University of Ukraine.

\medskip


\section{Introduction}
With the rapid development of convolutional neural networks (CNN) in recent years, many major breakthroughs have been made in the field of object detection \cite{dai2016r, he2017mask, redmon2018yolov3, ren2015faster}. Detection models are getting faster, more reliable, and more accurate. However, domain shift remains one of the major challenges in this area. For example, as shown in Figure \ref{fig:domain_shift}, models trained with normal weather images are unable to effectively detect objects in foggy weather. This is because the domain shift causes significant differences in the features extracted from the two types of data, making it impossible to simply apply the model trained on the source domain directly to the unlabeled target domain.

\begin{figure}[t]
	\begin{center}
		\includegraphics[width=1\linewidth]{./introduction.pdf}
	\end{center}
	\caption{Illustration of domain shift. (a) Detection result of supervised training on \textit{Cityscapes}. (b) Detection result on \textit{Foggy Cityscapes} using the model trained on \textit{Cityscapes}. The weather conditions cause great performance drop.}
	\label{fig:domain_shift}
\end{figure}

Although collecting more data for training may alleviate this problem, it is non-trivial because the annotations in object detection are particularly burdensome. To tackle the domain shift problem without introducing additional annotations, many researchers propose various domain adaptation methods to transfer the knowledge of the label-rich domain to the label-poor domain \cite{ganin2016domain, hoffman2017cycada, liu2017unsupervised, tzeng2017adversarial}. Such methods use adversarial learning to minimize the $\mathcal{H}$-divergence between the source and the target domains, searching for an appropriate feature space in which the distribution of the source and target objects is well aligned. Therefore, the model can extract domain-independent features of the objects and correctly detecting them via knowledge transfer without requiring additional annotations.

The key to achieving such a goal is how to measure the learned features in different domains and examine whether they are consistent. For example, Domain-Adversarial Neural Network (DANN) \cite{ganin2015unsupervised} uses a domain classifier to measure the features and achieves end-to-end adversarial learning by reversing the gradient from the subsequent level to the preceding layers, thereby improving the consistency of feature distributions in different domains. Domain Adaptive (DA) Faster R-CNN \cite{chen2018domain} extends this idea to object detection, matching features in the image stage and instance stage. In a more recent work, the authors demonstrate that global matching in DA Faster R-CNN may only address small domain shifts, but the detection accuracy related to large domain shifts may decrease \cite{saito2019strong}. Therefore, they propose a Strong-Weak DA Faster R-CNN that combines weak global alignment with strong local alignment.

In this work, we expect to optimize the original DA Faster R-CNN model to achieve more accurate cross-domain object detection without using additional annotations. One of the main problems with existing methods is that light-weighted domain classifiers cannot form effective adversarial learning with complex Faster R-CNNs. That is, Faster R-CNN can easily deceive the domain classifier, so that the feature alignment is highly possible to be ineffective. Inspired by the Strong-Weak DA Faster R-CNN~\cite{saito2019strong}, we propose a Multi-level Domain Adaptive Faster R-CNN. Our model has two advantages: First, we use different domain classifiers to supervise the feature alignments from multiple scales; Second, more domain classifiers enhance the model's discriminating ability and optimize overall adversarial training. Experiments have shown that aligning the feature distributions of intermediate layers can also alleviate covariate shift and achieve better domain adaptation. Furthermore, our model also follows the conclusion in \cite{saito2019strong} that local alignment should be stronger than global alignment. Because during the backpropagation, the lower feature extractors in Faster R-CNN are getting the reversal gradient from all subsequent domain classifiers, which means it should maintain stronger ability of feature alignment to deceive more domain classifiers. For higher levels, the need for this ability will be appropriately weakened.

We evaluate our approach on several datasets including \textit{Cityscapes} \cite{cordts2016cityscapes}, \textit{KITTI} \cite{geiger2013vision}, \textit{SIM 10k} \cite{johnson2017driving}. The qualitative and quantitative results demonstrate the effectiveness of our method for addressing the domain shift problem. Furthermore, the multiple domain classifiers are only used for model training and not for inference, which won't impact the inference efficiency.

\section{Related Work}
\paragraph{Domain adaptation}
Domain adaptation is a technique that adapts a model trained in one domain to another. Many related works try to define and minimize the distance of feature distributions between the data from different domains \cite{ganin2015unsupervised,long2015learning,ren2018cross,saito2018maximum, Tzeng_2017_CVPR,tzeng2014deep,zhang2018collaborative}. For example, deep domain confusion (DDC) model \cite{tzeng2014deep} explores invariant representations between different domains by minimizing the maximum mean discrepancy (MMD) of feature distributions. Long \etal propose to adapt all task-specific layers and explore multiple kernel variants of MMD \cite{long2015learning}. Ganin and Lempitsky report using the adversarial learning to achieve domain adaptation and learning the distance with the discriminator \cite{ganin2015unsupervised}.  Most of the mentioned works above are designed for classification or segmentation.

Huang \etal propose that aligning the distributions of activations of intermediate layers can alleviate the covariate shift \cite{huang2018domain}. This idea is similar to our work partly. However, instead of using a least squares generative adversarial network(LSGAN) \cite{mao2017least} loss to align distributions for semantic segmentation, we use multi-level image patch loss for object detection.

\paragraph{Domain adaptation for object detection}
Although domain adaptation has been studied for a long time in classification tasks, its application in object detection is still in its early stages. Chen \etal propose to align both image's features and instance's features to achieve cross-domain object detection \cite{chen2018domain}. Inoue \etal address cross-domain weakly supervised object detection using domain-transfer and pseudo labeling \cite{inoue2018cross}. More recently, Kim \etal use domain diversification and multi-domain-invariant representation learning to address the source-biased problem \cite{kim2019diversify}. Saito \etal propose global-weak alignment that puts less emphasis on aligning images that are globally dissimilar \cite{saito2019strong}. Zhu \etal focuses on mining the discriminative regions which are directly related to object detection and aligning them across different domains \cite{zhu2019adapting}.

\section{Method}

\begin{figure*}[!h]
	\begin{center}
		\includegraphics[width=0.88\linewidth]{./network.pdf}
	\end{center}
	\caption{The Illustration of proposed network. Our method uses multi-level domain classifiers to get a better alignment of feature distributions (see the t-SNE visualization). All domain classifiers utilize GRL layers to achieve adversarial learning.}
	\label{fig:network}
\end{figure*}

\subsection{Preliminaries}
Our work adopts the main idea of \textbf{Domain Adaptive Faster R-CNN (DA model)} \cite{chen2018domain}, which contains two major parts: 1. Image-Stage Adaptation; 2. Instance-Stage Adaptation.

\paragraph{Image-Stage Adaptation}  A domain classifier is used to predict the domain label for each image patch, which reduces the image-stage shift, such as image style, scale, \textit{etc.} The loss of image-stage adaptation can be formatted as,

\begin{equation}
\mathcal{L}_{i m g}=-\sum_{i, u, v}\left[D_{i} \log p_{i}^{u, v}+\left(1-D_{i}\right) \log \left(1-p_{i}^{u, v}\right)\right],
\label{eq_img}
\end{equation}

where $D_i$ denotes the domain label of the $i$-th image. And $p_{i}^{u, v}$ represents the probability that the pixel at $(u,v)$ on the final feature map belongs to the target domain. For each image patch, all corresponding activations on the feature maps will be classified. so we call this \textbf{patch-based domain adaptive loss}.

\paragraph{Instance-Stage Adaptation}
The instance-stage adaptation loss is defined as,
\begin{equation}
\mathcal{L}_{i n s}=-\sum_{i, j}\left[D_{i} \log p_{i, j}+\left(1-D_{i}\right) \log \left(1-p_{i, j}\right)\right],
\label{eq_ins}
\end{equation}

where $p_{i, j}$ represents the probability that the $j$-th region proposal in the $i$-th image is from target domain.

\subsection{Proposed Multi-level DA model}
\label{sec.method}

The \textbf{image-stage adaptation} of DA model uses the feature map after the last convolutional layer to align the global feature distribution of different domains. However, such a setting has two limitations. First, the model ignores the alignment of local features, making certain domain-sensitive local features weaken the generalization ability of the adaptive model. Second, single adaptation (one domain classifier) is difficult to cancel the data bias between the source domain and the target domain, because there are other non-transferable layers. \cite{long2015learning}.

\paragraph{Multi-level Patch-based Loss} To solve the aforementioned problems and to improve the cross-domain detection, we adopt the idea of layer-wise adaptation \cite{huang2018domain, long2015learning}.

We extract the output feature maps of multiple intermediate layers in the convolutional network, and build multiple corresponding image domain classifiers to supervise the feature alignments in the intermediate layers. The multiple level loss can be written as,

\begin{equation}
\begin{split}
\mathcal{L}_{multi}=-\sum_{i,k, u, v}[D_{i} \log f_k(\Phi_{i,k}^{u, v})+ \\
(1-D_{i}) \log(1-f_k(\Phi_{i,k}^{u, v}  ))],
\label{eq_mul}
\end{split}
\end{equation}

where $\Phi_{i,k}^{u, v}$ denotes the activation located at $(u, v)$ of the feature map of the $i$-th image after the $k$-th layers, and $f_k$ is its corresponding domain classifier. The multi-level domain adaptive components guarantee that the distributions of intermediate features in two domains are matched and enhance the robustness of the adaptive model. It is worth noting that the number of intermediate layers used in domain adaptation can vary in different datasets. For simplicity, the weight factors of all layers are set equal.

\paragraph{Total Objective Function}
We use Faster R-CNN \cite{ren2015faster} as our detection model. The instance-stage loss and consistency loss (detailed in \cite{chen2018domain}) are also adopted in our work. The overall objective is the summation of detection loss and adaptation loss.

\begin{equation}
L=L_{d e t}+\lambda(L_{multi}+L_{ins}+L_{cst}),
\end{equation}

where $\lambda$ is a weight factor to balance detection loss and adaptation loss. The detection loss is composed of the loss of Region Proposal Network(RPN) and the loss of R-CNN. The total objective $L$ can be optimized by the standard SGD algorithm.

As shown in Figure \ref{fig:network}, the main network architecture of our model contains a detection part (Faster R-CNN) and an adaptation part (local DA, global DA, and instance DA). In the training phase, the detection part is trained by labeled samples from source domain and the adaptation part is trained by samples from both source and target domains. We then use GRL layers \cite{ganin2015unsupervised} to reverse the sign of gradient, which encourages base network to learn domain-insensitive features. In the inference phase, only detection part will be used to predict the bounding boxes and classes of the objects. Thus, it will not lose time efficiency.

\begin{figure*}[!h]
	\begin{center}
		\includegraphics[width=0.95\linewidth]{./img_vis.pdf}
	\end{center}
	\caption{Visualization of image features at different levels using the t-SNE algorithm: (a) Local features from DA model (b) Local features from our Multi-level model (c) Global features from DA model (d) Global features from our Multi-level DA model. Each point represents feature of an image patch. The red is from source domain and the blue is from target domain.}
	\label{fig:img_vis}
\end{figure*}

\section{Experiments and Results}
In order to evaluate the effectiveness of our approach, we follow \cite{chen2018domain} to perform three experiments: \textbf{1) Adverse Weather Adaptation.} In this experiment, we aim to adapt networks from detecting objects in normal weather to that in foggy weather. \textbf{2) Synthetic Data Adaptation.} In this experiment, we aim to adapt networks for the data from video games to the data from real world. \textbf{3) Cross Camera Adaptation.} In this experiment, we aim to adapt networks for the photos under different camera setups. In addition, we evaluate the visualization of feature distribution to support our claim that adding multiple domain classifiers can enhance the model's overall discriminating ability and achieve more appropriate alignments.

\subsection{Experiment details}
In all experiments, only the source training data are provided with annotations. We set the shorter side of the image to 600 pixels. The VGG-16 model \cite{simonyan2014very} is pretrained on ImageNet and is used as the network backbone of our model. Because the first four convolutional layers of VGG are fixed in traning, we distribute the discriminators at equal intervals from the fifth layer to the final convolutional layer (e.g. 5th/13, 9th/13, 13th/13).  The network is then finetuned for 6 epochs with a learning rate of 0.002 and for another 4 epochs with a learning rate of 0.0002. We also use weight decay and momentum, which are set as 0.0005 and 0.9. During the training process, we flip the images for data augmentation and feed two images from the different domains into the network in every iteration. To evaluate the proposed method, we report mean average precision (mAP) with a threshold of 0.5 on the last epoch. Without specific notation, we set $\lambda=0.1$.

\subsection{Adverse Weather Adaptation}

\begin{table*}
	\begin{center}
		\begin{tabular}{|l|c|cccccccc|c|}
			\hline
			Method  & Backbone & Person & Rider & Car & Truck & Bus & Train & Motorcycle & Bicycle & Mean AP \\
			\hline
			Source(Supervised) & VGG-16 & 24.7 & 31.9 & 33.1 & 11.0 & 26.4 & 9.2 & 18.0 & 27.9 & 22.8 \\
			\hline
			DA Model$^{*}$ \cite{chen2018domain} & VGG-16 & 25.0 & 31.0 & 40.5 & 22.1 & 35.3 & 20.2 & 20.0 & 27.1 & 27.6 \\
			DA Model \cite{chen2018domain} & VGG-16 & 29.6 & 38.1 & 43.3 & 20.7 & 31.0 & 21.5 & 24.8 & 32.1 & 30.1 \\
			DT Model \cite{inoue2018cross} & VGG-16 & 25.4 & 39.3 & 42.4 & 24.9 & 40.4 & 23.1 & 25.9 & 30.4 & 31.5 \\
			SC-DA(Type3) \cite{zhu2019adapting} & VGG-16 & \textbf{33.5} & 38.0 & \textbf{48.5} & 26.5 & 39.0 & 23.3 & 28.0 & 33.6 & 33.8 \\
			SW-DA \cite{saito2019strong} & VGG-16 & 29.9 & 42.3 & 43.5 & 24.5 & 36.2 & 32.6 & 30.0 & 35.3 & 34.3 \\
			DD-MRL \cite{kim2019diversify} & VGG-16 & 30.8 & 40.5 & 44.3 & 27.2 & 38.4 & 34.5 & 28.4 & 32.2 & 34.6 \\
			MTOR \cite{cai2019exploring} & Resnet-50 & 30.6 & 41.4 & 44.0 & 21.9 & 38.6 & \textbf{40.6} & 28.3 & 35.6 & 35.1 \\
			\hline
			\textbf{Proposed(n=4)} & VGG-16 & \textbf{33.2} & \textbf{44.2} & 44.8 & \textbf{28.2} & \textbf{41.8} & 28.7 & \textbf{30.5} & \textbf{36.5} & \textbf{36.0} \\
			\hline
			Target(Supervised) & VGG-16 & 37.3 & 48.2 & 52.7 & 35.2 & 52.2 & 48.5 & 35.3 & 38.8 & 43.5 \\
			\hline
		\end{tabular}
	\end{center}
	\caption{Quantitative results on adaptation from \textit{Cityscapes} to \textit{Foggy Cityscapes}. The results of DA Model$^{*}$ is from its original paper and that of DA Model is implemented using our parameters. MTOR uses Resnet-50 as its backbone, while the others are VGG-16. Proposed(n) indicates that the model uses \textit{n} image domain classifiers.}
	\label{tab:foggy}
\end{table*}

In the real world, weather may change every day. It's critical that a detection model can perform consistently in different weather conditions. Therefore, we evaluate our model on \textit{Cityscapes} and \textit{Foggy Cityscapes} \cite{sakaridis2018semantic} datasets, which are used as source domain and target domain, respectively. The \textit{Foggy Cityscapes} dataset is rendered from \textit{Cityscapes} by adding fog noise, so it also has 2975 images in training set and 500 images in validation set. In this experiment, we report our results on all categories carried on the \textit{Foggy Cityscapes} validation set.

The results are summarized in Table \ref{tab:foggy}. The mAP of our method outperforms the baseline by +13.2\% and exceed all the other existing models. It is worth noting that the results of our method are only -7.4\% than the model supervised by target images. Among the performance of each category, our method performs as well as SC-DA(Type3) for person detection. We find the SC-DA(Type3) and MTOR model are very suited for car and train detection respectively, while the performances of other categories are greatly improved by our method.

\subsection{Synthetic Data Adaptation}

\begin{table}
	\begin{center}
		\begin{tabular}{|l|cccc|c|}
			\hline
			Method  & G & I & CTX & L & Car AP \\
			\hline
			Source(Supervised) & & & & & 34.3 \\
			\hline
			DA Model$^{*}$ \cite{chen2018domain} & $\checkmark$ & $\checkmark$ & & & 39.0 \\
			DA Model \cite{chen2018domain} & $\checkmark$ & $\checkmark$ & & & 39.4 \\
			SW-DA \cite{saito2019strong} & $\checkmark$ & & $\checkmark$ & $\checkmark$ & 40.1 \\
			SW-DA($\gamma=3$) \cite{saito2019strong} & $\checkmark$ & & $\checkmark$ & & 42.3 \\
			SC-DA(Type3) \cite{zhu2019adapting} & & & & & \textbf{43.0} \\
			\hline
			Proposed(n=3) & $\checkmark$ & $\checkmark$ & & $\checkmark$ & 42.3 \\
			Proposed(n=4) & $\checkmark$ & $\checkmark$ & & $\checkmark$ & 42.0 \\
			Proposed(n=5) & $\checkmark$ & $\checkmark$ & & $\checkmark$ & 42.7 \\
			Proposed(n=6) & $\checkmark$ & $\checkmark$ & & $\checkmark$ & \textbf{42.8} \\
			\hline
			Target(Supervised) & & & & & 62.7 \\
			\hline
		\end{tabular}
	\end{center}
	\caption{Results on adaptation from \textit{SIM 10k} to \textit{Cityscapes}. G, I, CTX, L indicate global alignment, instance-stage alignment, context-vector based regularization, and local alignment, respectively. DA Model$^{*}$ is from original paper and DA Model is implemented using our parameters.}
	\label{tab:sim10k}
\end{table}

We then show experiments about adaptation from synthetic images to real images. We utilized the \textit{SIM 10k} dataset as the synthetic source domain. This dataset contains 10000 images and 58701 bounding boxes of cars, which are collected from the video game Grand Theft Auto (GTA). All images are used in training. As for the target domain, we used \textit{Cityscapes} dataset. In addition, we only report the average precision of the cars on the validation set, since only the cars have annotations in \textit{SIM 10k}.

The results are summarized in Table \ref{tab:sim10k}. Specifically, compared with the baseline model which was supervised only on the source domain, the proposed model achieves +8.5\% performance gain using 6 domain classifiers. Compared with DA Model which doesn't use local alignment the proposed model achieves an improvement of +3.4\%. This indicates the importance of local alignment. SW-DA Model also adopts local alignment, but they only achieve +0.7\% performance gain, which suggests that the ability of one or two domain classifiers is limited. SC-DA(Type3) Model performs a little better than ours because their model is especially suitable for car detection as shown in the Adverse Weather Adaptation experiment. In addition, we find the performance can be further improved by increasing the number of domain classifiers from 3 to 6.

\begin{figure*}
	\begin{center}
		\includegraphics[width=0.85\linewidth]{./instance_vis.pdf}
	\end{center}
	\caption{Visualization of instance (ROI) features using the t-SNE algorithm: (a) Features from the baseline model (supervised learning by data from source domain) (b) Features from the DA model (c) Features from our multi-level DA model. The color of points represents the domain label and each shape indicates a category of this instance, best viewed in color.}
	\label{fig:ins_vis}
\end{figure*}

\begin{figure*}
	\begin{center}
		\includegraphics[width=0.8\linewidth]{./detection.pdf}
	\end{center}
	\caption{Qualitative results on adaptation from \textit{Cityscapes} to \textit{Foggy Cityscapes} dataset. (a) Raw images in \textit{Cityscapes}, (b) Baseline model (supervised on \textit{Cityscapes}), (c) DA model, (d) Proposed model (n=4). The raw images in \textit{Cityscapes} are for reading only. Boxes with green color mean correct results, red color denotes false positives and yellow color means misclassification.}
	\label{fig:foggy}
\end{figure*}

\subsection{Cross Camera Adaptation}

\begin{table*}
	\begin{center}
		\begin{tabular}{|l|ccccc|c|}
			\hline
			Method  & Person & Rider & Car & Truck & Train & Mean AP \\
			\hline
			Source(Supervised) & 47.8 & 22.0 & \textbf{75.2} & 12.4 & 12.6 & 34.0 \\
			\hline
			DA Model \cite{chen2018domain} & 40.9 & 16.1 & 70.3 & 23.6 & 21.2 & 34.4 \\
			\hline
			\textbf{Proposed(n=4)} & \textbf{53.0} & \textbf{24.5} & 72.2 & \textbf{28.7} & \textbf{25.3} & \textbf{40.7}  \\
			\hline
		\end{tabular}
	\end{center}
	\caption{Quantitative results on adaptation from \textit{Cityscapes} to \textit{KITTI}. Since the detection objects are changed, we only give the results of DA Model which is implemented using our parameters.}
	\label{tab:kitti}
\end{table*}

In this experiment, we aim to analyze the adaptation for the images under different camera setups. We utilize the \textit{Cityscapes} dataset as the source domain and \textit{KITTI} dataset as the target domain. The \textit{KITTI} dataset consists of 7481 images, which have original resolution of 1250x375. They are resized so that the shorter length is 600 pixels long. In addition, the \textit{KITTI} dataset is used in both adaptation and evaluation.

We report the mAP of 5 categories with a threshold of 0.5. However, we find the classification standard of categories in two domain datasets is different. So we classify 'Car' and 'Van' as 'Car', 'Person' and 'Person sitting' as 'Person', then we convert 'Tram' to 'Train', 'Cyclist' to 'Rider' in the \textit{KITTI} dataset, which is different from \cite{chen2018domain}.

The results are summarized in Table \ref{tab:kitti}. In our experimental settings, the baseline model already has a good ability for person, rider, and car detection because both the source and target domain datasets are from real world and the domain shift in these three categories is very small. We find the introduction of domain classifier caused a performance drop. However, our method not only reduces the bad influence in car detection but also greatly improve the detections of persons and riders. As for other categories, both DA model and our method perform better than the baseline but ours goes far beyond the other two. The results indicate our method can achieve a better performance when the domain shift is large, and reduce the possible instability caused by domain adaptation when the domain shift is very small.

\subsection{Analysis}

\paragraph{Visualization of image-stage features}
We visualize the image-stage features using the t-SNE algorithm \cite{maaten2008visualizing} in Figure \ref{fig:img_vis}. All samples are from validation set of \textit{Foggy Cityscapes} dataset. The global features are aligned well in both DA model and the proposed Multi-level DA model. However, the local features between source and target domain are mismatched in DA model. This result confirms the first limitation of DA model in Section~\ref{sec.method}. Our method can align local features more effectively, which benefits from the proposed strategy of multi-level adaptation.

\paragraph{Visualization of instance-stage features}
We extract the features of several region proposals (before the final classification and regression layer). The t-SNE embedding of these features (from \textit{Foggy Cityscapes} dataset) is shown in Figure \ref{fig:ins_vis}. Notice the truck (star) and bus (circle) in Figure \ref{fig:ins_vis}(b). Although the DA model can align the marginal distribution to some extent, the categories are not discriminated well. But our model can align distribution and discriminate categories better. Such improvement explains why our model outperforms the baseline and the DA model.

\paragraph{Qualitative examples of detection results}
Figure \ref{fig:foggy} shows some typical detection results. In the first row, the baseline model almost ignored all objects. The DA model successfully detects a car and a bicycle, but it incorrectly classifies the bus as a truck, and has some false positives. Our model correctly detected the bus. In the second row, our model correctly detects more persons and bicycles in fog, even if recognizing them is challenging for humans.

\section{Conclusion}
In this paper, we propose an effective approach for cross-domain object detection. We introduce multiple domain classifiers to enforce multi-level adversarial training to improve the overall feature alignment. The proposed method outperforms the existing methods in several experiments. Moreover, the visualizations of feature distributions prove that our model can get more effective alignment than other models. However, the implementation of adversarial training in our model is based on gradient reversal layers (GRLs), which may cause instability in training. In future work, we plan to further investigate how to improve the accuracy and robustness of our models.

\paragraph{Acknowledgments.} This work was supported by National Key R\&D Program of China (No. 2018YFC0910\-700), NSFC (81801778, 11831002, 61625201 and 6152\-7804), Beijing Natural Science Foundation (Z180001) and Qualcomm University Research Grant.
{\small
\bibliographystyle{ieee}


\section{Introduction}

The advancement of moving-camera technologies provides a new perspective for video surveillance. Unmanned aerial vehicles (UAVs), such as drones in the air, can provide top views of a group of subjects on the ground. Wearable cameras, such as Google Glass and GoPro, mounted over the head of a wearer (one of the subjects on the ground), can provide horizontal views of the same group of subjects. As shown in Fig.~\ref{fig:example}, the data collected from these two views well complement each other -- top-view images contain no mutual occlusions and well exhibit a global picture and the relative positions of the subjects, while horizontal-view images can capture the detailed appearance, action, and behavior of subjects of interest in a much closer distance.  Clearly, their collaborative analysis can significantly improve the video-surveillance capabilities such as human tracking, human detection, and activity recognition.

\begin{figure}[t]
	\centering
	\includegraphics[width=0.475\textwidth]{fig1-eps-converted-to.pdf}
	\caption{An illustration of the top-view (left) and horizontal-view (right) images. The former is taken by a camera mounted to a drone in the air and the latter is taken by a GoPro worn by a wearer who walked on the ground. The proposed method identifies on the top-view image the location and view angle of the camera (indicated by red box) that produces the horizontal-view image, and associate subjects, indicated by identical color boxes, across these two videos.}
	\label{fig:example}
\end{figure}

The first step for such a collaborative analysis is to accurately associate the subjects across these two views, i.e., we need to identify any person present in both views and identify his location in both views, as shown in Fig.~\ref{fig:example}. In general, this can be treated as a person re-identification (re-id) problem -- for each subject in one view, re-identify him in the other view. However, this is a very challenging person re-id problem because the same subject may show totally different appearance in top and horizontal views, not to mention that the top view of subjects contains very limited features by only showing the top of heads and shoulders and it can be very difficult to distinguish different subjects from their top views, as shown in Fig.~\ref{fig:example}.

Prior works~\cite{ardeshir2016ego2top,Ardeshir2018Egocentric,ardeshir2018integrating} tried to alleviate the challenge of this problem by assuming 1) the view direction of the top-view camera in the air has certain slope such that subjects' body, and even part of the background, are still partially visible in top views and can be used for feature matching to the horizontal views, and 2) the view angle of the horizontal-view camera on the ground is consistent with the moving direction of the camera wearer and can be easily estimated by computing optical flow in the top-view videos This can be used to identify the on-the-ground camera wearer in the top-view video. These two assumptions, however, limit the their applicability in practice, e.g., the horizontal-view camera wearer may turn head (and therefore the head-mounted camera) when he walks, leading to inconsistency between his moving direction and wearable-camera view direction.

In this paper, we develop a new approach to associate subjects across top and horizontal views without the above two assumptions. Our main idea is to explore the spatial distribution of the subjects for cross-view subject association. From the horizontal-view image, we detect all the subjects, and estimate their depths and spatial distribution using the sizes and locations of the detected subjects, respectively. On the corresponding top-view image, we traverse each detected subject and possible direction to localize the horizontal-view camera (wearer), as well as its view angle. For each traversed location and direction, we estimate the spatial distribution of all the visible subjects. We finally define a matching cost between the subjects' spatial distributions in top and horizontal views to decide the horizontal-view camera location and view angle, with which we can associate the subjects across the two views. In the experiments, we collect a new dataset consisting of image pairs from top and horizontal-views for performance evaluation. Experimental results verify that the proposed method can effectively associate multiple subjects across top and horizontal views.

The main contributions of this paper are:
1) We propose to use the spatial distribution of multiple subjects for associating subjects across top and horizontal views, instead of using subject appearance and motions in prior works.
2) We develop geometry-based algorithms to model and match the subjects’ spatial distributions across top and horizontal views.
3) We collect a new dataset of top-view and horizontal-view images for evaluating the proposed cross-view subject association.

The remainder of this paper is organized as follows. Section~\ref{sec:related} reviews the related work. Section~\ref{sec:method} elaborates on the proposed method and Section~\ref{sec:experiment} reports the experimental results, followed by a brief conclusion in Section~\ref{sec:conclusion}.

\section{Related Work}
\label{sec:related}

Our work can be regarded as a problem of associating first-person and third-person cameras, which has been studied by many researchers. For example, Fan et al. \cite{fan2017identifying} identify a first-person camera wearer in a third-person video by incorporating spatial and temporal information from the videos of both cameras. In \cite{ferland2009egocentric}, information from first- and third-person cameras, together with laser range data, is fused to improve depth perception and 3D reconstruction. Park et al. \cite{park2013predicting} predict gaze behavior in social scenes using both first- and third-person cameras. In \cite{xu2018joint}, first- and third-person cameras are synchronized, followed by associating subjects between their videos. In \cite{soran2014action}, a first-person video is combined to multiple third-person videos for more reliable action recognition. The third-person cameras in these methods usually bear horizontal views or views with certain slope angle. Differently, in this paper the third-person camera is mounted on a drone and produces top-view images, making cross-view appearance matching a very difficult problem.

As mentioned above, cross-view subject association can be treated as a person re-id problem, which has been widely studied in recent years. Most existing re-id methods can be grouped into two classes: similarity learning and representation learning. The former focuses on learning the similarity metric, e.g., the invariant feature learning based models~\cite{Liao2015Person, Rui2014Learning,Yang2014Salient},  classical metric learning models~\cite{K2012Large, Paisitkriangkrai2015Learning, Giuseppe2015Person}, and deep metric learning models~\cite{Low2016Learning, Varior2016A}. The latter focuses on feature learning, including
low-level visual features such as color, shape, and texture ~\cite{Gray2008Viewpoint, Ma2012Local}, and more recent CNN deep features~\cite{Zheng2017Person, Tong2016Learning}.
These methods assume that all the data are taken from horizontal views, with similar or different horizontal view angles, and almost all of these methods are based on appearance matching. In this paper, we attempt to re-identify subjects across top and horizontal views, where appearance matching is not an appropriate choice.

More related to our work is a series of recent works by Ardeshir and Borji~\cite{ardeshir2016ego2top,Ardeshir2018Egocentric,ardeshir2018integrating} on building association between top-view and horizontal-view cameras. In \cite{ardeshir2016ego2top,Ardeshir2018Egocentric}, by jointly handling a set of egocentric (first-person) horizontal-view videos and a top-view video, a graph-matching based algorithm is developed to locate all the horizontal-view camera wearers in the top-view video. In \cite{ardeshir2018integrating}, the problem is extended to locate not only the camera wearers, but also other horizontal-view subjects in the top-view video. However, as mentioned above, these methods are based on two assumptions: 1) the top-view camera bears certain slope angle to enable the partial visibility of human body and the use of appearance matching for cross-view association, and 2) the looking-at direction of the horizontal-view camera is the same as the moving direction of the camera wearer. In this paper, we remove these two assumptions and leverage the spatial distribution of subjects for cross-view subject association. The methods developed in \cite{ardeshir2016ego2top,Ardeshir2018Egocentric,ardeshir2018integrating} require multi-frame video inputs since it needs to estimate each subject's moving direction, while our method can associate a single frame in top view and its corresponding frame in horizontal view.

\section{Proposed Method}
\label{sec:method}
In this section, we first give an overview of the proposed method and then elaborate on the main steps.

\subsection{Overview}

\begin{figure}[htbp]
	\centering
	\includegraphics[width=\columnwidth]{top_view-eps-converted-to.pdf}
	\caption{An illustration of vector representation in (a) top view and (b) horizontal view.}
	\label{fig:top_view}
\end{figure}

Given a top-view image and a horizontal-view image that are taken by respective cameras at the same time, we detect all persons (referred to as subjects in this paper) on both images by a person detector~\cite{Redmon2016You}.
Let ${\mathcal{T}} = \{ O_i^{{\mathop{\rm top}\nolimits} }\}_{i=1}^{N}$ be the collection of $N$ subjects detected on the top-view image,
with $O_i^{{\rm{top}}}$ being the $i$-$\mathrm{th}$ detected subject.
Similarly, let ${\mathcal{E}} = \{ O_j^{{\mathop{\rm hor}\nolimits} }\}_{j=1}^M$ be the collection of $M$ subjects detected on the horizontal-view image, with $O_j^{{\rm{hor}}}$ being the $j$-$\mathrm{th}$ detected subject.
The goal of cross-view subject association is to identify all the matched subjects between ${\mathcal{T}} $ and ${\mathcal{H}}$ that indicate the the same persons.

In this paper, we address this problem by exploring the spatial distributions of the detected subjects in both views.
More specifically, from each detected subject $O_i^{{\rm{top}}}$ in the top view, we infer a vector $\mathbf{V}^\mathrm{top}_i = (x^\mathrm{top}_i, y^\mathrm{top}_i)$
that reflects its relative position to the horizontal-view camera (wearer) on the ground.
Then for each detected subject $O_j^{{\rm{hor}}}$ in the horizontal view, we also infer a vector $\mathbf{V}^\mathrm{hor}_j = (x^\mathrm{hor}_j, y^\mathrm{hor}_j)$ to reflect its relative position to the horizontal-view camera on the ground.
We associate the subjects detected in two views by seeking matchings between two vector sets
${{\bf{V}}^{{\mathop{\rm top}\nolimits} }}({\mathcal{T}},\theta,O) = \{\mathbf{V}^\mathrm{top}_i\}_{i=1}^N$
and $\mathbf{V}^\mathrm{hor} ({\mathcal{E}})= \{\mathbf{V}^\mathrm{hor}_j\}_{j=1}^M$,
where $O$ and $\theta$ are the location and view angle of the horizontal-view camera (wearer) in
the top-view image and they are unknown priorly.
Finally, we define a matching cost function $\phi$ to measure the dissimilarity between the two vector sets and optimize this function for finding the matching subjects between two views,
as well as the camera location  $O$, and camera view angle $\theta$.
In the following, we elaborate on each step of the proposed method.

\subsection{Vector Representation}

In this section, we discuss how to derive $\mathbf{V}^\mathrm{top}$ and $\mathbf{V}^\mathrm{hor}$. On the top-view image, we first assume that the horizontal-view camera location $O$ and its view angle $\theta$ are given. This way, we can compute its field of view in the top-view image and all the detected subjects' relative positions to the horizontal-view camera on the ground. Horizontal-view image is egocentric and we can compute the detected subjects' relative positions to the camera based on the subjects' sizes and positions on the horizontal-view image.

\subsubsection{Top-View Vector Representation}

As shown in Fig.~\ref{fig:top_view}(a), in the top-view image we can easily compute the left and right boundaries of the field of view of the horizontal-view camera, denoted by $\mathord{\buildrel{\lower3pt\hbox{$\scriptscriptstyle\rightharpoonup$}} \over L}$,
$\mathord{\buildrel{\lower3pt\hbox{$\scriptscriptstyle\rightharpoonup$}} \over R}$, respectively, based on the given camera location  $O$ and its view angle $\theta$.
For a subject at $P$ in the field of view, we estimate its relative position to the horizontal-view camera by using two geometry parameters ${\hat x}$ and ${\hat y}$, where
${\hat x}$ is the (signed) distance to the horizontal-view camera along the (camera) right direction $\mathord{\buildrel{\lower3pt\hbox{$\scriptscriptstyle\rightharpoonup$}} \over V}$, as shown in
Fig.~\ref{fig:top_view}(a) and ${\hat y}$ is the depth.
Based on pinhole camera model, we can calculate them by
\begin{equation} \label{eq:xy_top}
\left\{ \begin{array}{l}
{\hat x} = f\cot \langle \mathord{\buildrel{\lower3pt\hbox{$\scriptscriptstyle\rightharpoonup$}} \over {OP}}, \mathord{\buildrel{\lower3pt\hbox{$\scriptscriptstyle\rightharpoonup$}} 	\over V} \rangle\\
{\hat y} =|\mathord{\buildrel{\lower3pt\hbox{$\scriptscriptstyle\rightharpoonup$}} 	\over {OP}}|\cdot \sin \langle \mathord{\buildrel{\lower3pt\hbox{$\scriptscriptstyle\rightharpoonup$}} 	\over {OP}}, \mathord{\buildrel{\lower3pt\hbox{$\scriptscriptstyle\rightharpoonup$}}  \over V} \rangle,
\end{array} \right.
\end{equation}
where $ \langle \cdot, \cdot \rangle$ indicates the angle between two directions and $f$ is the focus length of horizontal-view camera.

Next we consider the range of $\hat x$. From Fig.~\ref{fig:top_view}(a), we can get
\begin{equation} \label{eq:xlr}
\left\{ \begin{array}{l}
{\hat x}_{\mathrm{min}} = f \cot  \langle \mathord{\buildrel{\lower3pt\hbox{$\scriptscriptstyle\rightharpoonup$}} \over {L}}, \mathord{\buildrel{\lower3pt\hbox{$\scriptscriptstyle\rightharpoonup$}} 	\over V} \rangle  = f\cot (\frac{{\pi  + \alpha }}{2})\\
{\hat x}_{\mathrm{max}}  = f \cot  \langle \mathord{\buildrel{\lower3pt\hbox{$\scriptscriptstyle\rightharpoonup$}} 	\over {R}}, \mathord{\buildrel{\lower3pt\hbox{$\scriptscriptstyle\rightharpoonup$}} 	\over V} \rangle  = f\cot (\frac{{\pi  - \alpha }}{2}),
\end{array}  \right.
\end{equation}
where $ \alpha \in [0,\pi]$ is the given field-of-view angle of the horizontal-view camera as indicated in Fig.~\ref{fig:top_view}(a). From Eq.~(\ref{eq:xlr}), we have ${\hat x}_{\mathrm{max}} =-{\hat x}_{\mathrm{min}} >0$.

To enable the matching to the vector representation from the horizontal view, we further normalize the value range of ${\hat x}$ to $[-1, 1]$, i.e.,
\begin{equation} \label{eq:norm_top}
\left\{ \begin{array}{l}
x^{\mathrm{top}}= \dfrac{\hat x}{{f \cot (\frac{{\pi  - \alpha }}{2})}} \\
y^{\mathrm{top}}={\hat y}.
\end{array} \right.
\end{equation}
With this normalization, we actually do not need the actual value of $f$ in the proposed method.

Let $O_k^{{{\rm top}} }$, ${k} \in {\mathcal{K}} \subset \{1,2,\cdots, N\}$ be the subset of detected subjects in the field of view in the top-view image. We can find the vector representation for all of them and sort them
in terms of $x^{\mathrm{top}}$ values in an ascending order. We then stack them together as
\begin{equation} \label{eq:v_top}
{{\bf{V}}^{\rm top}}{\rm{ = (}}{{\bf{x}}^{{\mathop{\rm top}\nolimits} }} , {{\bf{y}}^{{\mathop{\rm top}\nolimits} }}) \in \mathbb{R} {^{{|{\mathcal{K}}|} \times 2}}
\end{equation}
where  ${|\mathcal{K}|}$ is the size of ${\mathcal{K}}$,  and ${{\bf{x}}^{{\mathop{\rm top}\nolimits} }}$ and  ${{\bf{y}}^{{\mathop{\rm top}\nolimits} }}$ are the column-wise vectors of all the $x^{\mathrm{top}}$ and $y^{\mathrm{top}}$ values of the subjects in the field of view, respectively.

\subsubsection{Horizontal-View Vector Representation}
\label{sec:Hor_view_vec}

For each subject in the horizontal-view image, we also compute a vector representation to make it consistent to the top-view vector representation, i.e., $x$-value reflects the distance to the horizontal-view camera along the right direction and $y$-value reflects the depth to the horizontal-view camera.
As shown in Fig.~\ref{fig:top_view}~(b), in the horizontal-view image, let $Q=(\tilde{x}, \tilde{y})$ and $h$ be the location and height of a detected subject, respectively.
If we take the top-left corner of the image as the origin of the coordinate, $\tilde{x}-\frac{W}{2}$, with $W$ being the width of the horizontal-view image, is actually the subject's distance to the
horizontal-view camera along the right direction.
To facilitate the matching to the top-view vectors, we normalize its value range to $[-1, 1]$ by
\begin{equation} \label{eq:norm_hor}
\left\{ \begin{array}{l}
x^{\mathrm{hor}}= \dfrac{{\tilde x} - \frac{W}{2}}{\frac{W}{2}} \\
y^{\mathrm{hor}}=\frac{1}{h},
\end{array} \right.
\end{equation}
where we simply take the inverse of the subject height as its depth to the horizontal-view camera.

For all $M$ detected subjects in the horizontal-view image, we can find their vector representations and sort them
in terms of $x^{\mathrm{hor}}$ values in an ascending order. We then stack them together as
 \begin{equation} \label{eq:v_hor}
 {{\bf{V}}^{\rm hor}}{\rm{ = (}}{{\bf{x}}^{{\mathop{\rm hor}\nolimits} }} , {{\bf{y}}^{{\mathop{\rm hor}\nolimits} }}) \in \mathbb{R} {^{{M} \times 2}}
 \end{equation}
 where  ${{\bf{x}}^{{\mathop{\rm hor}\nolimits} }}$ and  ${{\bf{y}}^{{\mathop{\rm hor}\nolimits} }}$ are the column-wise vectors of all the $x^{\mathrm{hor}}$ and $y^{\mathrm{hor}}$ values of the $M$ subjects detected in the  horizontal-view image, respectively.

\subsection{Vector Matching}
\label{sec:vectormatch}

In this section we associate the subjects across two views by matching the vectors between the two vector sets ${{\bf{V}}^{{\mathop{\rm top}\nolimits} }}$ and ${{\bf{V}}^{{\mathop{\rm hor}\nolimits}}}$. Since the $x$ values of both vector sets have been normalized to the range of $[-1, 1]$, they can be directly compared. However, the $y$ values in these two vector sets are not comparable, although both of them reflect the depth to the horizontal-view camera: $y^{\mathrm{top}}$ values are in terms of number of pixels in the top-view image while $y^{\mathrm{hor}}$ values are in terms of the number of pixels in the horizontal-view image.
It is non-trivial to normalize them into a same scale given their errors in reflecting the true depth -- it is a very rough depth estimation by using $y^{\mathrm{hor}}$ since it is very sensitive to subject detection errors and height difference among subjects.

We first find reliable subset matchings between ${{\bf{x}}^{{\mathop{\rm top}\nolimits} }}$ and ${{\bf{x}}^{{\mathop{\rm hor}\nolimits} }}$  and use them to estimate the scale difference between their corresponding $y$ values. More specifically, we find a scaling factor $\mu$ to scale $y^{\mathrm{top}}$ values to make them comparable to the $y^{\mathrm{hor}}$ values. For this purpose, we use a RANSAC-alike strategy~\cite{FISCHLER1981Random}: for each element $x^{\mathrm{top}}$ in ${{\bf{V}}^{{\mathop{\rm top}\nolimits} }}$, we find the nearest $x^{\mathrm{hor}}$ in ${{\bf{V}}^{{\mathop{\rm hor}\nolimits} }}$. If $|x^{\mathrm{top}}-x^{\mathrm{hor}}|$ is less than a very small threshold value,
we consider $x^{\mathrm{top}}$ and $x^{\mathrm{hor}}$ a matched pair and take the ratio of their corresponding $y$ values and the average of this ratio over all the matched pairs is taken as the scaling factor $\mu$.

With the scaling factor $\mu$, we match ${{\bf{V}}^{{\mathop{\rm top}\nolimits} }}$ and ${{\bf{V}}^{{\mathop{\rm hor}\nolimits}}}$
using dynamic programming (DP)~\cite{Sniedovich2011Dynamic}. Specifically, we define a dissimilarity matrix ${{\bf D}}$ of dimension
	$|{\mathcal{K}}|\times M$, where $D_{ij}$ is the dissimilarity between ${{\bf{V}}_i^{{\mathop{\rm top}\nolimits} }}$ and ${{\bf{V}}_j^{{\mathop{\rm hor}\nolimits}}}$ and it is defined by
	\begin{equation} \label{eq:dis}
D_{ij} 	= \lambda |x_i^{\mathrm{top}}-x_j^{\mathrm{hor}}|+|\mu y_i^{\mathrm{top}}-y_j^{\mathrm{hor}}|,
	\end{equation}
where $\lambda>0$ is a balance factor. Given that ${{\bf{x}}^{{\mathop{\rm top}\nolimits} }}$ and ${{\bf{x}}^{{\mathop{\rm hor}\nolimits} }}$ are both ascending sequences, we use dynamic programming algorithm to search a monotonic path in $\bf D$ from $D_{1,1}$ to $D_{|{\mathcal{K}}|,M}$ to build the matching between ${{\bf{V}}^{{\mathop{\rm top}\nolimits} }}$ and ${{\bf{V}}^{{\mathop{\rm hor}\nolimits}}}$
with minimum total dissimilarities.
If a vector  ${{\bf{V}}^{{\mathop{\rm top}\nolimits} }}$ matches to multiple  vectors in  ${{\bf{V}}^{{\mathop{\rm hor}\nolimits}}}$, we only keep one with the smallest dissimilarity given in Eq.~(\ref{eq:dis}). After that, we check if a vector  ${{\bf{V}}^{{\mathop{\rm hor}\nolimits} }}$ matches to multiple  vectors in  ${{\bf{V}}^{{\mathop{\rm top}\nolimits}}}$ and we keep one with the smallest dissimilarity. These two-step operations will guarantee the resulting matching is one-on-one and we denote $\gamma$ to be the number of final matched pairs. Denote the resulting matched vector subsets to be ${{\bf{V}}_*^{{\mathop{\rm top}\nolimits} }}=({{\bf{x}}_*^{{\mathop{\rm top}\nolimits} }}, {{\bf{y}}_*^{{\mathop{\rm top}\nolimits} }})$ and ${{\bf{V}}_*^{{\mathop{\rm hor}\nolimits} }}=({{\bf{x}}_*^{{\mathop{\rm hor}\nolimits} }}, {{\bf{y}}_*^{{\mathop{\rm hor}\nolimits} }})$, both of dimension $\gamma\times 2$.  We define a matching cost between ${{\bf{V}}^{{\mathop{\rm top}\nolimits} }}$ and ${{\bf{V}}^{{\mathop{\rm hor}\nolimits}}}$ as
\begin{equation} \label{eq:cost}
\phi({{\bf{V}}^{{\mathop{\rm top}\nolimits} }}, {{\bf{V}}^{{\mathop{\rm hor}\nolimits} }})
=  \frac{1}{\gamma}\rho ^{\frac{L}{\gamma}}(\lambda \|{{\bf{x}}_*^{{\mathop{\rm top}\nolimits} }} -{{\bf{x}}_*^{{\mathop{\rm hor}\nolimits} }}\|_1 +\|{{\mu \bf{y}}_*^{{\mathop{\rm top}\nolimits} }} -{{\bf{y}}_*^{{\mathop{\rm hor}\nolimits} }}\|_1),
\end{equation}
where $\rho>1$ is a pre-specified factor and $L = \max(|{\mathcal{K}}|,M)$. In this matching cost, the term $\rho ^{\frac{L}{\gamma}}$ encourages the inclusion of more vector pairs into the final matching, which
is important when we use this matching cost to search for optimal camera location $O$ and view angle $\theta$ to be discussed next.

\subsection{Detecting Horizontal-View Camera and View Angle}
\label{sec:view_ang}

In calculating the matching cost of Eq.~(\ref{eq:cost}), we need to know the horizontal-view camera location $O$ and its view angle $\theta$ to compute the vector ${{\bf{V}}^{{\mathop{\rm top}\nolimits} }}$. In practice, we do not know  $O$ and $\theta$ priorly.  As mentioned earlier, we exhaustively try all possible values for $O$ and $\theta$ and then select the ones that lead to the minimum matching cost $\phi$. The matching with such minimum cost provides us the final cross-view subject association. For view angle $\theta$, we sample the its range $[0, 2\pi)$ uniformly with an interval of $\Delta \theta$ and in the experiments, we will report results by using different sample intervals. For the horizontal-view camera location $O$, we simply try every subject detected in the top-view image as the camera (wearer) location.

\begin{figure}[htbp]
	\centering
	\includegraphics[width=\columnwidth]{occ_search-eps-converted-to.pdf}
	\caption{An illustration of mutual occlusion in the horizontal view. (a) Top-view image and (b) horizontal-view image.}
	\label{fig:occ_search}
\end{figure}

An occlusion in the horizontal-view image indicates that two subjects and the horizontal-view camera are collinear, as shown by $P_1$ and $P_2$ in Fig.~\ref{fig:occ_search}(a).
In this case, the subject with larger depth, i.e., $P_2$,  is not visible in the horizontal view and we simply ignore this occluded subject in vector representation of ${{\bf{V}}^{{\mathop{\rm top}\nolimits} }}$.
In practice, we set a tolerance threshold $\beta=2^\circ$ and if $\langle \mathord{\buildrel{\lower3pt\hbox{$\scriptscriptstyle \rightharpoonup $}} \over {OP_1}}, \mathord{\buildrel{\lower3pt\hbox{$\scriptscriptstyle \rightharpoonup $}}  \over {OP_2}} \rangle<\beta$, we ignore the one with larger depth. The entire cross-view subject association algorithm is summarized in Algorithm~\ref{alg:practical}.

\begin{algorithm}[htbp]
	\small
	\label{alg:practical}
	\caption{Cross-View Subject Association:}
	\KwIn{${\mathcal{T}}$, ${\mathcal{H}}$: Subjects detected in top view and horizontal view respectively; parameters ${\rho}$, ${\lambda}$.}
	\KwOut{Matched vector pair ${{\bf{V}}_*^{{\mathop{\rm top}\nolimits} }}$ and ${{\bf{V}}_*^{{\mathop{\rm hor}\nolimits} }}$; camera location $O^*$; camera-view angle $\theta^*$.}

	Compute the horizontal-view vector ${\bf{V}^{\rm hor}}$ by Eq.~$(\ref{eq:v_hor})$;

	\For{$O\in{\mathcal{T}}$} {
		\For{$\theta\in[0, 2\pi), \mathrm{with~step~length}~\Delta\theta$}{
			Compute the top-view vector ${\bf{V}^{\rm top}}$ by Eq.~$(\ref{eq:v_top})$;

			Estimate scaling $\mu$ as discussed in Section~\ref{sec:vectormatch}.

			Calculate $\bf{D}$ by Eq.~$(\ref{eq:dis})$ using $\mu$ and $\lambda$;

			Calculate ${{\bf{V}}^{{\mathop{\rm top}\nolimits} }}$, ${{\bf{V}}^{{\mathop{\rm hor}\nolimits} }}$ based on $\bf{D}$ by DP algorithm;

			Calculate $\phi$ by Eq.~$(\ref{eq:cost})$
		}
		Find $\theta$ with the minimum $\phi$;
	}
	\Return $O^*$,  $\theta^*$, ${{\bf{V}}_*^{{\mathop{\rm top}\nolimits} }}$ and ${{\bf{V}}_*^{{\mathop{\rm hor}\nolimits} }}$ with the minimum $\phi$.
\end{algorithm}

\section{Experiment}
\label{sec:experiment}

In this section, we first describe the dataset used for performance evaluation and then introduce our experimental results.

\subsection{ Test Dataset}

We do not find publicly available dataset with corresponding top-view and horizontal-view images/videos and ground-truth labeling of the cross-view subject association.
Therefore, we collect a new dataset for performance evaluation.
Specifically, we use a GoPro HERO7 camera (mounted over wearer's head) to take horizontal-view videos and a DJI ``yu" Mavic 2 drone to take top-view videos.
Both cameras were set to have the same fps of 30.
We manually synchronize these videos such that corresponding frames between them are taken at the same time.
We then temporally sample these two videos uniformly to construct frame (image) pairs for our dataset.
Videos are taken at three different sites with different background and the sampling interval is set to 100 frames to ensure the variety of the collected images.
Finally, we obtain 220 image pairs from top and horizontal views, and for both views, the image resolution is $2,688 \times 1,512$.
We label the same persons across two videos on all 220 image pairs.
Note that, this manual labeling is quite labor intensive given the difficulty in identifying persons in the top-view images (see Fig.~\ref{fig:example} for an example).

For evaluating the proposed method more comprehensively, we examine all 220 image pairs and consider the following five attributes: \textbf{Occ}: horizontal-view images containing partially or fully occluded subjects;
\textbf{Hor\_mov}: the horizontal-view images sampled from videos when the camera-wearer moves and rotates his head.
\textbf{Hor\_rot}: the horizontal-view images sampled from videos when the camera-wearer rotates his head.
\textbf{Hor\_sta}: the horizontal-view images sampled from videos when the camera-wearer stays static.
\textbf{TV\_var}: the top-view images sampled from videos when the drone moves up, down and/or change camera-view direction.
Table~\ref{tab:data} shows the number of image pairs with these five attributes, respectively.  Note that some image pairs show multiple attributes listed above.

\begin{table}[htbp]
	\caption{Number of image pairs with the considered attributes.}
	\label{tab:data}
	\footnotesize
	\centering
		\vspace{5 pt}
	\begin{spacing}{1.25}
		\begin{tabular}{lccccc}
			\hline
			Attribute  & \textbf{Occ} & \textbf{Hor\_mov} & \textbf{Hor\_rot} & \textbf{Hor\_sta} & \textbf{TV\_var} \\
			\# image pairs   & 96  & 62       & 124       & 96       & 30      \\ \hline
		\end{tabular}
	\end{spacing}
\end{table}

For each pair of images, we analyze two more properties. One is the number of subjects in an image, which reflects the level of crowdedness. The other is the proportion between the number of shared subjects in two views and the total number of subjects in an image. Both of them can be computed against either the top-view image or the horizontal-view image and their histograms on all 220 image pairs
are shown in Fig.~\ref{fig:data_set}.

\begin{figure}
	\centering
	\includegraphics[width=0.5\textwidth]{data_set-eps-converted-to.pdf}
	\caption{(a) Histogram of the crowdedness in top- and horizontal-view images, respectively. (b) Histogram of the proportion of the shared subjects in top- and horizontal-view images, respectively.}
	\label{fig:data_set}
\end{figure}

In this paper, we use two metrics for performance evaluation. 1) The accuracy in identifying the horizontal-view camera wearer in the top-view image,  and 2)
the precision and recall of cross-view subject association.
We do not include the camera-view angle $\theta$ for evaluation because it is difficult to annotate its ground truth.

\subsection{Experiment Setup}

We implement the proposed method in Matlab and run on a desktop computer with an Intel Core i7 3.4GHz CPU.
We use the general YOLO~\cite{Redmon2016You} detector to detect subjects in the form of bounding boxes in both top-view and horizontal-view images~\footnote{We use the YOLOv3 version detector. For top-view subject detection, we fine-tune the network using 600 top-view human images that have no overlap with our test images.}.
The pre-specified parameters $\rho$ and $\lambda$ are set to 25 and 0.015 respectively.
We will further discuss the influence of these parameters in Section~\ref{sec:Analysis}.

We did not find available methods with code that can directly handle our top- and horizontal-view subject association.  One related work is~\cite{ardeshir2018integrating} for cross-view matching. However, we could not include it directly into comparison because 1) its code is not available to public, and 2) it computes optical flow for $\theta$ and therefore cannot handle a pair of static images in our dataset. Actually, the method in~\cite{ardeshir2018integrating} assumes a certain slope view angle of the top-view camera and use appearance matching for cross association. This is similar to the appearance-matching-based person re-id methods.

In this paper, we chose a recent person re-id method~\cite{sun2018beyond} for comparison. We take each subject detected in the horizontal-view image as query and search it in the set of subjects detected in the top-view image.
We tried two versions of this re-id method: one is retrained from scratch using 1,000 sample subjects collected by ourselves (no overlap with the subjects in our test dataset) and the other is to fine-tune from the version provided in~\cite{sun2018beyond} these 1,000 sample subjects.

\subsection{Results}

We apply the proposed method to all 220 pairs of images in our dataset. We detect the horizontal-view camera wearer on the top-view image as described in \ref{sec:view_ang} and the detection accuracy is
84.1\%. We also use the Cumulative Matching Characteristic (CMC) curve to evaluate the matching accuracy, as shown in Fig.~\ref{fig:rank_curve}(a), where the
horizontal and vertical axes are the CMC rank and the matching accuracy respectively.

\begin{figure}[htbp]
	\centering
	\includegraphics[width=0.45\textwidth]{rank_curve-eps-converted-to.pdf}
	\caption{(a) The CMC curve for horizontal-view camera detection. (b) Precision and recall scores in association,
		where the horizontal axis denotes a precision or recall score $x$, and the vertical coordinate denotes the proportion of image pairs with corresponding precision or recall score that is greater than $x$.}
	\label{fig:rank_curve}
\end{figure}

For a pair of images, we use the precision and recall scores to evaluate the cross-view subject association.
As shown in Table \ref{tab:res}, the average precision and recall scores of our method are 79.6\% and 77.0\% respectively.
In this table, `Ours w $O$' indicates the use of our method by giving the ground-truth camera location $O$.
We can find in this table that the re-id method, either retrained or fune-tuned, produces very poor result,
which confirms the difficulty in using appearance features for the proposed cross-view subject association.

\begin{table}[htbp]
	\caption{Comparative results of different methods. Prec.Avg and Reca.Avg are the average precision and recall scores over all image pairs. Prec.@1 and Reca.@1 are the proportion of image pairs with precision and recall scores of 1, respectively.}
	\vspace{-0.2cm}
	\label{tab:res}
	\centering
	\footnotesize
	\begin{spacing}{1.25}
		\begin{center}
			\begin{tabular*}{0.475\textwidth}[c]{lccccc}
				\Xhline{1pt}
				Method &Prec.Avg	&Reca.Avg 	&Prec.@1	 &Reca.@1 	 \\  	\hline
				Re-id~(fine-tune)  &14.0 &16.0 &- &-\\
				Re-id~(retrain) &22.0 &24.0 &- &- \\
				Ours (w $O$)  &86.6 &84.2 &66.4 &57.7\\
				Ours &79.6 &77.0 &60.0 &50.9 \\
				\hline
			\end{tabular*}
		\end{center}
	\end{spacing}
\end{table}

We also calculate the proportion of all the image pairs with precision or recall score of 1 (Prec.@1 and Reca.@1). They reach 60.0\% and 50.9\% respectively.
The distributions of these two scores on all 220 image pairs are shown in Fig.~\ref{fig:rank_curve}(b).
In Table~\ref{tab:att_per}, we report the evaluation results on different subsets with respective attributes.
We can see that the proposed method is not sensitive to the motion of both top-view and horizontal-view cameras, which is highly desirable for motion-camera applications.

\begin{table}[htbp]
	\caption{Comparative results on the subsets with different attributes.}
	\label{tab:att_per}
	\centering
	\footnotesize
	\begin{spacing}{1.25}
		\begin{tabular*}{0.45\textwidth}{lccccc}
			\Xhline{1pt}
			Attr  & \textbf{Occ} & \textbf{Hor\_mov} & \textbf{Hor\_rot} & \textbf{Hor\_sta} & \textbf{TV\_var} \\ \hline
			Prec.Avg   &76.6   &78.3        &80.5        &78.4       &53.3      \\
			Rec.Avg   &74.5   &74.9       &77.9      &75.7       &53.3     \\ \hline
		\end{tabular*}
	\end{spacing}
\end{table}

\subsection{Ablation Studies}

\label{sec:Analysis}

\textbf{Step Length for $\theta$.}
We study the influence of the value $\Delta \theta$, the step length for searching optimal camera view angle $\theta$ in the range $[0, 2\pi)$. We set the value of $\Delta\theta$ to 1$^\circ$, 5$^\circ$ and 10$^\circ$, respectively and the association results are shown in Table~\ref{tab:para-2}.
As expected, $\Delta\theta=1^\circ$ leads to the highest performance, although a larger step length, such as $\Delta\theta=5^\circ$ also produces acceptable results.

\begin{table}[htbp]
	\caption{Results by using different values for $\Delta \theta$.}
	\vspace{-0.cm}
	\label{tab:para-2}
	\centering
	\footnotesize
	\begin{spacing}{1.25}
		\begin{center}
			\begin{tabular*}{0.45\textwidth}[c]{lccccc}
				\Xhline{1pt}
				Step length &Prec.Avg	 &Reca.Avg 	&Prec.@1	 &Reca.@1 	 \\  	\hline
				$\Delta \theta$= 1\degree &\textbf{79.6} &\textbf{77.0} & \textbf{60.0} &\textbf{50.9} \\
				$\Delta \theta$= 5\degree &78.8 &76.9 &59.6 &51.8 \\
				$\Delta \theta$ = 10\degree &72.5 &71.1 &53.2 &46.8 \\

				\hline
			\end{tabular*}
		\end{center}
	\end{spacing}
\end{table}

\textbf{Vector representation.}
Next we compare the association results using different vector representation methods as shown in Table~\ref{tab:vec_rep}. The first row denotes that we represent the subjects in two views by one-dimensional vectors $\bf{x}^\mathrm{top}$ and $\bf{x}^\mathrm{hor}$ respectively.
The second row denotes that we represent the subjects in two views by one-dimensional vectors $\bf{y}^\mathrm{top}$ and $\bf{y}^\mathrm{hor}$, respectively, which are simply normalized to the range $[0,1]$ to make them comparable.
The third row denotes that we combine the one-dimensional vectors for the first and second rows to represent each view, which differs from our proposed method (the fourth row of Table~\ref{tab:vec_rep}) only on the normalization of $\bf{y}^\mathrm{top}$ and $\bf{y}^\mathrm{hor}$ -- our proposed method uses a RANSAC strategy.
By comparing the results in the third and fourth rows, we can see that the use of RANSAC strategy for estimating the scaling factor $\mu$ does improve the final association performance.
The results in the first and second rows show that using only one dimension of the proposed vector representation cannot achieve performance as good as the proposed method that combines both dimensions.
We can also see that $\bf{x}^\mathrm{top}$ and $\bf{x}^\mathrm{hor}$  provides more accurate information than  $\bf{y}^\mathrm{top}$ and $\bf{y}^\mathrm{hor}$ when used for cross-view subject association.

\begin{table}[htbp]
	\caption{Comparative study of using different vector representations. $\bf{\bar y}^\mathrm{top}$ and $\bf{\bar y}^\mathrm{hor}$ are normalized results of $\bf{y}^\mathrm{top}$ and $\bf{y}^\mathrm{hor}$, respectively, by simply scaled to $[0,1]$.}
	\vspace{-0.2cm}
	\label{tab:vec_rep}
	\centering
	\footnotesize
	\begin{spacing}{1.25}
		\begin{center}
			\begin{tabular*}{0.475\textwidth}[c]{lccccc}
				\Xhline{1pt}
				Vector	&Prec.Avg	 &Reca.Avg 	&Prec.@1	 &Reca.@1 	 \\  	\hline
				$\bf{x}^\mathrm{top,hor}$  &63.2 &61.6 &41.8 &35.9\\
				$\bf{\bar y}^\mathrm{top,hor}$ &23.4 &13.4 &6.8 &0.9 \\
				$\bf{x}^\mathrm{top,hor}$,  $\bf{\bar y}^\mathrm{top,hor}$   &67.7 &66.6 &46.4 &42.7 \\
				Ours &\textbf{79.6} &\textbf{77.0} & \textbf{60.0} &\textbf{50.9} 	\\
				\hline
			\end{tabular*}
		\end{center}
	\end{spacing}
\end{table}

\textbf{Parameters selection.}
There are two free parameters $\rho$ and $\lambda$ in Eq.~(\ref{eq:cost}). We select different values for them and see their influence to the final association performance.
Table~\ref{tab:para} reports the results by varying one of these two parameters while fixing the other one.
We can see that the final association precision and recall scores are not very sensitive to the selected values of these two parameters.

\begin{table}[htbp]
	\caption{Results by varying values of $\rho$ and $\lambda$.}
	\vspace{-0.2cm}
	\label{tab:para}
	\centering
	\footnotesize
	\begin{spacing}{1.25}
		\begin{center}
			\begin{tabular*}{0.5\textwidth}[c]{lcc|lcc}
				\Xhline{1pt}
				$\rho$	&Prec.Avg	 &Reca.Avg &$\lambda$	&Prec.Avg	  &Reca.Avg	 \\  	\hline
				$\rho=10$  &76.4 &73.7
				&$\lambda = 0.005$ &79.2 &76.6\\
				$\rho=25$ &\textbf{79.6} &\textbf{77.0}  & $\lambda = 0.015$ &\textbf{79.6} &\textbf{77.0} \\
				$\rho=40$  &79.1 &76.7 & $\lambda = 0.025$ &78.4 &75.9\\

				\Xhline{1pt}
			\end{tabular*}
		\end{center}
	\end{spacing}
\end{table}

{\bf{Detection method.}}
In order to analyze the influence of subjects detection's accuracy to the proposed cross-view association, we tried the use of different subject detections. As shown in Table~\ref{tab:det},
in the first row, we use manually annotated bounding boxes of each subject on both views for the proposed association.
In the second and third rows, we use manually annotated subjects on top-view images and horizontal-view images, respectively,  while using automatically detected subjects~\cite{Redmon2016You}
on the other-view images.
In the fourth row, we automatically detect subjects in both views first, and then only keep those that show an IoU$>0.5$ (Intersection over Union) against a manually annotated subject, in terms of their bounding boxes.
We can see that the use of manually annotated subjects produces much better cross-view subject association. This indicates that further efforts on improving subject detection will benefit the association.

\begin{table}[htbp]
	\caption{Results by using different methods for subject detection.}
	\vspace{-0.5cm}
	\label{tab:det}
	\centering
	\footnotesize
	\begin{spacing}{1.25}
		\begin{center}
			\begin{tabular*}{0.5\textwidth}[c]{lccccc}
				\Xhline{1pt}
				Subjects detection &Prec.Avg	 &Reca.Avg 	&Prec.@1	 &Reca.@1 	 \\  	\hline
				Manual   &{83.5} &{80.5} &\textbf{76.8} &\textbf{61.4}\\
				Manual-Top   &\textbf{84.8} &\textbf{82.0} &70.5 &59.1 \\
				Manual-Hor   &80.6 &77.4 &69.1 &55.5 \\
				Automatic w selection &80.7 &76.1 &69.6 &52.7 \\
				Automatic (Ours)  &79.6 &77.0 &60.0 &50.9\\
				\hline
			\end{tabular*}
		\end{center}
	\end{spacing}
\end{table}

\begin{figure}[htbp]
	\centering
	\includegraphics[width=0.45\textwidth]{pre_rec_data-eps-converted-to.pdf}
	\caption{(a) Association performance for image pairs with different number of associated subjects. (b) Association performance for image pairs with different proportion of associated subjects.}
	\label{fig:pre_rec_num_scale}
\end{figure}

\begin{table}[htbp]
	\caption{Association results using the proposed method with and without handling occlusions.}
	\label{tab:occ}
	\centering
	\footnotesize
	\begin{spacing}{1.25}
		\begin{center}
			\begin{tabular*}{0.45\textwidth}[c]{ l  c c c c}
				\Xhline{1pt}
				\multirow{2}{*}{{Method}}
				&\multicolumn{2}{c}{Whole dataset} 	&\multicolumn{2}{c}{Occ subset}	  \\ \cline{2-5}
				&{Prec.Avg}	 &Reca.Avg		&Prec.Avg	  &Reca.Avg		\\  	\hline
				Ours 	&79.6	 &77.0 		&76.6	 &74.5	  	\\
				Ours(w/o occ)    &65.2	 &65.1		&46.1	 &46.8	      \\
				\Xhline{1pt}
			\end{tabular*}
		\end{center}
	\end{spacing}\vspace{-0.5cm}
\end{table}

\begin{figure*}[htbp]
	\centering
	\includegraphics[width=0.95\textwidth]{dis_occ_det-eps-converted-to.pdf}
	\caption{Row 1: Two sample results on image pairs with occlusions. Row 2: Two sample results with large number of unshared subjects between two views.
		Vector sets ${{\bf{V}}^{{\mathop{\rm top}\nolimits} }}$ and ${{\bf{V}}^{{\mathop{\rm hor}\nolimits}}}$ are shown in the top-right corner of every image.}
	\label{fig:dis_occ_det}
\end{figure*}

\begin{figure*}[htbp]
	\centering
	\includegraphics[width=0.95\textwidth]{dis_fail-eps-converted-to.pdf}
	\caption{Two failure cases.}
	\label{fig:dis_fail}
\end{figure*}

\subsection{Discussion}

\textbf{Number of associated subjects.}
We investigate the correlation between the association performance and the number of associated subjects.
Figure~\ref{fig:pre_rec_num_scale}(a) shows the average association performance on the image pairs with different number of associated subjects.
We can see that the association results get worse when the number of associated subjects is too high or too low.
When there are too many associated subjects, the crowded subjects in the horizontal view may prevent the accurate detection of subjects.
When there are two few subjects, the constructed vector representation is not sufficiently discriminative to locate the camera location $O$ and camera-view angle $\theta$.
Figure~\ref{fig:pre_rec_num_scale}(b) shows the average association performance on the image pairs with different proportions of associated subjects.
More specifically, the performance at $x$ along the horizontal axis is the average precision/recall score on all the image pairs with the proportion of associated subjects (to
the total number of subjects in the top-view image) less than $x$.
This confirms that on the images with higher such proportion, the association can be more reliable.

\textbf{Occlusion.}
Occlusions are very common, as shown in  Table~\ref{tab:data}. Table~\ref{tab:occ} shows the association results on the entire dataset and the subset of data with occlusions, by using the proposed method with and without the step of identifying and ignoring occluded subjects. We can see that our simple strategy for handling occlusion can significantly improve the association performance on the image pairs with occlusions.
Sample results on image pairs with occlusions are shown in the top row of Fig.~\ref{fig:dis_occ_det}, where associated subjects bear same number labels. We can see that occlusions occur more often when 1) the subjects are crowded, and 2) one subject is very close to the horizontal-view camera.

\textbf{Proportion of shared subjects.}
It is a common situation that many subjects in two views are not the same persons. In this case, the shared subjects may only count for a small proportion in both top- and horizontal-views.
Two examples are shown in the second row of Fig.~\ref{fig:dis_occ_det}. In the left, we show a case where many subjects in the top view are not in the field of view of the horizontal-view camera.
In the right, we show a case where many subjects in the horizontal view are too far from the horizontal-view camera and not covered by the top-view camera.
We can see that the proposed method can handle these two cases very well, by exploring the spatial distribution of the shared subjects.

\textbf{Failure case.}
At last, we give two failure cases as shown in Fig.~\ref{fig:dis_fail} -- one caused by the error in subject detection (blue boxes) and the other is caused by the close distance of multiple subjects, e.g, subjects 3,4 and 5,
in either top or horizontal view, which lead to error detection of occlusions and incorrect vector representations.

\section{Conclusion}
\label{sec:conclusion}
In this paper, we developed a new method to associate multiple subjects across top-view and horizontal-view images by modeling and matching the subjects' spatial distributions. We constructed a vector representation for all the detected subjects in the horizontal-view image and another vector representation for all the detected subjects in the top-view image that are located in the field of view of the horizontal-view camera.  These two vector representations are then matched for cross-view subject association. We proposed a new matching cost function with which we can further optimize for the location and view angle of the horizontal-view camera in the top-view image. We collected a new dataset, as well as manually labeled ground-truth cross-view subject association, and experimental results on this dataset are very promising.

{\small
\bibliographystyle{ieee}