\section{Introduction}

In this paper, we study a couple of fine questions related to the homogenization of the following Hamilton-Jacobi equation:
\begin{equation}\label{HJ-ep}
\begin{cases}
u^\varepsilon_t + a\left(\frac{x}{\varepsilon}\right) |Du^\varepsilon| = 0 \quad &\text{ in } \mathbb{R}^n \times (0,\infty),\\
u^\varepsilon(x,0) = g(x) \quad &\text{ on } \mathbb{R}^n.
\end{cases}
\end{equation}
Here, $g\in {\rm BUC\,}(\mathbb{R}^n) \cap {\rm Lip\,}(\mathbb{R}^n)$ is the initial data, where ${\rm BUC\,}(\mathbb{R}^n)$ is the space of bounded, uniformly continuous functions on $\mathbb{R}^n$. The coefficient $a: \mathbb{R}^n \to \mathbb{R}$ determines the normal velocity in the underlying front propagation model; the small number $\varepsilon \in (0,1)$ is the spatial scale of variations in $a$. This problem hence models front propagations in oscillatory environment. We assume throughout the paper that $a$ is a continuous,  $\mathbb{Z}^n$-periodic and non-constant positive function.

The qualitative homogenization of \eqref{HJ-ep} fits in the classical and standard framework (see \cite{LPV, Ev1} for example).
Here is a brief review of the result. As $\varepsilon \to 0$, the solution $u^\varepsilon$ of the above problem converges, locally uniformly in $\mathbb{R}^n \times [0, \infty)$, to the solution of the effective or homogenized problem:
\begin{equation}\label{HJ}
\begin{cases}
u_t + \overline{H}(Du) = 0 \quad &\text{ in } \mathbb{R}^n \times (0,\infty),\\
u(x,0) = g(x) \quad &\text{ on } \mathbb{R}^n.
\end{cases}
\end{equation}
Here, $\overline{H}$ is the so-called effective Hamiltonian. For each $p\in \mathbb{R}^n$, $\overline{H}(p)$ is the unique real  number such that the following equation admits a continuous viscosity solution
\[
 {\rm (E)}_p \qquad a(y) |p +Dv_p(y)| = \overline{H}(p) \quad \text{ in } \mathbb{T}^n.
\]
This is the well-known cell problem.

An important feature of problem \eqref{HJ-ep}, due to its special structure, is that the effective Hamiltonian $\overline{H}$ is also intrinsically determined by a shape theorem which says the large time average of the reachable set  from the origin, determined by the environment function $a$, converges to a centrosymmetric convex set. It turns out that $\overline{H}$ is the support function of this convex set; see \eqref{eq:HD} below.

In this paper, we aim to study an inverse shape theorem: what class of convex sets are admissible as the limiting shapes of the averaged reachable sets for some environment function $a$?  We show that for $n \geq 3$, all centrally symmetric polytopes with rational coordinates and nonempty interior are admissible.
The second objective of this paper is to obtain optimal convergence rate of the homogenization problem, and we show this is possible in general when $\overline{H}$ is determined by centrally symmetric polytopes type limiting shapes.

\smallskip

Let us now present the reachable set framework and our main results.

\subsection{Admissible paths and reachable sets}

For $t>0$, let $\mathcal{A}_{0,t}$ be the set of admissible paths defined as follows
\begin{equation*}
\mathcal{A}_{0,t}=\left\{\gamma \in W^{1,\infty}([0,t],\mathbb{R}^n) \,:\, |\dot \gamma(s)| \leq a(\gamma(s)) \quad \text{ for a.e. } s \in [0,t] \right\}.
\end{equation*}
The reachable set $\mathcal{R}_t(x)$ at time $t>0$ emanating from $x\in \mathbb{R}^n$ at time $0$ is defined as following
\[
\mathcal{R}_t(x)=\left\{ y\in \mathbb{R}^n\,:\, \text{ there exists $\gamma \in \mathcal{A}_{0,t}$ such that } \gamma(0)=x, \gamma(t)=y \right\}.
\]
Note that since $a$ is periodic and positive, there exist a lower bound $\alpha >0$ and an upper bound $\beta > \alpha$ such that, for all $x\in \mathbb{R}^n$,
\begin{equation*}
0< \alpha \le a(x) \le \beta.
\end{equation*}
It is then easy to verify that, for any fixed $x \in \mathbb{R}^n$, the set $\mathcal{R}_t(x)$ is increasing with respect to $t$. We are interested in the large time average of $\mathcal{R}_t(x)$, that is, the behavior of
\[
\frac{\mathcal{R}_t(x)}{t} \qquad \text{ as } t \to \infty.
\]
We also denote by $\mathcal{R}_t(Y)$ the union of $\mathcal{R}_t(x)$, $x\in Y$. To describe this behavior, we need the following notions.
Let $\mathscr{C}$ denote the set of non-empty compact subsets of $\mathbb{R}^n$.
The Hausdorff distance between $E$ and $F$ in $\mathscr{C}$ is defined by
\begin{align*}
\rho(E,F) &= \max \left \{ \sup_{x\in F} \inf_{y \in E} |x-y|, \ \sup_{y \in E} \inf_{x \in F} |x-y|\right\}\\
&=\inf \left\{ s>0 \,:\, F \subset E + \overline{B}_s,\, E \subset F + \overline{B}_s \right\}.
\end{align*}
It is well known that $(\mathscr{C},\rho)$ is a complete metric space. The existence of the limiting shape below is well known to experts.
\begin{lem}[A Shape Theorem]\label{lem:R-t-conv}
There exists  a compact and convex set $D \subset \mathbb{R}^n$ such that
\begin{equation}\label{R-t-conv}
\lim_{t \to \infty} \frac{\mathcal{R}_t(x)}{t} = D \quad \text{ in } \quad (\mathscr{C},\rho).
\end{equation}
\end{lem}
If needed, we write $D_a$ instead of $D$ to demonstrate the clear dependence of the limit \eqref{R-t-conv} in $a$. An important property of \eqref{HJ-ep} is, as mentioned before, the effective Hamiltonian $\overline{H}$ of the homogenized problem, which is usually determined by ${\rm (E)}_p$, is the support function of the above limit shape $D$. This means: for $p \in \mathbb{R}^n$,
\begin{equation}\label{eq:HD}
\overline{H}(p) = \sup_{q \in D} \, p\cdot q.
\end{equation}
We then have the following result.

\begin{thm}[Qualitative homogenization result for \eqref{HJ-ep}]\label{thm:hom}
For each $\varepsilon>0$, let $u^\varepsilon$ be the unique viscosity solution to \eqref{HJ-ep}.
Let $u$ be the solution of \eqref{HJ}, with $\overline{H}$ defined by \eqref{eq:HD}. Then, as $\varepsilon \to 0$, $u^\varepsilon$ converges locally uniformly on $\mathbb{R}^n \times [0,\infty)$ to the solution $u$ of \eqref{HJ}.\end{thm}

The qualitative homogenization result for \eqref{HJ-ep} with $\overline{H}$ determined by the cell problem ${\rm(E)}_p$, even in a more general framework, is quite well-known in the literature (see \cite{LPV, Ev1} for example). The result above shows that in the specific case of \eqref{HJ-ep}, $\overline{H}$ can also be characterized by \eqref{eq:HD}. We used both forms of $\overline{H}$ in our analysis.

\subsection{Main results}
We proceed to study deeper properties of $\overline{H}$ and the rate of convergence of $u^\varepsilon$ to $u$. In the sequel, the usual $n$-dimensional flat torus is denoted by $\mathbb{T}^n= \mathbb{R}^n / \mathbb{Z}^n$, and all polytopes are assumed to be centered at the origin. We say a non-zero vector $p \in \mathbb{R}^n$ is rational if  $sp$ has integer coordinates for some $s>0$.

\begin{thm}\label{thm:polytope}
Assume that $n \geq 3$.
Let $P \subset \mathbb{R}^n$ be a centrally symmetric polytope with rational vertices and nonempty interior.
Then, there exists an environment function $a \in C^\infty(\mathbb{T}^n, (0,\infty))$ such that $D_a=P$.
\end{thm}

The set of centrally symmetric polytopes with rational vertices and nonempty interior is important since it is dense in the set of centrally symmetric convex sets with nonempty interior.
The same result has also been proved   in  \cite{BB2006, J2009} in the equivalent form of stable norms.
Constructions in these works are all based on Hedlund's original idea \cite{Hed} where a cubic $D$ is constructed.
The main difference lies in the methods/computations involved in verifying that $D$ is indeed the desired polytope.
Our proof based on PDE/weak KAM type approaches is much simpler than those delicate  geometric calculations in \cite{Hed,BB2006,J2009}.
This result  belongs to the ongoing project of studies of inverse problems in periodic homogenization of Hamilton-Jacobi equations (see \cite{LTY, JTY, TY}).
 A very interesting question is whether every  centrally symmetric convex set with nonempty interior is realizable within the class of $a \in C(\mathbb{T}^n,(0,\infty))$.
 A natural thought is to achieve this by approximations.
 However, our construction actually shows that $\overline H$ is not stable under convergence of $a$ that is weaker than the uniform convergence (see Remark \ref{rem:unstability}).
In addition, we would like to point out that, when $n=2$, polytopes are not realizable by $a\in C^1(\Bbb T^2,(0,\infty))$, because it is proved in  \cite{Car} that  $a \in C^1$ yields strictly convex limiting shape $D$ (equivalently, the dual set $\left\{\overline H=1\right\}$ is $C^1$). As long as the continuous class of $a$ is considered,  we believe that the above theorem should hold in two dimensions.
We will investigate these problems in the future.

\smallskip

Next is our optimal rate of convergence result in case that the effective front is a centrally symmetric polytope with nonempty interior.

\begin{thm}\label{thm:rate2}
Let $P \subset \mathbb{R}^n$ be a centrally symmetric polytope with nonempty interior.
Let $a \in C(\mathbb{T}^n, (0,\infty))$ be such that $D_a=P$.
Then, there exists a constant $C>0$ depending only on $a$ such that
\[
\|u^\varepsilon -u\|_{L^\infty(\mathbb{R}^n \times [0,\infty))} \leq C \varepsilon.
\]
\end{thm}

The first quantitative result in periodic homogenization of Hamilton-Jacobi equations was obtained in \cite{CDI} with rate $O(\varepsilon^{1/3})$.
Afterwards, optimal rate of convergences $O(\varepsilon)$ were derived in various convex settings in \cite{MTY}.
In particular, if the Hamiltonian $H=H(y,p)$ is homogeneous in the $p$ variable (e.g., $H(y,p)=a(y)|p|$ in our case) in two dimensions,  $O(\varepsilon)$ holds for any Lipschitz continuous initial data $g(x)$ (see  \cite[Theorem 1.2]{MTY}).
If $n\geq 3$, we believe that extra assumptions on $\overline H$ are necessary in general in order to obtain $O(\varepsilon)$ although an example with a fractional convergence rate is still elusive.
This theorem is a next development along the line of \cite{MTY} for $n\geq 3$. See Theorem 1.1 in \cite{MTY} for other more ``generic" assumptions.

\begin{rem}  The assumption  in Theorem \ref{thm:polytope} that the polytope $D$ has rational vertices is necessary in our constructive proof.
It is not clear to us,  for smooth $a$,  whether $D_a$ being a polytope automatically implies that all vertices are rational vectors.
According to \cite{BIK}, if $D$ is a polytope, no vertex could be an irrational vector (i.e., its coordinates are linearly independent over $\mathbb{Q}$).
However, there are vectors which are neither rational nor irrational, for example $q=(1,2,\sqrt{3}) \in \mathbb{R}^3$.
\end{rem}

\subsection*{Outline of the paper}
In Section \ref{sec:Hed}, we give the proof of Theorem \ref{thm:polytope}, which is a generalization of the classical Hedlund example.
In Section \ref{sec:op}, we prove the optimal convergence result (Theorem \ref{thm:rate2}).
In the Appendix, we present the results on large time average of reachable sets and the qualitative homogenization result (proofs of Lemma \ref{lem:R-t-conv} and Theorem \ref{thm:hom}).
Although Lemma \ref{lem:R-t-conv} and Theorem \ref{thm:hom} are classical and standard in the literature, for readers' convenience,  we provide their proofs based on the reachable set framework, which are quantifiable and more in line with the approaches used in this paper.

\subsection*{Notations}
Let $Y$ be the unit cell $[0,1]^n$ and $\tilde Y = - Y= [-1,0]^n$.
For any $x\in \mathbb{R}^n$, denote by $([x],\hat x)$ the unique pair in $\mathbb{Z}^n \times [0,1)^n$ such that $x=[x]+\hat x$.
Denote by $B_r(x), \overline{B}_r(x)$ the open ball, closed ball of center $x$, radius $r>0$ in $\mathbb{R}^n$, respectively.
We write $B_r=B_r(0), \overline{B}_r= \overline{B}_r(0)$ for short.
For $E,F \subset \mathbb{R}^n$ nonempty and $c\in \mathbb{R}$, we set $E+F = \{x+y\,:\,x\in E, y\in F\}$, and $cF=\{cx\,:\,x\in F\}$.

\section{A generalization of the classical Hedlund example} \label{sec:Hed}
In this section, we assume that $n \geq 3$ and we provide a proof of Theorem \ref{thm:polytope}. Our goal is: we construct an $a\in C^\infty(\mathbb{T}^n, (0,\infty))$, such that the effective Hamiltonian $\overline{H}$ determined by ${\rm (E)}_p$ corresponds to the support function of the polytope $P$ given in Theorem \ref{thm:polytope}.

\begin{proof}[Proof of Theorem \ref{thm:polytope}]
Assume that the vertices of $P$ are $\pm q_1, \pm q_2, \ldots, \pm q_m$, which are rational vectors.
Denote by $L_i=\{tq_i\,:\,  t\in  \mathbb{R} \}$ for $1 \leq i \leq m$.
Since $P$ is convex and has nonempty interior, $q_1, q_2,\ldots, q_m$ are mutually non-parallel and
\[
\text{span} \{q_1, q_2, \ldots, q_m\} = \mathbb{R}^n.
\]
As a result,
\begin{equation*}
\theta :=\min_{|p|=1}\max_{1\leq i\leq m}|p\cdot q_i|>0.
\end{equation*}

\emph{Step 1: Construction of $a$.} Let $x_1=0$.
Iteratively, for $k\leq m-1$, choose
\[
x_{k+1}\in (0,1)^n\backslash \bigcup_{i=1}^{k}\left\{x_i+sq_{k+1}+tq_i+\mathbb{Z}^n:\ s,t\in \mathbb{R}\right\}.
\]
Then for such selected $x_1,x_2, \ldots, x_m\in (0,1)^n$,  we have that  for $i \neq j$,
\[
(x_i+L_i+\mathbb{Z}^n)\cap (x_j+L_j+\mathbb{Z}^n)=\emptyset.
\]
Due to the fact that $\{q_i\}$ are rational vectors, the projection of $\{x_i + L_i + \mathbb{Z}^n\}$ to the flat torus $\mathbb{T}^n$ is a closed orbit.
As a result, we can choose a  sufficiently small number $\delta \in (0,1/3)$  so that, for $i \neq j$,
\[
T_{\delta,i} \cap T_{\delta,j} = \emptyset,
\]
where
\[
T_{\delta, i} = \{ x\in \mathbb{R}^n\,:\, d(x, x_i + L_i + \mathbb{Z}^n) \leq \delta \}.
\]

Choose a smooth $\mathbb{Z}^n$-periodic function $a:\mathbb{R}^n \to (0,\infty)$ such that
\[
\begin{cases}
a(x)=A|q_i|   \qquad &\text{on $x_i+L_i+\mathbb{Z}^n$ for $1\leq i \leq m$},\\
1\leq a(x)\leq A|q_i| \qquad &\text{on $T_{\delta, i}$ for $1\leq i \leq m$},\\
a(x)=1  \qquad &\text{on $\mathbb{R}^n\backslash \bigcup_{i=1}^{m}T_{\delta, i}$}.
\end{cases}
\]
Here, $A>0$ is a large positive constant to be determined later.
\smallskip

Next, for every unit vector $|p|=1$ and $1\leq i\leq m$, write
\[
p_{i}^{\perp}=p-{(p\cdot q_i)\over |q_i|^2}q_i,
\]
which is the projection of $p$ on the $(n-1)$-dimensional Euclidean space that is perpendicular to $q_i$.
Apparently, we can construct  a smooth $\mathbb{Z}^n$-periodic function $\phi$  satisfying that
\[
D\phi(x)=-p_{i}^{\perp} \qquad \text{in $T_{{\delta}, i}$ for all $1\leq i\leq m$},
\]
and
\[
\|D\phi\|_{L^\infty}\leq C_{\delta},
\]
for a constant  $C_{\delta}>0$ depending only on $\delta$, and $q_1, q_2,\ldots, q_m$.
We now pick $A$ such that
\[
A\geq \max\left\{{1+C_{\delta}\over \theta}, {1\over \min_{1\leq i\leq m}|q_i|}\right\}.
\]

\medskip

\emph{Step 2: Characterization of the effective Hamiltonian}. Let $\overline{H}$ be determined by ${\rm (E)}_p$ with $a$ defined above. We claim that
\begin{equation}\label{H-bar-goal}
\overline H(p)=A\max_{1\leq i\leq m} |q_i\cdot p| \qquad \text{ for all $p\in \mathbb{R}^n$}.
\end{equation}

\smallskip

We only need to prove this claim for unit vectors $|p|=1$. Fix such a $p$. Firstly, by using $\phi$ above, it is clear that
\begin{align*}
\overline H(p)&\leq \max_{x\in \mathbb{R}^n}a(x)|p+D\phi(x)|\\
&\leq \max\left\{A\max_{1\leq i\leq m} |q_i\cdot p|,  1+C_{\delta}\right\}
=A\max_{1\leq i\leq m} |q_i\cdot p|.
\end{align*}
Secondly, let $v=v_p$ be a solution of (E)$_p$.
We assume $v \in C^1(\mathbb{T}^n)$ (to make this rigorous, one needs to do convolution with a standard mollifier, but we omit it here).
 Then, for each $1 \leq i \leq m$,
\[
A|q_i||p+Dv(x)|=\overline H(p)  \quad \text{for $x\in x_i+L_i+\mathbb{Z}^n$}.
\]
Denote by $u(x)=p\cdot x+v(x)$ for $x\in \mathbb{R}^n$.
Choose $m\in \mathbb{Z}$ such that $mq_i\in \mathbb{Z}^n$. Then
\[
u(x_i+mq_i)-u(x_i)=mp\cdot q_i.
\]
Since $|u(x_i+mq_i)-u(x_i)|\leq  m|q_i|\max_{x\in x_i+L_i}|Du(x)|$, we deduce that
\[
\overline H(p)\geq A|p\cdot q_i|.
\]
Accordingly, \eqref{H-bar-goal} holds true.

\medskip

\emph{Step 3: The corresponding shape $D$}. Let $a_A(x)={a(x)\over A}$. Then by scaling the result of Step 2, the effective Hamiltonian $\overline{H}_A(p)$ determined by ${\rm(E)}_p$ with $a$ replaced by $a_A$ satisfies
\begin{equation*}
\overline{H}_A (p) = \max_{1\leq i\leq m} |q_i\cdot p| = \max_{q\in \{\pm q_1, \cdots, \pm q_m\}} q\cdot p = \max_{q \in P} q\cdot p, \qquad \text{ for all $p\in \mathbb{R}^n$}.
\end{equation*}
Compare with Theorem \ref{thm:hom} and in particular the relation \eqref{eq:HD}, we conclude that $P = D_{a_A}$. This completes the proof.
\end{proof}

 \begin{rem}\label{rem:unstability}
From the constructions in the proof, we observe that, by properly choosing those rational vectors $q_i$ and $\delta$, it is not hard to construct a sequence $\{a_m(\cdot)\} \subset C^{\infty}(\mathbb{T}^n)$ such that
\[
0<a_m\leq 1, \qquad \lim_{m\to \infty}a_m(x)=0 \quad \text{ for a.e. $x\in \mathbb{T}^n$},
\]
and
\[
\lim_{m \to \infty}\overline H_m(p)=|p| \quad \text{locally uniformly in $\mathbb{R}^n$}.
\]
\end{rem}

\section{The optimal rate of convergence} \label{sec:op}
This section is strongly motivated by \cite{MTY}.
It is clear from the proof of Theorem \ref{thm:hom} in the Appendix that, in order to quantify the rate of convergence of $u^\varepsilon$ to $u$, we need a quantitative version of \eqref{K-conv}.
The following result, which is similar to \cite[Lemma 2.1]{MTY}, shows this connection.

\begin{lem}\label{lem:rate-connection}
Let $D$ be the limit shape of Lemma \ref{lem:R-t-conv}. Assume that there exists $C>0$ such that
\begin{equation}\label{R-t-rate}
\rho\left(\frac{\mathcal{R}_t(Y)}{t}, D \right) \leq \frac{C}{t} \quad \text{ for } t>0.
\end{equation}
Then, there exists $C>0$ such that
\begin{equation}
\label{eq:hom_err}
\|u^\varepsilon-u\|_{L^\infty(\mathbb{R}^n \times [0,\infty))} \leq C \varepsilon.
\end{equation}
\end{lem}

\begin{proof}
By \eqref{R-t-rate},
\[
 \rho \left( \frac{\mathcal{R}_{t/\varepsilon}(Y)}{t/\varepsilon},D\right)= \rho \left( \varepsilon \frac{\mathcal{R}_{t/\varepsilon}(Y)}{t},D\right) \leq \frac{C}{t/\varepsilon} = \frac{C\varepsilon}{t},
\]
which implies
\[
 \rho \left( \varepsilon \mathcal{R}_{t/\varepsilon}(Y),tD\right) \leq C \varepsilon.
\]
We then use this result in \eqref{u-HL} and \eqref{u-ep-rep} to conclude the proof.
\end{proof}

\subsection{Proof of Theorem \ref{thm:rate2}}
We first give a definition on backward characteristics (see calibrated curves in \cite{Fa}, backward characteristics in \cite[Chapter 5]{Tr}).

\begin{defn} \label{def:back}
For each $p\in  \mathbb{R}^n$,  let $v_p \in {\rm Lip\,}(\mathbb{T}^n)$ be a viscosity solution to the cell problem {\rm (E)$_p$}.
Then, $\xi: (-\infty, 0]\to \mathbb{R}^n$ is called a backward characteristic of $v_p$ if
\begin{equation}\label{char}
p\cdot \xi (t_1)+v_p(\xi (t_1))-p\cdot \xi (t_2)-v_p(\xi (t_2))=\int_{t_2}^{t_1}L(\xi (t),\dot \xi(t))+\overline H(p)\,dt
\end{equation}
for all $t_2<t_1\leq 0$. Here $L$ is the Legendre transform of the Hamiltonian $H$.
\end{defn}
\noindent For our problem \eqref{HJ-ep}, the Hamiltonian $H(y,p)=a(y)|p|$ only has linear growth at infinity. The corresponding Lagrangian $L$ is defined by
\begin{equation*}
L(y,v) = \sup_{p \in \mathbb{R}^n} \left( v\cdot p - H(y,p) \right) = \begin{cases}
0 &\qquad |v| \le a(y),\\
+\infty &\qquad |v| > a(y).
\end{cases}
\end{equation*}
By approximating $H$ by smooth and strictly convex Hamiltonians, it is easy to show that for each $p\in \mathbb{R}^n$,
there exists a viscosity solution $v_p \in {\rm Lip\,}(\mathbb{T}^n)$ of (E)$_p$ such that for every $y\in  \mathbb{R}^n$,
there is a backward characteristic $\xi_y$ of $v_p$ with $\xi_y(0)=y$. Of course, due to \eqref{char}, $L(\xi_y(s),\dot \xi_y(s))=0$ for all $s<0$.
Hence, $|\dot \xi_y(s)| \leq a(\xi_y(s))$, which yields $\xi_y(s) \in \mathcal{R}_{|s|}(y)$  for all $s<0$.
In other words, \eqref{char} for $\xi_y$ can be written in an equivalent way as following
\begin{equation}\label{char-n}
\begin{cases}
\xi_y(s) \in \mathcal{R}_{|s|}(y)  &\text{for } s<0,\\
p\cdot \xi (t_1)+v_p(\xi (t_1))-p\cdot \xi (t_2)-v_p(\xi (t_2))=(t_1-t_2) \overline{H}(p) &\text{for } t_2<t_1 \leq 0.
\end{cases}
\end{equation}

\begin{lem}\label{lem:vertex-rate}
Assume all the hypotheses of Theorem \ref{thm:rate2}.
Let $v$ be a vertex of $P$.
Pick $p \in \mathbb{R}^n$ such that $\overline{H}$  is differentiable at $p$ and $D\overline{H}(p)=v$.
Let  $\xi :(-\infty, 0]\to \mathbb{R}^n$ be a  backward characteristic of $v_p\in {\rm Lip\,}(\mathbb{T}^n)$, a solution of {\rm (E)}$_p$.
Then, $\xi(t) \in \mathcal{R}_{|t|}(\xi(0))$ for all $t<0$. Furthermore, there exists $C_p>0$ depending on $H, \overline{H}, p$ such that
\begin{equation}\label{v-rate}
\left |{\xi(t)-\xi(0)\over t}- v\right|\leq {C_p\over |t|}  \quad \text{for $t <0$}.
\end{equation}
\end{lem}

\begin{proof}
In fact, $\overline{H}$ is linear in a neighborhood of $p$, say $B(p,r)$, and $D^2 \overline{H}(p)=0$.

By the definition of backward characteristics and \eqref{char-n}, $\xi(t) \in \mathcal{R}_{|t|}(\xi(0))$, and
\[
p\cdot \xi(0)-p\cdot \xi(t)+v_p(\xi (0))-v_p(\xi (t))=|t| \overline H(p),
\]
 for all $t<0$.
Next, for $\tilde p\in  \mathbb{R}^n$, let $v_{\tilde p} \in {\rm Lip\,}(\mathbb{T}^n)$ be a solution to (E)$_{\tilde p}$.  It is clear that
\[
\tilde p\cdot \xi(0)-\tilde p\cdot \xi(t)+v_{\tilde p}(\xi (0))-v_{\tilde p}(\xi (t))\leq \int_{t}^{0}L( \xi(s),\dot\xi(s) )+\overline H(\tilde p)\,ds= |t| \overline{H}(\tilde p).
\]
It suffices to prove \eqref{v-rate} for $t\leq -1$. Set
\[
w={{\xi (t)-\xi(0)\over t}}-D\overline H(p)={\xi(t)-\xi(0)\over t}- v.
\]
There is nothing to prove if $w=0$. We therefore may assume that $w \neq 0$.
Accordingly, for $|\tilde p-p| < r$,
\begin{equation}\label{conv-rot-2}
\overline H(\tilde p)-\overline H(p)\geq  {(\tilde p-p)\cdot {\xi (t)-\xi(0)\over t}}-{{C(1+|p|)}\over |t|}.
\end{equation}
Since $\overline H$ is linear in $B(p,r)$, for $|\tilde p-p| <r$,
\[
\overline{H}(\tilde p)  = \overline{H}(p) + D\overline{H}(p)\cdot (\tilde p - p).
\]
Combine this with the above inequality to deduce that
\[
  (\tilde p-p)\cdot \left( {{\xi (t)-\xi(0)\over t}}-D\overline H(p)\right) \leq {{C(1+|p|)}\over |t|}.
\]
Choose $\tilde p=p+{r \over 2} {w\over  |w|} $ to complete the proof.
\end{proof}

\begin{lem}\label{lem:P-rate1}
Assume all the hypotheses of Theorem \ref{thm:rate2}.
Then, there exists $C>0$ such that
\[
P \subset \frac{\mathcal{R}_t(Y)}{t} + B_{\frac{C}{t}}.
\]
\end{lem}

\begin{proof}
Let $v_1,\ldots, v_k$ be the vertices of $P$.
By Lemma \ref{lem:vertex-rate}, there exist $\xi_1(\cdot), \ldots, \xi_k(\cdot)$ and a constant $C>0$ such that
\[
\begin{cases}
\xi_i(t) \in \mathcal{R}_t(Y) \quad &\text{ for all } 1\leq i \leq k,\, t>0,\\
|\xi_i(t) - t v_i| \leq C \quad &\text{ for all } 1\leq i \leq k,\, t>0.
\end{cases}
\]
For any point $v \in P$, $v$ can be written as a convex combination of the vertices $v_1,\ldots, v_k$, that is,
\[
v = \alpha_1 v_1 + \cdots + \alpha_k v_k,
\]
for some $\alpha_1,\ldots, \alpha_k \geq 0$ and $\sum_{i=1}^k \alpha_i=1$.
We construct $\xi(\cdot)$ as a convex combination of the paths $\xi_1(\cdot), \ldots, \xi_k(\cdot)$ (roughly, $\alpha_1 \xi_1(\cdot)+ \cdots + \alpha_k \xi_k(\cdot)$ with connectors as in \eqref{eq:connector}) so that
\[
\xi(t) \in \mathcal{R}_t(Y), \ |\xi(t) - tv| \leq C \quad \text{ for all } t>0.
\]
The proof is complete.
\end{proof}

We now need to obtain the converse inclusion.
It is extremely important noting that the following result always holds true in general setting without any restriction on $P$.
This result is similar to \cite[lower bound (1.2)]{MTY}.

\begin{lem}\label{lem:P-rate2}
There exists $C>0$ such that
\[
 \frac{\mathcal{R}_t(Y)}{t} \subset P+ B_{\frac{C}{t}}.
\]
\end{lem}

\begin{proof}
Let $z$ be a point on the boundary of $P$. For given $t>0$, consider $\gamma:[0,t] \to \mathbb{R}^n$ such that
\[
\begin{cases}
\gamma(0) \in Y, \gamma(t) = \lambda z,\\
|\dot \gamma(s)| \leq a(\gamma(s)) \quad \text{ for a.e. } s \in (0,t),
\end{cases}
\]
where $\lambda>0$ is given. We now proceed to estimate how big $\lambda$ can be.
For each $p\in P$, let $v_p \in {\rm Lip\,}(\mathbb{T}^n)$ be a solution of (E)$_p$. Then,
\[
\overline{H}(p) = a(\gamma(s))|p+Dv_p(\gamma(s))| \geq \dot \gamma(s) \cdot (p + Dv_p(\gamma(s))) \quad \text{ for a.e. } s \in (0,t).
\]
To make the above rigorous, one needs to do convolution with a standard mollifier, but we omit it here.
Integrate this on $[0,t]$ to deduce that
\[
t \overline{H}(p) \geq \int_0^t \dot \gamma(s) \cdot (p + Dv_p(\gamma(s))) \,ds= p\cdot (\lambda z  - \gamma(0)) + v_p(sz) - v_p(\gamma(0)),
\]
which means
\[
p \cdot\frac{\lambda z}{t} - \overline{H}(p) \leq \frac{C}{t}.
\]
If $\lambda \leq 1$, then $\gamma(t) \in P$.
Otherwise, for $\lambda>1$, we take the supremum of the above over $p \in P$ to yield
\begin{equation*}
c\left(\frac{\lambda}{t}-1\right) \leq \sup_{p\in P} \left( p \cdot \frac{\lambda z}{t} - \overline{H}(p)\right) \leq \frac{C}{t},
\end{equation*}
for some $c>0$. Thus, we arrive at
\[
\frac{\lambda}{t} \leq 1 + \frac{C}{t},
\]
which gives the conclusion.
\end{proof}

\begin{proof}[Proof of Theorem \ref{thm:rate2}]
Thanks to Lemmas \ref{lem:P-rate1} and \ref{lem:P-rate2}, we infer that
\[
\rho\left(\frac{\mathcal{R}_t(Y)}{t}, D \right) \leq \frac{C}{t} \quad \text{ for } t>0.
\]
We then apply Lemma \ref{lem:rate-connection} to conclude the proof.
\end{proof}


\section{Introduction}\label{intro}

Orientation estimation is a significant step of obtaining comprehensive information of 3D objects \cite{p1}.
In autonomous driving, accurate estimation of pedestrian orientation helps the ego vehicle obtain intentions (e.g. details of movements and future trajectories) of pedestrians in the related environment, and improve its self-decisions according to the intentions \cite{p16}.
Such self-decisions include basic policy makings such as path planning and steering control, and safety measures such as collision avoidance and accident prewarning \cite{p18, p19}.

Orientation estimation is also one of the most difficult tasks among all steps of pedestrian information extraction, especially in monocular solutions, with reasons as follows:

\begin{itemize}
\item
More complex appearances \cite{p17} .
Appearances of pedestrians are usually composed of multiple colors and patterns on garments and ornaments.
This makes it easier for monocular solutions to confuse them with the background environment.
Since orientation and patterns are largely irrelevant, monocular solutions face a harder problem of finding orientation-related characteristics.
\item
Smaller sizes.
Pedestrians have smaller height and far smaller width and length than other traffic objects in the environment (e.g. vehicles), making them more difficult to be detected.
The subsequent orientation estimation process is certainly more challenging, since smaller sizes reduce the differences between different orientations of pedestrians.
\item
Higher-level deformation \cite{p17}.
Unlike other traffic objects, pedestrians are deformable, and have quite different shapes and sizes under different states and gestures.
This indicates that a variety of approaches, such as 3D modeling, are not suitable for pedestrian orientation estimation.
Moreover, deformation indicates that the same orientation may correspond to a variety of shape characteristics, which is highly confusing.
\end{itemize}

From the analyses above, it is obvious that accurate pedestrian orientation estimation requires very deep and diverse information of pedestrians, as well as related high-performance information processors.
This is a huge challenge for orientation estimation solutions, especially monocular ones which lack depth information of objects and the related environment.
This results in performance restrictions of the solutions.
Therefore, orientation estimation solutions need information enhancements.

For monocular solutions in which camera captures are the only input data, information enhancement relies largely on internal information mining.
Most state-of-the-art solutions have the same feature extraction and recognition process dealing with both vehicles and pedestrians, thus neglecting the requirement for more information of pedestrian orientation estimation.
Therefore, almost all state-of-the-art monocular solutions have relatively lower performance and huger growth potential in pedestrians.
The official KITTI Object Detection and Orientation Estimation Evaluation benchmark \cite{p2} shows that general precision of pedestrian orientation estimation (represented as average orientation similarity, AOS) is far less than that of cars and cyclists.
Determination of supplementary information and its mining process are urgently needed.

Logic relationship is one of the kinds of information that is important but most likely to be neglected.
It lies between inputs and outputs of the solution model, and between outputs if the model has multiple variables to predict.
Since orientation is the only desired variable of the problem, there is no logic relationship without the introduction of another output.
Therefore, in this paper, we introduce the 2D and 3D dimensions of pedestrians and analyze the logic relationship between orientation and them.
The 2D dimensions of the pedestrian are the width and height information of his/her 2D bounding box in the image plane, while the 3D dimensions of the pedestrian are the width, height and length information of his/her 3D bounding box in the camera 3D space.
Illustration of the pedestrian's 2D and 3D dimensions is shown in Fig. \ref{pedestrian_2D_3D_illustration}.

\begin{figure}
\centering
\includegraphics[height=4cm]{pedestrian_2D_3D_illustration}
\caption{Illustration of 2D and 3D dimensions of an exemplary pedestrian.}
\label{pedestrian_2D_3D_illustration}
\end{figure}

Based on it, a novel pedestrian orientation estimation model, called FFNet, is proposed.
Apart from the camera capture, the model further uses the introduced characteristics (i.e. 2D and 3D dimensions of pedestrians) as two other inputs to make orientation estimations.
In the model, 2D dimensions of the pedestrian are from 2D object detection models, while 3D dimensions of the pedestrian are predictions of the ego model.
To further utilize the two characteristics, two feedforward links are established respectively from 2D and 3D dimensions outputs to the orientation estimator of the model.
Intermediates of the model (i.e. 2D and 3D dimensions of pedestrians) all have physical significance, enhancing the interpretability of the model.
Meanwhile, the feedforward links strengthen the logicality of the model.
Experiments show that the proposed model with 2D-3D feedforward outperforms common state-of-the-art models by 1.72\% AOS with identical training processes.
The proposed model also gets ranked \nth{9} in the official KITTI Object Detection and Orientation Estimation Evaluation benchmark.
Our contributions are summarized as follows:

\begin{itemize}
\item
We propose a novel pedestrian orientation estimation model which has 2D and 3D dimensions as additional inputs compared with other models to make orientation estimations.
The proposed orientation estimation model takes advantage of the relationship between pedestrian's orientation and its 2D and 3D dimensions related to the image plane and camera 3D space respectively, and has outstanding performance compared with most state-of-the-art models.
\item
From the proposed model, we show a way of exploring intermediate properties of a specific problem, followed by another approach (i.e. feedforward links) of exploiting them.
Based on the intermediates and feedforward links, we propose an approach to enhance the logicality and interpretability of neural network models, by utilizing intermediates and establishing feedforward links according to the logic relationship between them, which can be applied to a variety of solution models with multiple predicted variables.
\end{itemize}

The paper is organized as follows:
we discuss the related work in Section \ref{relatedwork}.
In Section \ref{model}, we introduce the proposed model and analyze the logic relationship and feedforward links in it.
We conduct experiments to compare the proposed model with common state-of-the-art ones and make further discussions in Section \ref{experiments}.
We conclude this paper in Section \ref{conclusion}.

\section{Related Work}\label{relatedwork}

There are many Faster R-CNN \cite{p9} based monocular 3D pedestrian detection and orientation estimation solutions having accurate predictions and competitive performance.
Among them, many solution models have competitive results in the KITTI Object Detection \cite{p10, p11, p12, p13, p14, p15} and Orientation Estimation \cite{p3, p4, p5, p6} Evaluation benchmark.
According to the aspects of improvements they focus on, they can be categorized as follows:

\begin{itemize}
\item
Improvements on predictions of the model.
Mono3D \cite{p3} shows an approach of generating bounding boxes in 3D space and projecting them to the image plane.
The authors further proposed several metrics to judge the accurateness of the boxes and make adjustments.
In MonoPSR \cite{p5}, the authors proposed a model that predicts object-centered point cloud information to help determine objects' 3D locations and localizations, along with a novel loss function related to it.
These solutions often have multiple outputs and complex loss functions and metrics related to them.
\item
Improvements on the network structure of the model.
SubCNN \cite{p4} introduces the feature extrapolating layer and the subcategory conv layer to make use of subcategory information in proposal generating.
In Shift R-CNN \cite{p6}, a ShiftNet structure is used for refining 3D object translation predictions.
The ShiftNet takes both 3D object translation predictions and pre-stage 3D dimensions and orientation as inputs, and combines the information to reduce noise disturbance in images.
These solutions may only output predictions of a few basic variables.
However, their related models have complex structures and multiple branches within.
\end{itemize}

All solution models above are from the KITTI Object Detection and Orientation Evaluation benchmark.
They have one shared essence:
mining for extra information for refining or assessing predictions.
However, all of the solutions above focus on information from external sources (i.e. data inputted to the model) rather than internal sources (i.e. intermediates outputted by the model which can be further utilized by itself).
Similarly, external information sources receive much more attention than internal sources.
While a variety of external-information-enhancing algorithms significantly improve performance of neural networks (e.g. pyramid structure for multi-scale detection \cite{p27} and local feature extraction \cite{p28} for typical information-insufficient cases like dimension reduction), mining of internal information is seldom researched.

Besides, they have a common drawback:
using the same tactics to vehicles and pedestrians.
A model that can handle deformable objects with large scale variations is supposed to be more advantageous in pedestrian orientation estimation.
With the task becoming harder, the solution model is required to have the ability to extract more and deeper features in images.
Apart from solution models in the KITTI benchmark, there are a variety of pedestrian detection models focusing on the unique characteristics of pedestrians, based on a variety of sensor inputs not limited to monocular cameras.
In \cite{p20}, multiple parts of pedestrians are respectively recorded and learned by different detectors, which are combined by a decision tree, to solve occlusion problems.
In \cite{p21, p22, p23}, thermal images are studied and applied in nighttime pedestrian detection.
Thermal images take advantage of temperature features of human, and have positive effects on differentiating pedestrians from the background environment.
In \cite{p29}, illumination conditions are further considered to make better combination and use of color and thermal information of images.
In \cite{p30} and \cite{p31}, features of human attributes and 3D poses, as well as their relationships with image features, are respectively studied and combined with visual features of images for related recognition tasks.
In \cite{p24}, Oriented Spatial Transformer Network (OSTN) is proposed to rectify distorted pedestrian features in a detector-friendly manner in pedestrian detection with fish-eye cameras.

None of the solutions above have innovation in orientation estimation.
Most orientation estimation models have one more branch compared with object detection models, used to output orientation predictions.
However, there are tough cases in which extra branches are not qualified for accurate orientation estimations.
Such cases require more complex network structures to deal with.
In this paper, the proposed model focuses on the logic relationship between orientation and other characteristics of pedestrians, which is neglected by all models above.

\section{Pedestrian orientation estimation model based on deep 2D-3D feedforward}\label{model}
\subsection{Logic relationship in pedestrian orientation estimation}\label{logic}

Pedestrian orientation estimation requires more information than vehicle orientation estimation.
For monocular models, utilization of external information can hardly be improved.
Therefore, mining of internal information is significant in this case.

Logic relationship is the most significant internal information.
Use of logic relationship in neural networks has another powerful advantage: avoiding predictions not according with ground truth (i.e. illogical in external terms), or contradictory (i.e. illogical in internal terms).
In deep learning solutions, the logic relationship between inputs and outputs is the base of network training, with related information stored in weights in network layers.
In this paper, we pay main attention to the logic relationship between outputs.
In the proposed model, we introduce 2D and 3D dimensions of pedestrians to the network, to construct logic links to orientation estimation from them.

\begin{figure}
\centering
\includegraphics[height=3cm]{3D_2D_orientations}
\caption{Logic relationship between the 2D-3D dimensions and orientation of exemplary pedestrians.}
\label{3D_2D_orientations}
\end{figure}

2D and 3D dimensions, and orientation angles of exemplary pedestrians from the KITTI Object Detection dataset are listed in a control-variate manner in Fig. \ref{3D_2D_orientations}.
To eliminate the influences of scaling on the final results, selected pedestrians' 2D (3D) height is scaled equal when 3D (2D) dimensions are fixed in the control variate method.
Figure \ref{3D_2D_orientations} shows the monotonic influences of pedestrians' 2D and 3D dimensions on its orientation angle respectively.
Assumed that distortion in camera captures has already been rectified, while the camera and pedestrians are at the same height, the width span of pedestrians in camera X-axis is as Equation \ref{eq1}.

\begin{equation}
\overline{w_1}=
\begin{cases}
w_1\sin\theta+l_1\cos\theta, &\text{if $0\le\theta<\frac{\pi}{2}$}\\
w_1\sin\theta-l_1\cos\theta, &\text{if $\frac{\pi}{2}\le\theta<\pi$}\\
l_1\cos\theta-w_1\sin\theta, &\text{if $-\frac{\pi}{2}\le\theta<0$}\\
-l_1\cos\theta-w_1\sin\theta, &\text{if $-\pi\le\theta<-\frac{\pi}{2}$}
\end{cases}
\label{eq1}
\end{equation}

Values of 2D dimensions ($h$, $w$), 3D height $h_1$ and width span $\overline{w_1}$ satisfy Equation \ref{eq2}.

\begin{equation}
\frac{h}{w}=\frac{h_1}{\overline{w_1}}
\label{eq2}
\end{equation}

Corresponding to Fig. \ref{3D_2D_orientations}, Equation \ref{eq1} and \ref{eq2} represent the logic relationship in mathematic terms:
with $\theta$ changing in the range $(-\pi, \pi)$, $\overline{w_1}$ changes with a relatively large amplitude, which can be reflected in the ratio of $h$ to $w$.
This can be used in pedestrian cases, especially walking pedestrians which have huge differences between $w_1$ and $l_1$.

Since object orientation is desired, 2D and 3D dimensions are at the beginning of the logic links, while orientation is at the end.
Therefore, the logic links are established from 2D and 3D dimensions to orientation in the proposed model.

\subsection{2D-3D feedforward based pedestrian orientation estimation model}\label{sectionnetwork}

The proposed FFNet model is based on the implementation of the logic links proposed above.

For the implementation of a logic link, the most direct approach is to make those variables at the beginning of it as inputs, and those variables at the end of it as outputs, to make the value of the former variable influence prediction of the latter one.
Therefore, in the network of the proposed model, there are layers connected to orientation output, with values of 2D and 3D dimensions as inputs.
These layers with 2D and 3D dimensions inputs are respectively combined and represented by feedforward links in the proposed model.

The feedforward mechanism in neural networks has many successful implementations in object detection.
In Shift R-CNN \cite{p6}, a feedforward link lies between the predicted characteristics of the object and the 3D translation predictor, to take the object's 2D and 3D characteristics as inputs for its 3D translation regression.
In Mask Scoring R-CNN \cite{p7}, the mask IoU prediction process takes the predicted mask as input as well as RoI features to achieve a more accurate mask IoU prediction result.
In the field of click feature prediction in image recognition, feedforward is used for label generation in semi-supervised learning \cite{p26} and supplementary information (i.e. hypergraph manifold) offering \cite{p32}.
However, all solution models above use feedforward for regression or error checking instead of direct prediction, and none of them pays attention to the logic relationship between the predicted characteristics of objects.
In this paper, the proposed model implements feedforward in a direct one-step prediction process.

The overall orientation estimation process of the proposed model is shown in Fig. \ref{process}, in which the feedforward links are marked red, and the logic links to orientation estimation respectively from 2D and 3D dimensions of pedestrians are marked blue.
Input of the 3D feedforward link is provided by the feature extractor in the network model.
Therefore, the network model makes 3D dimensions regression before making orientation estimation, and the 3D dimensions result is used as input to the 3D feedforward link.
In contrast, input of the 2D feedforward link is considered as external information, and is provided by an external 2D object detection model.
In this paper, the 2D object detection model \cite{p14} is adopted for providing the 2D dimensions information.

\begin{figure}
\centering
\includegraphics[height=3cm]{whole_process}
\caption{The overall orientation estimation process of the proposed model.}
\label{process}
\end{figure}

Based on the feedforward links above, the proposed model has the network structure shown in Fig. \ref{network}.
The network adopts the VGG16 \cite{p25} frame, with added 2D and 3D dimensions processors and a related concatenation process.
The 2D and 3D dimensions processors are constructed by 3 fully-connected layers, with the number of neurons 2 (2D dimensions processor) / 3 (3D dimensions processor), 512 and 2048 respectively.
Outputs of the two processors are then directly concatenated to the flattened feature maps outputted by the conv layers.

\begin{figure}
\centering
\includegraphics[height=3cm]{network_structure}
\caption{Network structure of the proposed model.}
\label{network}
\end{figure}

Direct concatenation is widely adopted in the implementation of feedforward.
In Shift R-CNN, prediction of 3D translation, 2D bounding box information, 3D dimensions and orientation are all combined and inputted to the 3D translation refinement network.
In Mask Scoring R-CNN, RoI feature maps and the predicted mask are stacked to form an ensemble and fed into the mask scoring network.
Since outputs of 2D and 3D dimensions processors have no planarity other than outputs of the two models above, the concatenation process connects elements of outputs to form a larger-scale fully-connected layer.
The fully-connected layer is then connected to 2 fully-connected layers with the number of neurons 512 and 2 respectively.
The last fully-connected layer outputs $\sin$ and $\cos$ values of the orientation prediction.

Note that in two network structures lie the way of extracting and exploiting the 2D and 3D dimensions information the network model adopts: the fully-connected layers in the feedforward links and after the concatenation process.
The extraction and exploitation of the information are not label-guided, since it is encouraged that the network model takes consideration of the related environment and possible ground truths, which cannot be labeled.
The effectiveness of the model is reflected by the accuracy of its orientation output, and is improved through the training process.

In the 3D feedforward link, the back-propagation process starts from orientation prediction, and continues until it reaches the first conv layer.
This makes the errors of orientation estimation influence the feature extraction and 3D dimensions prediction processes above.
To avoid this, gradient truncation is used at the 3D dimensions output (i.e. preventing the gradient from propagating towards the 3D dimensions prediction process), to eliminate the problems raised above.

\subsection{Definitions of loss functions}\label{loss}

The loss function is as Equation \ref{eq3}.

\begin{equation}
    loss=loss_{dimensions}+loss_{orientation}
\label{eq3}
\end{equation}

Dimensions loss adopts the L2-loss form, while orientation loss has the form as Equation \ref{eq4}.

\begin{equation}
\begin{split}
&loss_{orientation}=\sum_{i=1}^{bin}loss_i\\
loss_i=1 &-\sin\overline{orientation_i}\times\sin orientation_i\\
         &-\cos\overline{orientation_i}\times\cos orientation_i
\end{split}
\label{eq4}
\end{equation}

In Equation \ref{eq4}, $orientation_i$ and $\overline{orientation_i}$ are respectively the estimated and labeled orientation value.
The orientation loss is positive if orientation output does not equal its truth value, and has a positive correlation with the difference between them.

Meanwhile, determination of the estimated orientation angle has the form as Equation \ref{eq5}.

\begin{equation}
\begin{split}
orientation_i &=\arctan\frac{SinValue_i}{CosValue_i}\\
              &+
\begin{cases}
0, &\text{if $CosValue_i>0$}\\
\pi, &\text{if $CosValue_i<0$ \& $SinValue_i>0$}\\
-\pi, &\text{if $CosValue_i<0$ \& $SinValue_i<0$}
\end{cases}
\end{split}
\label{eq5}
\end{equation}

In Equation \ref{eq5}, $SinValue_i$ and $CosValue_i$ are respectively $\sin$ and $\cos$ output of the network.
In some ways, the orientation loss is like the MultiBin orientation loss proposed in the Deep3DBox model \cite{p8}.
The MultiBin orientation estimation algorithm performs two steps.
First, it divides the orientation space into multiple bins and predicts the bin the orientation angle stays in.
Specifically, if the orientation angle stays near the edge of two bins, MultiBin considers it `in both bins', and assigns classification confidence 0.5 to each bin.
Second, it predicts the residual rotation between the orientation angle and the center of the predicted bin.
The two-step process of the MultiBin algorithm is shown in the upper branch in Fig. \ref{multibin}.

\begin{figure}
\centering
\includegraphics[height=4cm]{multibin_improve}
\caption{The proposed orientation estimation algorithm based on the MultiBin algorithm.}
\label{multibin}
\end{figure}

Although the MultiBin algorithm simplifies the orientation estimation problem and turns it into semi-classification-semi-regression, it has two main drawbacks:

\begin{itemize}
\item
MultiBin loss neglects the differences between angles near the edges and angles with little classification confidence, since the algorithm outputs maximum classification confidence 0.5 for angles near the edges.
The confusion in classification confidence directly influences confidence of orientation estimation in the Deep3DBox model.
\item
MultiBin relies largely on the classification result at the first stage, thus introducing another type of error to the orientation estimation process.
In the case shown in Fig. \ref{multibin}, misclassification results in $90^\circ$ orientation error in average.
\end{itemize}

In order to mitigate the two problems of MultiBin, the proposed orientation loss abandons classification, but adopts the multi-bin mechanism, and takes consideration of prediction results from all bins.
The proposed orientation estimation algorithm is shown in the lower branch in Fig. \ref{multibin}.
In this way, the first problem can be eliminated.
In the meantime, multiple prediction results act as voters to exclude false outputs among them, with excluding algorithm presented below.

In ideal conditions, after offset adjustments, orientation outputs from all bins should be equal.
For orientation output from each bin, if it satisfies both of the following 2 conditions, it will be excluded from orientation loss calculating.

\begin{itemize}
\item
Differences between it and outputs from other bins are all greater than a predefined threshold.
\item
Differences between outputs from other bins are all smaller than the threshold.
\end{itemize}

\section{Experiments, results, and discussions}\label{experiments}

\subsection{Experiment setup}\label{setup}

The proposed model is tested and compared in two aspects.
First, we construct a plain model, which is the same as the proposed model but without logic relationship and 2D-3D feedforward, and compare the performance of them.
Second, we test the proposed model on the KITTI Object Detection and Orientation Estimation Evaluation benchmark and compare it with other state-of-the-art models.

In order to investigate the effect of feedforward links on feature combination, we construct a new loss function according to Equation \ref{eq1} and \ref{eq2}:

\begin{equation}
\begin{split}
\overline{w_1}_{prediction}={w_1}_{prediction}\times\left|\sin\theta\right|+{l_1}_{prediction}\times\left|\cos\theta\right|\\
\overline{w_1}_{prediction}^1=w_1\times\left|\sin\theta_{prediction}\right|+l_1\times\left|\cos\theta_{prediction}\right|
\end{split}
\label{eq6}
\end{equation}

\begin{equation}
\begin{split}
loss\_consistency_{dimensions}=h\times\overline{w_1}_{prediction}-w\times h_1\\
loss\_consistency_{orientation}=h\times\overline{w_1}_{prediction}^1-w\times h_1
\end{split}
\label{eq7}
\end{equation}

The two consistency losses are added to the total loss (i.e. Equation \ref{eq3}) with weight 0.01 each.
The consistency loss combines 2D and 3D dimensions with orientation prediction in the loss function perspective, without intervention of feedforward links.

For comparisons in the first aspect, there are four models involved: the proposed model and the plain model, respectively with and without consistency loss.
The four models are trained and validated through identical training and validation processes with the same dataset.
Since the KITTI benchmark is used for testing the proposed model, the related KITTI dataset is used for training and validation.
5-fold cross validation is conducted, with 6000 images as training data and 1481 images as validation data.
Parameters and environment of the training and validation processes are listed in TABLE \ref{param}.
For comparisons in the second aspect, the well-trained model in the first aspect is tested in the KITTI benchmark, with results and rankings compared with other models.

\begin{table}[]
\centering
\caption{Parameters and environment of the training and validation process.}
\begin{tabular}{cc}
\hline\hline
Parameters / Environment & Value / Choice\\
\hline
\tabincell{c}{Number of bins in the\\multi-bin mechanism} & 4\\
Hardware environment & NVIDIA GeForce GTX 1080Ti\\
Batch size & 32 (training) / 1 (validation)\\
Number of batches (training) & 22500\\
Learning rate (training) & \tabincell{c}{$10^{-3}$ (first $6\times 10^3$ batches)\\$10^{-4}$ (last $1.65\times 10^4$ batches)}\\
\hline\hline
\end{tabular}
\label{param}
\end{table}

In training and validation, learning speed, minimum loss value, inference accuracy, runtime resource occupancy and runtime speed are all concerned metrics in the comparisons.

\subsection{Experiment results with the plain models}\label{resultplain}

In training, given the same loss function, the proposed and the plain model have the loss convergence processes shown in Fig. \ref{totalloss}.
While the two models have similar speed of loss convergence at the first few batches, the plain model reaches the minimum loss value roughly at the \nth{5000} batch while the proposed model still has a stable loss decreasing trend.
The proposed model reaches the minimum loss value roughly at the \nth{17500} batch.
The proposed model has the minimum loss value of only 0.13, while that of the plain model is 3.4.
Therefore, logic links and 2D-3D feedforward help to enhance the training performance of the network model.

\begin{figure}
\centering
\includegraphics[height=4cm]{total_loss}
\caption{Loss convergence process of the proposed model and the plain model in training (without consistency loss). 3 of the 5-fold cross validation processes are shown.}
\label{totalloss}
\end{figure}

In validation, given the same loss function, orientation error distributions of the proposed and the plain model on all validation data samples are shown in Fig. \ref{errordis}(a), while 3D dimensions error distributions are shown in Fig. \ref{errordis}(b).
The proposed model has orientation estimation error massed in 0-$30^\circ$, and has far better performance of recognizing the main axes of pedestrians, proven by its much fewer samples with errors around $180^\circ$.
Therefore, logic links and 2D-3D feedforward have positive effects on the network model in validation.
This is in accordance with the better training results of the proposed model shown above.

\begin{figure}
\centering
\subfigure[]{\includegraphics[height=4.3cm]{angle_distribution}}
\subfigure[]{\includegraphics[height=4.3cm]{dim_distribution}}
\caption{Orientation and 3D dimensions error distributions of the proposed and the plain model on all validation data samples.}
\label{errordis}
\end{figure}

With consideration of consistency loss, the proposed and the plain model have the loss convergence processes shown in Fig. \ref{losscon}.
From Fig. \ref{losscon}, it is obvious that consistency loss reduces accuracy of the proposed and the plain model.
For the plain model, this is partly because the network focuses more on converging consistency losses than the parts in Equation \ref{eq3}.
Since consistency losses do not reflect definite accuracy of dimensions regression and orientation estimation, the model is worse at accuracy if with consistency loss.
For the proposed model, another important reason is that consistency losses are not compatible with the approaches the network adopts for extraction and exploitation of 2D and 3D information, which are analyzed in Section \ref{sectionnetwork}.
Therefore, it is wise to let the network learn its way of dealing with 2D and 3D information.
Note that consistency losses are calculated in training, but not calculated in plotting Fig. \ref{losscon}, to make better horizontal comparisons between the models with and without consistency loss.

\begin{figure}
\centering
\includegraphics[height=4cm]{total_loss_consistency}
\caption{Loss convergence process of the proposed model and the plain model in training (with consistency loss).}
\label{losscon}
\end{figure}

\subsection{Comparisons of time and resource consumption between the proposed and the plain model}\label{timememory}

In training, the proposed model takes 7.6s and roughly 4500MB runtime memory to learn from 100 batches of data, while those of the plain model are 7.1s and 4400MB.
This is because the added 2D-3D dimensions processors, related feedforward links and the concatenation layer require extra processing time and resources.
However, slight differences in processing time and runtime memory between the two models can be neglected in applications.

In validation, considering the time of reading and preprocessing data, the proposed model takes 22.9s and roughly 3900MB runtime memory to complete the inference process on all validation data (3814 samples), while those of the plain model are 21.8s and 3900MB.
The cause of the differences is the same as that in training, and can also be neglected in applications.

\subsection{Test results in the KITTI benchmark with state-of-the-art models}\label{kitti}

The proposed model is further tested in the KITTI Object Detection and Orientation Estimation Evaluation benchmark. Test results as well as that of other state-of-the-art models \cite{p3, p4, p5, p6} are shown in TABLE \ref{kittiresult}.

\begin{table}[]
\centering
\caption{Benchmark results of the proposed model, the plain model, and other state-of-the-art models.}
\begin{tabular}{cccccc}
\hline\hline
Model & Moderate & Easy & Hard & Runtime & Ranking\\
\hline
\textbf{FFNet (proposed)} & \textbf{59.17\%} & \textbf{69.17\%} & \textbf{54.95\%} & \textbf{0.22s} & \textbf{\nth{2}}\\
Mono3D \cite{p3} & 58.12\% & 68.58\% & 54.94\% & 4.2s & \nth{3}\\
SubCNN \cite{p4} & 66.28\% & 78.33\% & 61.37\% & 2s & \nth{1}\\
MonoPSR \cite{p5} & 56.30\% & 70.56\% & 49.84\% & 0.2s & \nth{4}\\
Shift R-CNN \cite{p6} & 48.81\% & 65.39\% & 41.05\% & 0.25s & \nth{5}\\
Plain & 38.92\% & 46.36\% & 35.63\% & 0.21s & \nth{6}\\
\hline\hline
\end{tabular}
\label{kittiresult}
\end{table}

The proposed model has competitive results in the KITTI benchmark, ranked \nth{2} among common state-of-the-art models and \nth{9} on the leaderboard.
This proves the effectiveness of the logic links and 2D-3D feedforward.

Corresponding to the results shown in TABLE \ref{kittiresult}, OS-R (Orientation Similarity \cite{p2} - Recall) curves of the proposed model, the plain model, and other state-of-the-art models are shown in Fig. \ref{osr}.
From Fig. \ref{osr}, it is obvious that the proposed model does not have competitive results in the low-recall area, but has the best result in the high-recall area among all state-of-the-art models.
This is caused by two main factors.
First, unlike other models, the proposed model has no other structural improvements except feedforward links, making it not competitive in high-precision occasions.
Second, feedforward links play a quite significant role in relatively difficult orientation estimation tasks, making it the model with highest precision in high-recall occasions.
Higher-recall occasions involve much more small/occluded/truncated pedestrians.
In these feature-insufficient occasions, feedforward links provide useful internal information, making the proposed model more qualified for orientation estimation of those pedestrians.

\begin{figure}
\centering
\includegraphics[height=4cm]{OS-R_curves}
\caption{OS-R curves of the proposed model, the plain model, and other state-of-the-art models.}
\label{osr}
\end{figure}

\subsection{Discussions}\label{discussions}

In the training process, the convergence processes of $loss_{dimensions}$ and $loss_{orientation}$ are shown in Fig. \ref{respective}.
The process can be divided into three stages.
At the first stage, $loss_{dimensions}$ decreases to its minimum while $loss_{orientation}$ has no sign of convergence.
This is because of the logic link from 3D dimensions output to the orientation estimator.
Orientation estimation is not accurate without precise 3D dimensions predictions.
At the second stage, $loss_{dimensions}$ has reached its minimum while $loss_{orientation}$ is still floating on a certain level.
This is because the network requires time and data to learn the logic relationship between 2D dimensions, 3D dimensions and orientation of pedestrians.
After successfully mastering the logic relationship, in the final stage, the network has its orientation loss gradually converging to its minimum.
After the 3 stages, the network reaches relatively high accuracy, and masters the logic relationship within the three characteristics.

\begin{figure}
\centering
\includegraphics[height=4cm]{respective_loss}
\caption{The 3-stage convergence process of 3D dimensions and orientation loss.}
\label{respective}
\end{figure}

Furthermore, we go deeper and investigate the influence on orientation estimation from 2D and 3D feedforward branch respectively.
In this experiment, the control variate method is used: the input of one branch is fixed while that of the other branch changes in a small range.
Note that in the 3D branch, the input of the feedforward link, rather than truth values of 3D dimensions, is changed.

A pedestrian with relatively fine orientation estimation accuracy is used for illustration.
Orientation and 3D dimensions errors of the prediction are roughly $0.42^\circ$ and $0.043m^2$ (mean squared error) respectively.
The characteristics of the pedestrian are listed in TABLE \ref{pedchar}.

\begin{table}[]
\centering
\caption{Characteristics of a well-predicted pedestrian for illustration.}
\begin{tabular}{cc}
\hline\hline
Characteristic & Value\\
\hline
2D height $h$ / pixel & 86\\
2D width $w$ / pixel & 33\\
3D height $h_1$ / m & 1.68\\
3D width $w_1$ / m & 0.50\\
3D length $l_1$ / m & 0.42\\
Orientation $\theta$ / $^\circ$ & -127.26\\
\hline\hline
\end{tabular}
\label{pedchar}
\end{table}

With the change of the pedestrian's 2D width $w$ from 0.1 to 2 times the original value, orientation estimations of the model are shown in Fig. \ref{widthchange}.
Figure \ref{widthchange} shows that the model successfully learns the monotonicity of Equation \ref{eq1} with the argument $w$.
However, the amplitude of the orientation change in Fig. \ref{widthchange} is much smaller than that in Equation \ref{eq1}.
This is caused by two main factors.
First, the orientation output of the model relies largely on feature extraction and recognition of the image input (i.e. the middle link in Fig. \ref{process}). Second, there are differences between Equation \ref{eq1} and the approach of extracting and exploiting dimensions information adopted by the model, which is analyzed in Section \ref{resultplain}.

\begin{figure}
\centering
\includegraphics[height=4cm]{width_change}
\caption{Orientation estimations of the model with the change of the pedestrian's 2D width.}
\label{widthchange}
\end{figure}

With the change of the pedestrian's 3D height $h_1$ from 0.1 to 2 times the original value, orientation estimations of the model are shown in Fig. \ref{heightchange}.
Figure \ref{heightchange} shows that the model successfully learns the monotonicity of Equation \ref{eq1} with the argument $h_1$.
However, the amplitude of the orientation change in Fig. \ref{heightchange} is much smaller than that in Equation \ref{eq1}, with the same reasons as above.

\begin{figure}
\centering
\includegraphics[height=4cm]{height_change}
\caption{Orientation estimations of the model with the change of the pedestrian's 3D height.}
\label{heightchange}
\end{figure}

According to Equation \ref{eq1}, a fixed 2D-3D dimensions pair corresponds up to 8 possible orientation angles.
Therefore, the proposed model yet fails to eliminate main-axes problems.
Solutions to this problem will be important future work in this field.

The proposed model shows a universal feedforward mechanism, which determines the logic sequence of the predicted variables and establishes logic links between them.
The mechanism can be adopted in various kinds of solutions dealing with multiple-prediction problems with logic relationship within.

\section{Conclusion}\label{conclusion}

In this paper, we propose a novel pedestrian orientation estimation model called FFNet.
The model makes use of the logic relationship between 2D-3D dimensions and orientation of pedestrians, and implements it by establishing logic links between them and orientation predictions.
The logic links are realized by 2D-3D feedforward branches.
Experiments show that the proposed model has a great performance increase compared with the same model without logic links and 2D-3D feedforward.
Test results on the KITTI Object Detection and Orientation Estimation Evaluation benchmark further verify our claim.
The proposed feedforward branches act as a mechanism that can be applied to a variety of solution models with multiple predictions.

\section*{Acknowledgements}

This work is supported by the National Natural Science Foundation of China (U1764264/61873165), International Chair on automated driving of ground vehicle.

\bibliographystyle{unsrt}


\section{Projective representations}
A fundamental quantum property is that the physical measurable transition
probabilities are given by the square of the modulus of the states.
This results in a quantum phase that is responsible for quantum
phenomena such as interference. Physical measurable transition probability
from state $\alpha $ to $\beta $ is given by,
\begin{equation}
P( \alpha \rightarrow \beta ) =|\left( \Psi _{\beta },\Psi _{\alpha
}\right) |^{2}.%
\label{transprob}
\end{equation}

The physical states $\Psi $ are rays that are equivalence classes
of states in a Hilbert space $\text{\boldmath $\mathrm{H}$}$ defined
up to a phase, $\Psi =[|\psi \rangle ]=\{e^{i \theta }\overset{
}{|\psi \rangle }\}$, $|\psi \rangle \in \text{\boldmath $\mathrm{H}$}$,
$\theta \in \mathbb{R}$. Suppose that you have a representation
$\upsilon$ of a Lie group $\mathcal{G}$ on $\text{\boldmath $\mathrm{H}$}$
such that $\upsilon ( g) :\text{\boldmath $\mathrm{H}$}\rightarrow
\text{\boldmath $\mathrm{H}$}:\tilde{|\psi \rangle }\mapsto \upsilon
( g) \overset{ }{|\psi \rangle }$, $g\in \mathcal{G}$. This induces
transformations on the rays $\tilde{\Psi }=\upsilon ( g)  \Psi $. Then
$\mathcal{G}$ is a symmetry group and $\upsilon $ is called a projective
representation if it preserves the transition probabilities,
\begin{equation}
P( \alpha \rightarrow \beta ) =|\left( {\tilde{\Psi }}_{\beta },{\tilde{\Psi
}}_{\alpha }\right) |^{2}=|\left( \upsilon ( g)  \Psi _{\beta },\upsilon
( g)  \Psi _{\alpha }\right) |^{2}=|\left(  \Psi _{\beta },\Psi
_{\alpha }\right) |^{2}.%
\label{transprobprojrep}
\end{equation}

These projective representations that describe quantum symmetries
also have a phase degree of freedom that are responsible for many
of their uniquely quantum properties. Several basic theorems enable
us to compute projective representations for symmetry groups of
physical interest. First, Wigner \cite{wigner}, showed
that any projective representation acting on a separable Hilbert
space is equivalent to a representation that is either linear and
unitary or anti-linear and anti-unitary (see Appendix B of Chapter
2 in \cite{Weinberg1}). As a Lie group that is topologically
path connected to the identity does not admit anti-linear, anti-unitary
representations, projective representations of connected Lie groups
are always equivalent to a representation that are linear and unitary. The
anti-linear, anti-unitary representations only are required for
groups such as the extended Lorentz group $\mathcal{I}\mathcal{O}(
1,n) $ that has disconnected components.

The inhomogeneous symplectic group that is of interest in this paper
is topologically path connected and so we restrict to the case of
connected Lie symmetry groups that considerably simplifies the analysis. We
then have the fundamental theorem that is due to the work of Bargmann
\cite{bargmann} and Mackey \cite{mackey2},\cite{mackey}:
\begin{theorem} The projective representations
of a topologically path connected Lie group {\upshape $ \mathcal{G}$}
acting on a separable Hilbert space $\text{\boldmath $\mathrm{H}$}$
are equivalent to the ordinary unitary representations of its unique
maximal central extension $\check{\mathcal{G}}$.\label{Theorem:projrep}
\end{theorem}

The essentials of the proof are as follows. The central extension
$\check{\mathcal{G}}$ of a group $\mathcal{G}$ is defined by the
short exact sequence, $e\rightarrow \mathcal{Z}\overset{i}{\rightarrow
}\check{\mathcal{G}}\overset{\rho }{\rightarrow }\mathcal{G}\rightarrow
e$. $e$ is the trivial group and as the sequence is exact, $i,\rho
$ are homomorphisms such that $\operatorname{image}( i) \simeq \ker
( \rho ) $. As the extension is central, $\mathcal{Z}$ is an abelian
group that is injected into the center of $\check{\mathcal{G}}$, $i(
\mathcal{Z}) \subset \check{\mathcal{G}}$.

The unitary representations of an abelian group are its characters
that are given by phases. Therefore, for the unitary representations
of the group $\mathcal{G}$, its center is realized as phases. These
phases are normed to unity by the square of the modulus in the computation
of the transition probabilities (\ref{transprob}). From this,
it is straightforward to show that if ordinary unitary representations
of a symmetry group $\mathcal{G}$ satisfy (\ref{transprobprojrep}),
then the unitary representations of any central extension of $\check{\mathcal{G}}$
will also satisfy (\ref{transprobprojrep}). Conversely, if the
ordinary unitary representations of a centrally extended group $\check{\mathcal{G}}$
satisfies (\ref{transprobprojrep}), then the ordinary unitary
representations of any group $\mathcal{G}$ that it is a central
extension of will also satisfy (\ref{transprobprojrep}). Note
that the complete set of unitary irreducible representations of
a central extension $\check{\mathcal{G}}$ includes the representations
of $\mathcal{G}$ that is homomorphic to it as degenerate representations.

The central extension with respect to $\mathcal{Z}$ is maximal if
any other admissible central extension with respect to $\tilde{\mathcal{Z}}$
is such that $\tilde{\mathcal{Z}}\subset \mathcal{Z}$. This unique
maximal central extension exists for topologically connected Lie
groups \cite{Azcarraga}:
\begin{theorem} The unique maximal
central extension $\check{\mathcal{G}}$ of a topologically path
connected Lie group $\mathrm{\mathcal{G}}$ exists and is given by
the short exact sequence\label{Theorem:projrepisunitaryCE}
\begin{equation}
 e\rightarrow \pi _{1}( \mathcal{G}) \otimes H^{2}( \mathcal{G},\mathbb{R})
\overset{i}{\rightarrow }\check{\mathcal{G}}\overset{\mathrm{\rho
}}{\rightarrow }\mathrm{\mathcal{G}}\rightarrow e
\end{equation}where $\pi _{1}( \mathcal{G}) $ is the fundamental
homotopy group of {\upshape $ \mathcal{G}$} and $H^{2}( \mathcal{G},\mathbb{R})
$ is the second cohomology group of $\mathrm{\mathcal{G}}$. Consequently,
$\check{\mathcal{G}}$ is connected and simply connected.
\end{theorem}

Note that $\mathbb{A}\simeq \pi _{1}( \mathcal{G}) $ is a discrete
(i.e finite or countably infinite) abelian group and $\mathcal{A}\simeq
H^{2}( \mathcal{G},\mathbb{R}) $ is a continuous group that is isomorphic
to $\mathcal{A}( m) $ for some $m$ where $\mathcal{A}( m) $ is
the continuous abelian group given by the reals under addition, $\mathcal{A}(
m) \equiv (\mathbb{R}^{m},+)$. The central extension with respect
to just the homotopy group $\pi _{1}( \mathcal{G}) $ is referred
to as the topological central extension. For a connected Lie group,
this defines the unique universal covering lie group $\overline{\mathcal{G}}$,
$e\rightarrow \pi _{1}( \mathcal{G}) \rightarrow \overline{\mathrm{\mathcal{G}}}\rightarrow
\mathrm{\mathcal{G}}\rightarrow e$.

The central extension with respect to just the second cohomology
group is referred to as the algebraic central extension. It is a
local property that is completely defined by the Lie algebra. Suppose
that the Lie algebra of $\mathcal{G}$ has a basis $\{X_{a}\}$, $a,b=1,..
\dim  (\mathcal{G})$, that satisfy the commutation relations
$[X_{a},X_{b}]=c_{a,b}^{c}X_{c}$. Then consider all possible
central generators $\{I_{\alpha }\}$,
\begin{equation}
\left[ X_{a},X_{b}\right] =c_{a,b}^{c}X_{c}+d_{a,b}^{\alpha }I_{\alpha
}, \ \ \left[ X_{a},I_{\alpha }\right] =0,\  \ \left[ I_{\alpha },I_{\beta
}\right] =0.
\end{equation}

These must satisfy the Jacobi identities which constrains admissible
central generators and trivial extensions that are just translations
of the generators $X_{a}\rightarrow X_{a}+I_{a}$ are discarded.
The remaining $m$ nontrivial generators $\{I_{\alpha }\}$, $\alpha
=1,..,m$ are the generators of the abelian group that is the second
cohomology group, $\mathcal{A}( m) \simeq H^{2}( \mathcal{G},\mathbb{R})
$. The unique universal covering group with this Lie algebra
is the central extension $\check{\mathcal{G}}$.

\subsection{Central extension examples}

This method may be straightforwardly applied to groups of interest.

First, for the abelian group $\mathcal{A}( n) $, $\pi _{1}( \mathcal{A}(
n) ) =e$ and $H^{2}( \mathcal{A}( n) ,\mathbb{R}) \simeq \mathcal{A}(
\frac{n(n-1)}{2}) $. This follows directly by noting that the
generators $A_{i}$ of the Lie algebra of $\mathcal{A}( n) $, $[A_{i},A_{j}]=0$,
$i,j =1,...,n$, admit $\frac{n(n-1)}{2}$ nontrivial central generators
$I_{i,j}$, $i<j$ that satisfy $[A_{i},A_{j}]=I_{i,j}$.

Next, we consider the connected component of the inhomogeneous Lorentz
group. We denote $\mathcal{L}( 1,n) $ to be the orthochronus
Lorentz group that is the connected subgroup of $\mathcal{O}( 1,n)
$. The inhomogeneous Lorentz group $\mathcal{I}\mathcal{L}( 1,n)
\simeq \mathcal{L}( 1,n) \ltimes\mathcal{A}( n+1) $ has a trivial
second cohomology group, $H^{2}( \mathcal{I}\mathcal{L}( 1,n)
,\mathbb{R}) =e$. This calculation is given in \cite{Weinberg1},\cite{Azcarraga}.
The Jacobi identities involving the Lorentz generators constrain
the admissible nontrivial central generators of the abelian normal
subgroup to zero. The fundamental homotopy group of $\mathcal{L}(
1,n) $ is $\mathbb{Z}_{2}$ and so $\pi _{1}( \mathcal{I}\mathcal{L}(
1,n) ) =\mathbb{Z}_{2}$. The unique maximal central extension is
the cover,
\begin{equation}
 e\rightarrow \mathbb{Z}_{2}\rightarrow \mathcal{S}pin( 1,n) \ltimes
\mathcal{A}( n+1) \rightarrow \mathcal{I}\mathcal{L}( 1,n) \rightarrow
e,
\end{equation}

\noindent where $\mathcal{S}pin( 1,n) \simeq \overline{\mathcal{L}}(
1,n) $ and, in particular, $\mathcal{S}pin( 1,3) \simeq \mathcal{S}\mathcal{L}(
2,\mathbb{C}) $.

Finally, the inhomogeneous symplectic group $\mathcal{I}\mathcal{S}p(
2n) \simeq \mathcal{S}p( 2n) \ltimes\mathcal{A}( 2n) $ is a
connected Lie group. It has a nontrivial second cohomology group
that is a one dimensional abelian group, $H^{2}( \mathcal{I}\mathcal{S}p(
2n) ,\mathbb{R}) \simeq \mathcal{A}( 1) $; the central extension
of the Lie algebra admits the single nontrivial central generator
$I$ satisfying the commutation relations $[A_{i},A_{j}]=\delta _{i,j}I$
where $A_{i}$ are the generators of the abelian $\mathcal{A}( 2n)
$ subgroup. This calculation is given in \cite{Low}. The
unique simply connected universal covering group with this Lie algebra
is the Weyl-Heisenberg group $\mathcal{H}( n) $. \ The fundamental
homotopy group of $\mathcal{S}p( 2n) $ is $\mathbb{Z}$ and so $\pi
_{1}( \mathcal{I}\mathcal{S}p( 2n) ) =\mathbb{Z}$. The unique maximal
central extension is,
\begin{equation}
 e\rightarrow \mathbb{Z}\otimes \mathcal{A}( 1) \rightarrow \overline{\mathcal{H}\mathcal{S}p}(
2n) \rightarrow \mathcal{I}\mathcal{S}p( 2n) \rightarrow e,%
\label{CEofisp}
\end{equation}

\noindent where $\mathcal{H}\mathcal{S}p( 2n) \simeq \mathcal{S}p(
2n) \ltimes\mathcal{H}( n) $ and where $\mathcal{H}( n)
\simeq \mathcal{A}( n) \ltimes\mathcal{A}( n+1) $. The full
central extension that is the universal cover is of the form
\begin{equation}
\overline{\mathcal{H}\mathcal{S}p}( 2n) \simeq \overline{\mathcal{S}p}(
2n) \ltimes\mathcal{H}( n) .
\end{equation}

\subsection{Unitary representations of central extensions}

Levi's decomposition theorem \cite{barut} states that any
Lie algebra may be uniquely decomposed into a semidirect sum of
a semisimple algebra and a radical that is the maximum solvable
ideal and this is unique up to equivalence. A direct corollary is
that the unique simply connected Lie group with this Lie algebra
is always equivalent to a semidirect product of a simply connected
semi-simple Lie subgroup $\mathcal{K}$ and a simply connected solvable
normal Lie subgroup $\mathcal{N}$. As the central extension of
a connected Lie group is simply connected, it always is of the form
of this semidirect product,
\begin{equation}
\check{\mathcal{G}}\mathrm{\simeq }\mathcal{K}\ltimes\mathcal{N}
.%
\label{CEsemidirect}
\end{equation}

Whitehead's lemma \cite{barut} establishes that the second
cohomology group of any semisimple group is trivial, $H^{2}( \mathcal{K},\mathbb{R})
\simeq e$, and so $H^{2}( \mathcal{G},\mathbb{R}) \subset
\mathcal{N}$.

\subsubsection{Unitary representations of semidirect product groups:}

The semidirect product structure of the central extension enables
us to make some general remarks about their unitary irreducible
representations before specializing to the inhomogeneous symplectic
group of primary interest in this paper. These unitary representations
have been studied by Mackey \cite{mackey2},\cite{mackey}
for a general class of locally compact topological groups. Our restriction
here to topologically connected smooth Lie groups radically simplifies
the treatment.

We begin by reviewing the very well known special case of direct
products of the form $\mathcal{G}\simeq \mathcal{K}\otimes \mathcal{N}$
for which the elements satisfy $g=h k=k h$ where $g\in \mathcal{G},k\in
\mathcal{K}, h\in \mathcal{N}$. In this case, both subgroups
are normal and the inner automorphisms are trivial; $\tilde{h}=
k h k^{-1}=h$ and $\tilde{k}= h k h^{-1}=k$. \ The complete set
of unitary irreducible representations $\upsilon $ of $\mathcal{G}$
are
\begin{equation}
\upsilon ( g) :{\text{\boldmath $\mathrm{H}$}}^{\upsilon }\rightarrow
{\text{\boldmath $\mathrm{H}$}}^{\upsilon },\ \   \upsilon =\sigma
\otimes \xi ,\ \    {\text{\boldmath $\mathrm{H}$}}^{\upsilon
}={\text{\boldmath $\mathrm{H}$}}^{\sigma }\otimes {\text{\boldmath
$\mathrm{H}$}}^{\xi } , \text{}
\end{equation}

\noindent where $\text{$\xi$}$ are unitary irreducible representations
of $\mathcal{N}$ on ${\text{\boldmath $\mathrm{H}$}}^{\xi }$, $\xi
( h) :{\text{\boldmath $\mathrm{H}$}}^{\xi }\rightarrow {\text{\boldmath
$\mathrm{H}$}}^{\xi }$ and $\text{$\sigma$}$ are unitary irreducible
representations of $\mathcal{K}$ on ${\text{\boldmath $\mathrm{H}$}}^{\sigma
}$, $\sigma ( k) :{\text{\boldmath $\mathrm{H}$}}^{\sigma }\rightarrow
{\text{\boldmath $\mathrm{H}$}}^{\sigma }$. (We label the Hilbert
space with the representation as the representation and group determine
the Hilbert space.) Representations of automorphisms are trivial
$\xi ( \tilde{h}) =\upsilon (k)\xi ( h) \upsilon ( k^{-1}) \equiv
\xi ( h) $ and likewise for $\sigma $.

For a semidirect product $\mathcal{G}\simeq \mathcal{K}\ltimes\mathcal{N}$
that is not a direct product, $\mathcal{N}$ is normal but $\mathcal{K}$
is not normal and the inner automorphisms $\tilde{h}=k^{-1}h
k$ are generally not trivial. Elements can always be decomposed
uniquely as $g= h k=k \tilde{h}$, $\tilde{h}=k^{-1}h k$.

The general form of the unitary irreducible representations are:

\begin{equation}
\upsilon ( g) :{\text{\boldmath $\mathrm{H}$}}^{\upsilon }\rightarrow
{\text{\boldmath $\mathrm{H}$}}^{\upsilon },\ \   \upsilon =\sigma
\otimes \rho ,\ \    {\text{\boldmath $\mathrm{H}$}}^{\upsilon
}={\text{\boldmath $\mathrm{H}$}}^{\sigma }\otimes {\text{\boldmath
$\mathrm{H}$}}^{\xi },
\end{equation}

\noindent where $\rho $ is a nontrivial representation of some $\mathcal{G}^{0}=\mathcal{K}^{0}\otimes
_{s}\mathcal{N}\subset \mathcal{G}$, with $\mathcal{K}^{0}\subset
\mathcal{K}$ such that
\begin{equation}
\rho ( g) :{\text{\boldmath $\mathrm{H}$}}^{\xi }\rightarrow {\text{\boldmath
$\mathrm{H}$}}^{\xi },\ \   g\in \mathcal{G}^{0},  \ \rho
|_{\mathcal{N}}=\xi  .
\end{equation}

\noindent Representations of the automorphisms are now generally
nontrivial,
\begin{equation}
\xi ( \tilde{h}) =\upsilon ( k) \xi ( h) {\upsilon ( k) }^{-1}=\rho
( k) \xi ( h) {\rho ( k) }^{-1}.%
\label{repaut}
\end{equation}

As the group is simply connected, its group elements are globally
given by the exponentials of the generators of the Lie algebra. Suppose $\{X_{a}\}$ are
a basis of the Lie algebra of $\mathcal{N}$ with and $\{Y_{\alpha
}\}$ are a basis of the Lie algebra of $\mathcal{K}$. Then,
$\rho ^{\prime }( X_{a}) =\xi ^{\prime }( X_{a}) $ span the Hilbert
space ${\text{\boldmath $\mathrm{H}$}}^{\xi }$. Note that a subset
of these generators $X_{i}=I_{i}$ may be central generators. Therefore,
by Shurr's lemma, their representation is a multiple of the identity,
$\xi ^{\prime }( X_{i}) =\xi ^{\prime }( I_{i}) =\lambda _{i}\text{\boldmath
$1$}$, $\lambda _{i}\in \mathbb{R}$. As the $\rho $ also act
on the Hilbert space ${\text{\boldmath $\mathrm{H}$}}^{\xi }$, its
representations $\rho ^{\prime }$ of the Lie algebra of $\mathcal{K}$
must be defined in terms of the enveloping algebra of the Lie algebra
of $\mathcal{N}$. That is, $\rho ^{\prime }( Y_{\alpha }) =p_{\alpha
}( \xi ^{\prime }( X_{a}) ) =\rho ^{\prime }( p_{\alpha }( X_{a})
) $ for some polynomial $p_{\alpha }$. For the faithful representation,
this means that $Y_{\alpha }=p_{\alpha }( X_{a}) $. This must
satisfy the Lie algebra of $\mathcal{K}$ up to factors of central
generators.

\subsubsection{Abelian normal subgroup:}

The case where the normal subgroup $\mathcal{N}\simeq \mathcal{A}(
n) $ is an abelian group has been extensively studied. The enveloping
algebra of the Lie algebra of an abelian group is abelian and therefore
there the Lie algebra of a nonabelian group $\mathcal{K}$ cannot
be nontrivially realized in its enveloping algebra. Consequently,
the extension $\rho $ is trivial, $\rho ( k) \equiv 1$ for $k\in
\mathcal{K}$ and $\mathcal{K}^{0}=e$. Then, the automorphism
condition (\ref{repaut}) reduces to
\begin{equation}
\xi ( \tilde{h}) =\xi ( k h k^{-1}) =\rho ( k) \xi ( h) {\rho (
k) }^{-1}=\xi ( h)  \forall  h\in \mathcal{N}.
\end{equation}

\noindent This is the fixed point equation in the unitary dual $\text{\boldmath
$U$}( \mathcal{A}( n) ) \simeq \mathbb{R}^{*n}$ that defines the
little groups $\mathcal{K}\mbox{}^{\circ}$,
\begin{equation}
\xi ( k h k^{-1}) =\xi ( h) \ \   \forall  h\in \mathcal{N},\  k\in
\mathcal{K}\mbox{}^{\circ}.
\end{equation}

The unitary irreducible representations $\upsilon \mbox{}^{\circ}=\sigma
\mbox{}^{\circ}\otimes \xi $ may then be defined on the stabilizers
$\mathcal{G}\mbox{}^{\circ}\simeq \mathcal{K}\mbox{}^{\circ}\otimes
_{s}\mathcal{N}$. The Mackey Theorems for semidirect products with
abelian normal subgroups then state that the representations induced
on the full group by these representations of the stabilizer subgroups
defines the full set of unitary irreducible representations of $\mathcal{G}\simeq
\mathcal{K}\ltimes\mathcal{A}( n) $.

\subsubsection{Perfect-Mackey group:}

Suppose $\mathcal{N}$ is nonabelian and the representation $\rho
$ exists globally on all of $\mathcal{G}$,
\begin{equation}
\xi ( \tilde{h}) =\xi ( k h k^{1}) =\rho ( k) \xi ( h) {\rho ( k)
}^{-1}\ \   \forall  h\in \mathcal{N},\  k\in \mathcal{K}.%
\label{perfectmackeyaut}
\end{equation}

Then, the representations $\upsilon =\sigma \otimes \rho $ acting
on ${\text{\boldmath $\mathrm{H}$}}^{\upsilon }={\text{\boldmath
$\mathrm{H}$}}^{\sigma }\otimes {\text{\boldmath $\mathrm{H}$}}^{\xi
}$ are a complete set of faithful unitary irreducible representations
of $\mathcal{G}$. We call groups with this property {\itshape
perfect-Mackey} groups due to the simple form of the faithful representations.

\section{Projective representations of the inhomogeneous symplectic
group}

The inhomogeneous symplectic group is a smooth Lie group that is
also a matrix group and therefore we can apply Theorem \ref{Theorem:projrep}
to compute its projective representations in terms of the ordinary
unitary representations of its central extension. The central
extension of $\mathcal{I}\mathcal{S}p( 2n) $ is the group $\overline{\mathcal{H}\mathcal{S}p}(
2n) $ given in (\ref{CEofisp}). Note that this has the required
semidirect product structure $\mathcal{K}\ltimes\mathcal{N}$ given
in (\ref{CEsemidirect}) where $\mathcal{K}=\overline{\mathcal{S}p}(
2n) $ is the simply connected semi-simple group and $\mathcal{N}=\mathcal{H}(
n) $ is the simply connected solvable normal subgroup.

\subsection{$\overline{\mathcal{H}\mathcal{S}p}( 2n) $ is a perfect-Mackey
group}

We establish that $\overline{\mathcal{H}\mathcal{S}p}( 2n) $ is
a perfect-Mackey group as described above by explicitly constructing
the $\rho ^{\prime }$ representation for all of $\overline{\mathcal{H}\mathcal{S}p}(
2n) $ \cite{Low}, \cite{Low13}. \ Let $Z_{\alpha
}$ be the generators of algebra of $\mathcal{H}( n) $ and $W_{\alpha
,\beta }$ the generators of the algebra of $\overline{\mathcal{S}p}(
2n) $, $\alpha ,\beta ,..=1,...,2n$. The Lie algebra of $\mathcal{H}(n)$
has nonzero commutators,
\begin{equation}
\left[ Z_{\alpha },Z_{\beta }\right] =\zeta _{\alpha ,\beta }I,\ \ \
\mbox{{\textlbrackdbl}}\zeta _{\alpha ,\beta }\mbox{{\textrbrackdbl}}=\left(
\begin{array}{ll}
 0 & 1 \\
 -1 & 0
\end{array}\right) .
\end{equation}

The Hermitian representation of the generators are denoted ${\hat{Z}}_{\alpha
}=\rho ^{\prime }( Z_{\alpha }) =\xi ^{\prime }( Z_{\alpha }) $
and $\hat{I}=$$\rho ^{\prime }( I) =\xi ^{\prime }( I) $ that act
on the Hilbert space ${\text{\boldmath $\mathrm{H}$}}^{\xi }=L^{2}(
\mathbb{R}^{n},\mathbb{C}) $. As $I$ is a central generator,
by Shurr's lemma its representation is $\hat{I}=\lambda  \text{\boldmath
$1$}$ , $\lambda \in \mathbb{R}\backslash \{0\}$ where {\bfseries
1} is the identity operator on ${\text{\boldmath $\mathrm{H}$}}^{\xi
}$. Then the representation is the familiar Heisenberg commutation
relations
\begin{equation}
\left[ {\hat{Z}}_{\alpha },{\hat{Z}}_{\beta }\right] =i \lambda
\zeta _{\alpha ,\beta }. %
\label{Heisenbergcomm}
\end{equation}

Define generators $W_{\alpha ,\beta }$ of $\overline{\mathcal{S}p}(
2n) $ in terms of elements in the enveloping algebra of the Weyl-Heisenberg
group, $W_{\alpha ,\beta }=\zeta _{\alpha ,\gamma }Z_{\gamma
}Z_{\beta }+\zeta _{\beta ,\gamma }Z_{\gamma }Z_{\alpha }$. The
commutation relations satisfy the Lie algebra of the symplectic
group up to a central factor $I$,
\begin{equation}
\begin{array}{l}
 \left[ W_{\alpha ,\beta },Z_{\kappa }\right] =I \left( \zeta _{\alpha
,\kappa }Z_{\beta }+\zeta _{\beta ,\kappa }Z_{\alpha }\right) ,
\\
 \left[ W_{\alpha ,\beta },W_{\kappa ,\epsilon }\right] =I( \zeta
_{\alpha ,\epsilon }W_{\beta ,\kappa }+\zeta _{\alpha ,\epsilon
}W_{\beta ,\kappa }+\zeta _{\alpha ,\epsilon }W_{\beta ,\kappa }+\zeta
_{\alpha ,\epsilon }W_{\beta ,\kappa }) .
\end{array}
\end{equation}

The $\rho ^{\prime }$ Hermitian representations of generators ${\hat{W}}_{\alpha
,\beta }=\zeta _{\alpha ,\gamma }{\hat{Z}}_{\gamma }{\hat{Z}}_{\beta
}+\zeta _{\beta ,\gamma }{\hat{Z}}_{\gamma }{\hat{Z}}_{\alpha }$ act
on the Hilbert space ${\text{\boldmath $\mathrm{H}$}}^{\xi }$,
\begin{equation}
\begin{array}{l}
  \left[ {\hat{W}}_{\alpha ,\beta },{\hat{Z}}_{\kappa }\right] =i
\lambda ( \zeta _{\alpha ,\kappa }{\hat{Z}}_{\beta }+\zeta _{\beta
,\kappa }{\hat{Z}}_{\alpha }) ,  \\
 \left[ {\hat{W}}_{\alpha ,\beta },{\hat{W}}_{\kappa ,\epsilon }\right]
=i \lambda ( \zeta _{\alpha ,\epsilon }{\hat{W}}_{\beta ,\kappa
}+\zeta _{\alpha ,\epsilon }{\hat{W}}_{\beta ,\kappa }+\zeta _{\alpha
,\epsilon }{\hat{W}}_{\beta ,\kappa }+\zeta _{\alpha ,\epsilon }{\hat{W}}_{\beta
,\kappa }) .
\end{array}%
\label{repofhspalg}
\end{equation}

\noindent Rescaling the generators ${\hat{Z}}_{\beta }\rightarrow
\frac{1}{\sqrt{\lambda }}{\hat{Z}}_{\beta }$ removes the $\lambda
$ and (\ref{Heisenbergcomm}) and (\ref{repofhspalg}) define
the required Hermitian representation $\rho ^{\prime }$ of the Lie
algebra of $\overline{\mathcal{H}\mathcal{S}p}( 2n) $ acting
on the Hilbert space ${\text{\boldmath $\mathrm{H}$}}^{\xi }$ to
establish that it is a perfect-Mackey group. \

\subsection{Unitary representations of $\overline{\mathcal{H}\mathcal{S}p}(
2n) $}

By Theorem \ref{Theorem:projrep}, the projective representations
of $\mathcal{I}\mathcal{S}p( 2n) $ are equivalent to the ordinary
unitary representation $\overline{\mathcal{H}\mathcal{S}p}( 2n)
$. As $\overline{\mathcal{H}\mathcal{S}p}( 2n) $ is a perfect-Mackey
group, its faithful unitary representations are $\upsilon =\sigma
\otimes \rho $. That is, for $g\in \mathcal{G}$, $h\in \mathcal{H}(
n) $, $k\in \overline{\mathcal{S}p}( 2n) $, $g=h k$,
\begin{equation}
\upsilon ( g) =\upsilon ( h k) =\sigma ( k) \otimes \rho ( h k)
=\sigma ( k) \otimes \xi ( h) \rho ( k) .%
\label{unitrephsp}
\end{equation}

\noindent $\sigma $ are ordinary unitary irreducible representations
of $\overline{\mathcal{S}p}( 2n) $,
\begin{equation}
\sigma ( k) :{\text{\boldmath $\mathrm{H}$}}^{\sigma }\rightarrow
{\text{\boldmath $\mathrm{H}$}}^{\sigma },\  k\in \overline{\mathcal{S}p}(
2n) .%
\label{unitrepsp}
\end{equation}

\noindent $\xi $ are ordinary unitary irreducible representations
of $\mathcal{H}( n) $,
\begin{equation}
\xi ( h) :{\text{\boldmath $\mathrm{H}$}}^{\xi }\rightarrow {\text{\boldmath
$\mathrm{H}$}}^{\xi },\  h\in \mathcal{H}( n) .
\end{equation}

\noindent $\rho $ are the unitary representations of $\overline{\mathcal{S}p}(
2n) $ that satisfy (\ref{perfectmackeyaut}),
\begin{equation}
\rho ( k) :{\text{\boldmath $\mathrm{H}$}}^{\xi }\rightarrow {\text{\boldmath
$\mathrm{H}$}}^{\xi },\  k\in \overline{\mathcal{S}p}( 2n) .%
\label{rhospbar}
\end{equation}

Note that the defining equations for the Weil metaplectic representation
are
\begin{gather}
\tilde{\rho }( k) :{\text{\boldmath $\mathrm{H}$}}^{\xi }\rightarrow
{\text{\boldmath $\mathrm{H}$}}^{\xi },\  k\in \mathcal{M}p( 2n),
\end{gather}
\begin{equation}
\xi ( \tilde{h}) =\xi ( k h k^{1}) =\tilde{\rho }( k) \xi ( h) {\tilde{\rho
}( k) }^{-1}\ \   \forall  h\in \mathcal{H}( n).
\end{equation}

\noindent where the metaplectic group $\mathcal{M}p( 2n) $ is the
double cover of the symplectic group satisfying the short exact
sequence
\begin{equation}
e\rightarrow \mathbb{Z}_{2}\rightarrow \mathcal{M}p( 2n) \rightarrow
\mathcal{S}p( 2n) \rightarrow e.
\end{equation}

\noindent $\overline{\mathcal{S}p}( 2n) $ is the cover of $\mathcal{M}p(
2n) $ with the short exact sequence, $e\rightarrow \mathbb{Z}/\mathbb{Z}_{2}\rightarrow
\overline{\mathcal{S}p}( 2n) \overset{\pi }{\rightarrow }\mathcal{M}p(
2n) \rightarrow e$. Then $\rho =\tilde{\rho }\circ \pi $ is the
$\rho $ representation (\ref{rhospbar}) of $\overline{\mathcal{S}p}(
2n) $ that satisfies (\ref{perfectmackeyaut}). These representations
may be computed explicitly as described in Chapter 4 of \cite{folland}
and \cite{Low}.

In principle we can compute the ordinary unitary irreducible representations
$\sigma $ of the cover of the symplectic group (\ref{unitrepsp}). These
may be combined in (\ref{unitrephsp}) to give the faithful unitary
irreducible representations of the group $\overline{\mathcal{H}\mathcal{S}p}(
2n) $. The degenerate representations that are the unitary representations
of the groups that are homomorphic to $\overline{\mathcal{H}\mathcal{S}p}(
2n) $ must also be considered for a complete set of representations.
These include $\mathcal{I}\overline{\mathcal{S}p}( 2n) $ and $\overline{\mathcal{S}p}(
2n) $. The unitary representations of the inhomogeneous group
can be computed usuing the Mackey theory for abelian normal subgroups.  \

\section{Summary}

Our discussion started with physically measurable quantum transition
probabities that are given by the square of the modulus of states.
Quantum symmetries leaving the transition probabiities invariant
are described by projective representations of Lie groups. The projected
representations of the topologically connected Lie groups are equivalent
to the ordinary unitary representations of its unique maximal central
extension. This central extension is with respect to its first homotopy
group direct product with its second cohomology group. The second
cohomology group of the inhomogeneous symplect group is the one
dimensional abelian group and the central extension turns the $2n$
dimension abelian normal subgroup of translations of $\mathcal{I}\mathcal{S}p(
2n) $ into the Weyl-Heisenberg group. This is the quantum symmetry
origin of the Heisenberg commutation relations that are the Hermitian
representation of the Lie algebra of the Weyl-Heisenberg group.