\section{Introduction}\label{S1}

Cubic hypersurfaces in algebraic geometry are sort of~analogous to the baby bear's porridge in~the tale of~Goldilocks and the three bears. Hypersurfaces of~degrees one and two are too cold and uninteresting, while degrees four and higher are too hot to handle. Degree three is just right. The author thanks Dirk Kreimer and Pierre Vanhove for explaining the importance for physics of~two-loop Feynman graphs whose amplitudes are (mixed, or relative) periods of~certain singular cubic hypersurfaces. In particular, the paper~\cite{K} suggests the central role the singularities of~the second Symanzik hypersurface play. The basic setup of~iterated subdivision of~the two-loop sunset grew out of~collaboration with Vanhove~\cite{V}. The kite graph (written $(2,1,2)$ in the notation explained in Appendix~\ref{app}) also leads to an elliptic curve. The situation for the kite is more elementary since one can fall back on the classical theory of~cubic threefolds with isolated double points in~$\P^4$. Vanhove has been able to calculate the $j$-invariant for the resulting elliptic curve. A similar calculation for the double box looks difficult. The author was also influenced by unpublished work of Matt Kerr classifying Feynman motives associated to a~number of~other two-loop diagrams.

This paper focuses on what is perhaps the most striking example, the massive second Syman\-zik motive associated to the double box graph, see Figure~\ref{fig1}.
The massive second Symanzik in our case is a~(singular) cubic hypersurface $X_{3,1,3}$ in $\P^6$. For a smooth cubic hypersurface $X\subset \P^6$, the Hodge structure is the Hodge structure associated with the cohomology in middle dimension, in~this case $H^5(X,{\mathbb Q})$. It is known \cite[p.~16]{H}, that $H^5(X,{\mathbb Q})\cong H^1(A,{\mathbb Q}(-2))$ where $A$ is a certain abelian variety of dimension $21$. For the double box hypersurface with generic momenta and masses, $H^5(X_{3,1,3},{\mathbb Q})$ is a mixed Hodge structure with weights $\le 5$. We will construct a specific resolution of singularities $\pi\colon Z \to X_{3,1,3}$. We will show $F^3H^5(Z) ={\mathbb C}$ and $F^4H^5(Z) =(0)$. From this it will follow that as a Hodge structure, $H^5(Z,{\mathbb Q}) \cong H^1(E,{\mathbb Q}(-2))$ for a suitable elliptic curve $E$. (We do not specify the elliptic curve.) The dimension of the abelian variety drops from $21$ to $1$!

\begin{figure}[ht]\centering
\begin{tikzpicture}
\draw (-3,1) -- (3,1) -- (3,-1) -- (-3,-1) -- (-3,1);
\draw (0,2) -- (0,-2);
\draw (-3.75,1.75) -- (-3,1);
\draw (-3.75,-1.75) -- (-3,-1);
\draw (3.75,-1.75) -- (3,-1);
\draw (3.75,1.75) -- (3,1);
\node at (-1.5,-0.75) {\small 1};
\node at (-1.5,0.75) {\small 3};
\node at (1.5,-0.75) {\small 7};
\node at (-0.15,0) {\small 4};
\node at (1.5,0.75) {\small 5};
\node at (2.85,0) {\small 6};
\node at (-2.85,0) {\small 2};
\end{tikzpicture}
\\[2ex]
\begin{tikzpicture}
\draw (0,1.5) -- (0,-1.5);
\draw (0,1) .. controls (0.75,0.75) and (0.75,-0.75) .. (0,-1);
\draw (0,1) .. controls (-0.75,0.75) and (-0.75,-0.75) .. (0,-1);
\node at (-0.5,-0.85) {\small 1};
\node at (-0.5,0.85) {\small 3};
\node at (0.5,-0.85) {\small 7};
\node at (0.15,0) {\small 4};
\node at (0.5,0.85) {\small 5};
\node at (0.75,0) {\small 6};
\node at (-0.75,0) {\small 2};
\filldraw (0,1) circle (0.04);
\filldraw (0,-1) circle (0.04);
\filldraw (-0.5,-0.4) circle (0.04);
\filldraw (0.5,-0.4) circle (0.04);
\filldraw (0.5,0.4) circle (0.04);
\filldraw (-0.5,0.4) circle (0.04);
\draw (-0.5,-0.4) -- (-1,-0.65);
\draw (0.5,-0.4) -- (1,-0.65);
\draw (0.5,0.4) -- (1,0.65);
\draw (-0.5,0.4) -- (-1,0.65);
\node at (2,0) {\small $=(3,1,3)$};
\node at (-3.0,0.2) {\small $\leftrightarrow$ subdivided};
\node at (-2.6,-0.2) {\small sunset graph:};
\filldraw[transparent] (4.2,0) circle (0.04);
\end{tikzpicture}
\caption{Double box graph.}\label{fig1}
\end{figure}

{\samepage From the structure of the map $\pi$ we will deduce our main result
\begin{thm}
With the above notation,
\begin{gather*}
H^1(E,{\mathbb Q}(-2)) \cong {\rm gr}^W_5H^5(X_{3,1,3},{\mathbb Q}).
\end{gather*}
\end{thm}}

The proof is given in three steps.
\begin{enumerate}\itemsep=0pt
\setlength{\leftskip}{0.45cm}
\item[\textit{Step} 1.] We construct the resolution of singularities $Z \to X_{3,1,3}$ and we show $F^3H^5(Z,{\mathbb C})\cong {\mathbb C}$. This is Theorem~\ref{mainthm} below.

\item[\textit{Step} 2.] We show $F^4H^5(Z,{\mathbb C}) = (0)$, so the Hodge type of $H^5(Z)$ is $(0,0,1,1,0,0)$, from which it follows that $H^5(Z)\cong H^1(E,{\mathbb Q}(-2))$ for a suitable elliptic curve.

\item[\textit{Step} 3.] We show $H^5(Z,{\mathbb Q})\cong {\rm gr}^W_5H^5(X_{3,1,3},{\mathbb Q})$.
\end{enumerate}

The study of motives associated to two-loop graphs involves a lot of detailed algebraic geometry. The author can only hope he has gotten the details right! Another approach, which might yield more information in this case about the elliptic curve, could be based on the study of~linear spaces contained in the second Symanzik variety. For example, projecting from a double point on the variety realizes a singular cubic hypersurface of dimension $n$ as (birational to) the blowup of an intersection of a quadric and a cubic hypersurface on~$\P^n$~\cite{C}. When $n=5$, this intersection will be a (singular) Fano threefold, and one may expect a rough dictionary
$$\begin{array}{c}
\text{intermediate jacobian of Fano} \leftrightarrow \text{lines on Fano} \leftrightarrow
\text{planes on cubic fivefold}
\\[.5ex]
\leftrightarrow \text{intermediate jacobian of cubic fivefold.}
\end{array}$$
The weight $5$ piece of $H^5$ for the double box $X$ (with generic parameters) is seen to have Hodge type $(2,3)$, $(3,2)$ which suggests the role of~$\P^2$'s. Namely, we might expect a correspondence of~the form
\begin{gather*}
H^1(C) \xrightarrow{\alpha^*} H^1(P) \xrightarrow{\beta_*} \big(W_5H^5(X)\big)(2).
\end{gather*}
Here $P \xrightarrow{\alpha} C$ is a family of $\P^2$'s parametrized by a curve $C$. A deeper understanding of the elliptic curve arising from $H^5$ of the double box~\eqref{1} should come from a study of the Fano variety of planes on~\eqref{1}. (Presumably this Fano is a surface, see \cite[Section~1]{C}.) This is a~wonderful problem! In the case of the kite graph $(2,1,2)$ one is able to write the elliptic curve as an intersection of two quadrics in $\P^3$. I do not know if similar methods will work for the double box. Another approach, suggested by the referee, would be to study the Picard--Fuchs equation associated to $H^5$ of the double box~\eqref{1}, relating it if possible to the Picard--Fuchs equation for~a~family of elliptic curves.

More generally it would be interesting to generalize motivic considerations to cover all $2$-loop graphs with generic parameters. A physicist in the audience at the Newton institute suggested that the non-planar case would be more subtle. I asked him to email me so we could correspond, but he never did, so I cannot give him credit, but he is certainly correct. A calculation for the graph $(3,2,2)$ shows that $F^3H^5$ (resolution of $(3,2,2)$) has dimension $\ge 5$. On the other hand, the graphs $(n,1,n)$ seem better behaved.

\section[Singularities of X\{3,1,3\}]{Singularities of $\boldsymbol{X_{3,1,3}}$}
We focus now on singularities for the double box (for details, see Proposition~\ref{2symm1n})
\begin{gather}
X_{3,1,3}\colon\ 0=\Psi_{3,1,3}=Q(x_5,x_6,x_7)(x_1+x_2+x_3+x_4)\nonumber
\\ \hphantom{X_{3,1,3}\colon\ 0=\Psi_{3,1,3}=}
{}+Q'(x_1,x_2.x_3)(x_4+x_5+x_6+x_7) +x_4{\mathcal A}. \label{1}
\end{gather}

Here ${\mathcal A}$ has degree $2$ and is linear separately in each variable $x_1,\dots,x_7$.
\begin{prop}
The singular locus $X_{3,1,3,{\rm sing}}= C\amalg C'\amalg {\mathcal S}$, where
\begin{gather*}
C\colon\ Q(x_5,\dots,x_7) = 0 = x_{1}=x_{2}=\cdots = x_{4}, \nonumber
\\
C'\colon\ Q'(x_{1},\dots,x_{3}) = 0 = x_{4}=x_{5}=\cdots = x_{7}
\end{gather*}
Here $C$ $($resp.~$C')$ is a smooth quadric in $\P^{2}$. Finally ${\mathcal S}$ is a finite set of ordinary double points on $X_{3,1,3}$ defined by
\begin{gather*}
Q(x_5,x_6,x_7)=Q'(x_1,x_2,x_3)=x_4=x_1+x_2+x_3=x_5+x_6+x_7={\mathcal A}=0
\end{gather*}
The double points ${\mathcal S}$ are disjoint from $C$ and $C'$.
\end{prop}
\begin{proof} The reader can easily check that points of $C$, $C'$, and ${\mathcal S}$ are singular on $X_{3,1,3}$. We~can understand points of ${\mathcal S}$ by first imposing the linear relations $x_4=x_1+x_2+x_3=x_5+x_6$ $+\,x_7=0$,~so
\begin{gather*}
{\mathcal S}\colon\ Q(x_5,x_6,-x_5-x_6)=Q'(x_1,x_2,-x_1-x_2)
\\ \hphantom{{\mathcal S}\colon\ Q(x_5,x_6,-x_5-x_6)}
{}={\mathcal A}(x_1,x_2,-x_1-x_2,0,x_5,x_6,-x_5-x_6)=0
\end{gather*}
in $\P^3_{x_1,x_2,x_5,x_6}$.
\end{proof}
\begin{Remark}It will be important in the sequel that points in ${\mathcal S}$ all lie on the hyperplane $x_4=0$.
\end{Remark}
\section{Blowings up}
We blow up $C\amalg C'$ and ${\mathcal S}$ on $\P^6$ and on $X_{3,1,3}$, obtaining the diagram
\begin{equation*}\qquad\qquad\qquad\qquad\qquad\qquad
\begin{CD}Z @>>> W \\
@VVV @VV\sigma=\text{blow }{\mathcal S} V \\
Y @>>> V \\
@VVV @VV \pi=\text{blow $C\amalg C'$} V \\
X_{3,1,3} @>>> \P^6
\end{CD}
\end{equation*}
The exceptional divisor in $V$ is written $E\amalg E'$ with
\begin{gather*}
E = \P\big(N^\vee_{C\subset \P^6}\big),\qquad E' = \P\big(N^\vee_{C'\subset \P^6}\big).
\end{gather*}
Both conormal bundles have the form $N^\vee = {\mathcal O}(-1)^4\oplus {\mathcal O}(-2)$. Since ${\mathcal S}$ is disjoint from $E\amalg E'$, the map $\pi\circ\sigma$ is itself a blowup, with exceptional fibre
\begin{gather*}
\amalg_6 \P^5\amalg E\amalg E'.
\end{gather*}
\begin{lem}$Z$ is a smooth divisor on $W$.
\end{lem}
\begin{proof}The points of ${\mathcal S}$ are ordinary double points on $X_{3,1,3}$. Since the fibres of $Z$ over these points are smooth quadrics, it suffices to show $Y\cap (E\amalg E')$ is smooth. The situation is symmetric so we can focus on $Y\cap E$. We have $C\colon x_1=x_2=x_3=x_4=Q(x_5,x_6,x_7)=0$. Let $I\subset {\mathcal O}_{\P^6}$ be the ideal of $C$. Since $\Psi_{3,1,3}$ vanishes to order $2$ along $C$, $\Psi$ defines a map ${\mathcal O}_{\P^6}(-3) \to I^2$. The image of this map defines a homogeneous sheaf of ideals of degree $2$ for the graded sheaf of~algebras~$\bigoplus I^k/I^{k+1}$ with corresponding variety $E\cap Y$.

To avoid having to keep track of the grading on $I/I^2$, write $y_i=x_i/x_5$. Write $q = Q(x_5,x_6,x_7)/x_5^2$ for the coordinate in the fibre. In contrast $q'(y_1,y_2,y_3)=Q'(x_1,x_2,x_3)/x_5^2$. The defining equation locally looks like
\begin{gather}\label{6}
q(y_1+y_2+y_3+y_4)+q'(y_1,y_2,y_3)(1+y_6+y_7)+y_4L(y_1,y_2,y_3,y_4).
\end{gather}
Here $L$ is defined (compare the definition of ${\mathcal A}$ in~\eqref{11a}).
\begin{gather*}
L(y_1,y_2,y_3,y_4) = (y_1,y_2,y_3,y_4)
\begin{pmatrix}a_{15}&a_{16}&a_{17} \\a_{25}&a_{26}&a_{27} \\
a_{35}&a_{36}& a_{37}\\ a_{45}&a_{46}& a_{47} \end{pmatrix}
\begin{pmatrix}1\\y_6 \\y_7\end{pmatrix}\!.
\end{gather*}
Note that~\eqref{6} does not vanish to order $\ge 2$ at any point of $E\cap Y$. (It may clarify to remark that $E\cap Y$ is not smooth over $C$. Indeed, at the point of $C$ defined by $1+y_6+y_7=0$, the equation in the fibre becomes $q(y_1+y_2+y_3+y_4)+y_4L(y_1,y_2,y_3,y_4)=0$ and this can vanish to order $2$.)
\end{proof}
Smoothness of $Z$ permits us to calculate the Hodge filtration on $H^6(W-Z)$ using the pole order filtration \cite[Chapter~II, Proposition~3.13]{D}. $W$ is obtained by blowing up points and smooth rational curves on $\P^6$, so all cohomology classes on $W$ are Hodge. In particular the Gysin sequence yields
\begin{gather}\label{8}
F^3H^5(Z) \cong F^4H^6(W-Z) \cong \H^2\big(W,\Omega^4_W(Z) \xrightarrow{d} \Omega^5_W(2Z) \xrightarrow{d}\Omega^6_W(3Z)\big).
\end{gather}

\section{Leray}
To evaluate $F^3H^5(Z)$ in~\eqref{8}, we will use the Leray spectral sequence for $\pi\colon W \to \P^6$.
\begin{prop}\label{vanishing}
We have $R^i\pi_*\Omega^j_W(kZ)=(0)$, $i, k\ge 1$, $j\ge 0$.
\end{prop}
\begin{proof}For $i\ge 1$ it is clear that the support of $R^i\pi_*\Omega^j_W(kZ)$ is contained in $C\amalg C'\amalg {\mathcal S}$.
\begin{lem} For $i\ge 1$, $R^i\pi_*\Omega^j_W(kZ)$ is killed by the ideal $J\subset {\mathcal O}_{\P^6}$ of functions vanishing on~$C\amalg C'\amalg {\mathcal S}$.
\end{lem}
\begin{proof}[Proof of lemma]
As a consequence of Zariski's formal function theorem,
\begin{gather*}
R^i\pi_*\Omega^j_W(kZ)^\wedge \cong \varprojlim_n R^i\pi_*\big(\Omega^j_W(kZ)\otimes {\mathcal O}_W/J^n\big).
\end{gather*}
Since completion is faithfully flat, it will suffice to show the individual sheaves $R^i\pi_*\big(\Omega^j_W(kZ)\otimes {\mathcal O}_W/J^n\big)$ are killed by $J$. For this it will suffice to show
\begin{gather*}
R^i\pi_*\big(\Omega^j_W(kZ)\otimes J^{n-1}/J^n\big) = (0),\qquad i\ge 1,\qquad n\ge 2,\qquad j,k\ge 0.
\end{gather*}
Since $X_{3,1,3}$ vanishes to order $2$ along $C\amalg C'\amalg {\mathcal S}$, it follows that locally, the divisor $Z\cdot{}$excep\-tional divisor corresponds to the line bundle ${\mathcal O}_{\P(N^\vee)}(2)$ where $N^\vee$ is the conormal bundle of $C\amalg C'\amalg {\mathcal S}$. Thus we need to show
\begin{gather*}
R^i\pi_*\big(\Omega_V^j\otimes {\mathcal O}_{\P(N^\vee)}(2k+n-1)\big)=(0).
\end{gather*}
For $j=0$ this is standard because we are computing cohomology in degree $>0$ of a positive multiple of the tautological bundle on projective space. For $j>0$, we have exact sequences (writing $E$ for the exceptional divisor)
\begin{gather}
0 \to {\mathcal O}_E(-E) \to \Omega^1_W|E \to \Omega^1_E \to 0,\nonumber
\\
0 \to \Omega^{j-1}_{\P(N^\vee)}(1) \to \Omega^j_W|E \to \Omega^j_{\P(N^\vee)} \to 0. \label{12}
\end{gather}
It therefore suffices to show $R^i\pi_*\big(\Omega^\ell_{\P(N^\vee)}(m)\big)=(0)$ for $i, m\ge 1$, $\ell > 0$. We have a filtration on~$\Omega^\ell_{\P(N^\vee)}$ locally over the base with graded pieces $\Omega^q_{\P(N^\vee)/C\amalg C'\amalg {\mathcal S}}$, $q=\ell-1, \ell$. The assertion is local on the base, so finally we have to check that $H^i\big(\P^3, \Omega^q_{\P^3}(m)\big)=(0)$ for $i, m\ge 1$. This is standard, because $\Omega_{\P^s}^j$ admits a resolution by direct sums of line bundles ${\mathcal O}(-r)$ with $r\le s+1$.
\end{proof}
We return now to the proof of Proposition~\ref{vanishing}. Since for $i\ge 1$, $R^i\pi_*\big(\Omega^j_W(kZ)\big)$ is supported on~$C\amalg C'\amalg {\mathcal S}$, it follows from the lemma that
\begin{gather*}
R^i\pi_*\big(\Omega^j_W(kZ)\big) \cong R^i\big(\pi|_{\pi^{-1}(C\amalg C'\amalg {\mathcal S})}\big)_*\big(\Omega^j_W(kZ)\big)|_{(\pi^{-1}(C\amalg C'\amalg {\mathcal S})}.
\end{gather*}
Finally, it follows from~\eqref{12} that $R^i\pi_*(\Omega^j_W(kZ)) = (0)$ for $k\ge 1$, proving Proposition~\ref{vanishing}.
\end{proof}

In order to evaluate~\eqref{8}, we need finally to calculate $\pi_*\Omega^{3+m}_W(mZ)$ for $m=1,2,3$. Note that sections of $\pi_*\Omega^{3+m}_W(mZ)$ coincide with sections $\omega$ of $\Omega^{3+m}_{\P^6}(mX)$ such that $\pi^*(\omega)$ has no pole along the exceptional divisor. First consider what happens over the ordinary double points~${\mathcal S}$. Here $\Psi_{3,1,3}$ vanishes to order $2$ so a pullback of a section of $\Omega^{3+m}_{\P^6}(mX)$ gets a pole of order~$2m$ along the exceptional divisor coming from the pullback of $mX$. On the other hand if the singular point is defined by $z_1=\cdots =z_6=0$, then writing ${\rm d}z_j = {\rm d}(z_i(z_j/z_i))$ it follows that any form ${\rm d}z_{i_1}\wedge\cdots\wedge {\rm d}z_{i_{3+m}}$ downstairs pulls back to a form vanishing to at least order $2+m$ along the exceptional divisor upstairs. For $m=1,2$ the pole order $2m-(2+m)$ upstairs is $\le 0$ so there is no pole. On the other hand, for $m=3$ we get a pole of order $1$ along the exceptional divisor. Let ${\mathcal I}({\mathcal S})\subset {\mathcal O}_{\P^6}$ be the ideal of functions vanishing on ${\mathcal S}$. Then
\begin{gather*}
{\mathcal I}({\mathcal S})\Omega^6_{\P^6}(3X)
\end{gather*}
have no poles on the exceptional divisor lying over ${\mathcal S}$.

Over the curves $C$ and $C'$ local defining equations for the singular set have the form $z_1=\cdots =z_5=0$. A complete set of parameters locally on $\P^6$ becomes $z_1,\dots, z_5,\, t$ for some $t$. Locally, a $4$-form downstairs pulls back to have a zero of order at least $2$ on the exceptional divisor upstairs, from which it follows that
\begin{gather*}
\pi_*\Omega^4_W(Z) = \Omega^4_{\P^6}(X).
\end{gather*}
Sections of $\Omega^4_{\P^6}$ pull back to vanish to order $4$ on the exceptional divisor, so writing $I\subset {\mathcal O}_{\P^6}$ for the ideal of functions vanishing on $C\amalg C'$, we have
\begin{gather*}
\pi_*\Omega^6_W(3Z) = ({\mathcal I}({\mathcal S}))\cap I^2\Omega^6_{\P^6}(3X).
\end{gather*}

Finally, over $C\amalg C'$, the situation for $\pi_*\Omega^5_W(2X)$ is more complicated. Let ${\mathcal I}\subset {\mathcal O}_{\P^6}$ be the ideal of $C\amalg C'$. Write
\begin{gather*}
\Omega^1_{\P^6,{\mathcal I}}:={\mathcal I}\Omega^1_{\P^6}+d{\mathcal I} = \ker\big(\Omega^1_{\P^6} \to \Omega^1_{C\amalg C'}\big)
\end{gather*}
We have
\begin{gather*}
\pi_*\Omega^5_W(2Z) = \Omega^5_{\P^6,{\mathcal I}}(2X):=\operatorname{Image}\bigg(\bigwedge^5 \Omega^1_{\P^6, {\mathcal I}} \to \Omega^5_{\P^6}\bigg)(2X).
\end{gather*}
Since cohomology in degree $>0$ of $\Omega^i_{\P^n}(j)$ vanishes for $j>0$, we conclude in~\eqref{8} that
\begin{gather*}
F^3H^5(Z) \cong
\H^2\big(W, \Omega^4_{\P^6}(X)\to \Omega^5_{\P^6,{\mathcal I}}(2X)\to \big({\mathcal I}({\mathcal S})\cap I^2\big)\Omega^6_{\P^6}(3X)\big) \\ \hphantom{F^3H^5(Z)}
{}\cong\H^1\big(\P^6, \Omega^5_{\P^6,{\mathcal I}}(2X) \to \big({\mathcal I}({\mathcal S})\cap I^2\big)\Omega^6_{\P^6}(3X)\big).
\end{gather*}

\section[The computation of F3H5(Z)]{The computation of $\boldsymbol{F^3H^5(Z)}$}
\begin{thm}
\label{mainthm} With notation as above, we have $F^3H^5(Z) \cong {\mathbb C}$.
\end{thm}

\begin{proof}
\begin{lem}\label{firstlem}
$H^0\big( \Omega^5_{\P^6,{\mathcal I}}(2X)\big)=(0)$.
\end{lem}
\begin{proof}[Proof of lemma] We have $\Omega^5_{\P^6}(2X) \cong T_{\P^6}(-1)$. Also ${\mathcal I}\Omega^5_{\P^6}\subset \Omega^5_{\P^6,{\mathcal I}}(2X)$, so
\begin{gather*}
T_{\P^6}|_{C\amalg C'}(-1) \twoheadrightarrow \Omega^5_{\P^6}(2X)/\Omega^5_{\P^6,{\mathcal I}}(2X).
\end{gather*}
Locally $\Omega^5_{\P^6,{\mathcal I}}$ is generated by ${\mathcal I}\Omega^5_{\P^6}$ and ${\rm d}z_1\wedge\cdots \wedge {\rm d}z_5$. It follows that
\begin{gather*}
\Omega^5_{\P^6}(2X)/\Omega^5_{\P^6,{\mathcal I}}(2X) \cong N_{C\amalg C'/\P^6}(-1) = T_{\P^6}(-1)|_{C\amalg C'}/T_{C\amalg C'}(-1).
\end{gather*}
\minCDarrowwidth.6cm
We build a diagram (ignore for the moment the $\delta_i$ and the bottom line which will only be used in the next lemma):
\begin{gather}\label{17}
\begin{CD} 0 @>>> H^0\big(T_{\P^6}(-1)\big) @ >a>> H^0\big(C\amalg C', N_{C\amalg C'/\P^6}(-1)\big) @>b>> H^1\big(\Omega^5_{\P^6, {\mathcal I}}(6)\big) @>>> 0 \\
@. @VV \delta_1 V @VV \delta_2 V @VV \delta_3 V \\
@. H^0\big({\mathcal O}_{\P^6}(2)\big) @>a'>> H^0\big({\mathcal O}_{\P^6}/{\mathcal I}^2(2)\big) @>b'>> H^1\big({\mathcal I}^2{\mathcal O}_{\P^6}(2)\big) @>>> 0
\end{CD}
\end{gather}
Recall that ${\mathcal I}$ is the ideal of $C\amalg C'$, where $C$ is defined in homogeneous coordinates by $x_1=x_2=x_3=x_4=Q(x_5,x_6,x_7)=0$ and $C'$ is defined by $x_4=x_5=x_6=x_7=Q'(x_1,x_2,x_3)=0$. The kernel of $a'$ above consists of homogeneous polynomials of degree $2$ on $\P^6$ which vanish to degree~$2$ both on $C$ and on $C'$. I.e., $\ker(a') = {\mathbb C} x_4^2$. Also $b'$ is surjective because $H^1\big({\mathcal O}_{\P^6}(2)\big)=(0)$ and~$b$ is surjective because $H^1(\P^6, T_{\P^6}(-1))=(0)$.

To see that $a$ is injective, note the Euler sequence
\begin{gather*}
0 \to {\mathcal O}_{\P^6}(-1) \to \bigoplus_0^6{\mathbb C}\frac{\partial}{\partial x_i} \to T_{\P^6}(-1) \to 0
\end{gather*}
yields
\begin{gather*}
H^0\big(\P^6, T_{\P^6}(-1)\big)= \bigoplus_0^6{\mathbb C}\frac{\partial}{\partial x_i}.
\end{gather*}
Recall $C\colon x_1=x_2=x_3=x_4=Q(x_5,x_6,x_7)=0$. The projection of $a$ onto $N_{C/\P^6}(-1)$ is given by $a(\partial/\partial x_i) = \partial/\partial x_i$, $i=1,2,3,4$ and $a(\partial/\partial x_j) = (\partial Q/\partial x_j)\partial/\partial Q$, $j=5,6,7$. This map is injective. Since $H^0\big( \Omega^5_{\P^6,{\mathcal I}}(2X)\big)=(0)\subset \ker(a)$, the lemma follows.
\end{proof}
\begin{lem}\label{lem9}
The arrow
\begin{gather*}
H^1\big(\P^6, \Omega^5_{\P^6, {\mathcal I}}(2X)\big) \xrightarrow{d} H^1\big(\P^6, ({\mathcal I}({\mathcal S})\cap {\mathcal I}^2)\Omega^6_{\P^6}(3X)\big)
\end{gather*}
is injective.
\end{lem}
\begin{proof}[Proof of lemma]
The remaining arrows in~\eqref{17} are defined as follows. We identify
\begin{gather*}
H^1\big(T_{\P^6}(-1)\big)= \bigoplus_0^6 \frac{\partial}{\partial x_i}
\end{gather*}
as above, and define
\begin{gather*}
\delta_1\bigg(\frac{\partial}{\partial x_i}\bigg) =\frac{ \partial(\Psi_{3,1,3})}{\partial x_i} \in H^0\big({\mathcal O}_{\P^6}(2)\big).
\end{gather*}
Note that in fact $\delta_1(\partial/\partial x_i) \in H^0\big({\mathcal I}{\mathcal O}_{\P^6}(2)\big)$ so we get a well-defined map
\begin{gather*}
T_{\P^6}(-1)|_{\mathcal C} \to {\mathcal O}_{\P^6}/{\mathcal I}^2(2).
\end{gather*}
Tangent vectors along ${\mathcal C}$ stabilize powers of ${\mathcal I}$ so in fact the map factors through $N_{{\mathcal C}/{\mathcal P}^6}(-1) \to {\mathcal O}_{\P^6}/{\mathcal I}^2(2)$, defining $\delta_2$ in~\eqref{17}. Finally $\delta_3$ exists because the top row in~\eqref{17} is exact.

It is straightforward to check that the diagram
$$
\begin{CD}
H^1\big(\P^6, \Omega^5_{\P^6, {\mathcal I}}(2X)\big)@>d>> H^1\big(\P^6, \big({\mathcal I}({\mathcal S})\cap {\mathcal I}^2\big)\Omega^6_{\P^6}(3X)\big) \\
@VV \delta_3 V @VVV \\
H^1\big({\mathcal I}^2{\mathcal O}_{\P^6}(2)\big) @= H^1\big(\P^6, {\mathcal I}^2\Omega^6_{\P^6}(3X)\big)
\end{CD}
$$
is commutative, so the lemma will follow if we show $\delta_3$ is injective. It is convenient to isolate this statement as a separate sublemma.
\begin{lem}[sublemma]\label{sublem10}
With reference to diagram~\eqref{17}, we have
\begin{gather*}
\operatorname{Image}(\delta_2)\cap \operatorname{Image}(a') = \operatorname{Image}(a'\circ \delta_1).
\end{gather*}
As a consequence, $\delta_3$ is injective.
\end{lem}
\begin{proof}
Fix $C\colon x_1=x_2=x_3=x_4=Q(x_5,x_6,x_7)=0$ to be one component of ${\mathcal C}$. Let $I\supset {\mathcal I}$ be the ideal of $C\subset {\mathcal C}$. The normal bundle with a $-1$ twist is $N_{C/\P^6}(-1) \cong {\mathcal O}_C^4\oplus {\mathcal O}_C(1)$. Here we have to be careful because ${\mathcal O}_C(1)$ is a line bundle of degree $2$ on the conic $C$ so $h^0({\mathcal O}_C(1))=3$ and the composition
\begin{gather*}
H^0\big(T_{\P^6}(-1)\big) \xrightarrow{a} H^0\big({\mathcal C},N_{{\mathcal C}/\P^6}(-1)\big) \xrightarrow{\text{proj}} H^0\big(C,N_{{\mathcal C}/\P^6}(-1)\big)
\end{gather*}
is an isomorphism of vector spaces of dimension $7$.

Suppose now we have in the sublemma that $\delta_2(u)=a'(v)$. By the above, we can modify~$u$ (resp.~$v$) by some $a(w)$ (resp.~$\delta_1(w)$) so that $u$ is trivial on $C$, that is $a'(v) \in I^2(2)=(x_1,x_2,x_3,x_4)^2$. By assumption, $a'(v) \in \delta_2(H_{C'/\P^6}(-1))$ which means we can write (here $\Psi:=\Psi_{3,1,3}$)
\begin{gather}
a'(v) = c_4\frac{\partial\Psi}{\partial x_4}+\cdots +c_7\frac{\partial \Psi}{\partial x_7}
+c_8\frac{\partial Q'}{\partial x_1}\frac{\partial \Psi}{\partial Q'}+
c_9\frac{\partial Q'}{\partial x_2}\frac{\partial\Psi}{\partial Q'} + c_{10}\frac{\partial Q'}{\partial x_3}\frac{\partial \Psi}{\partial Q' }\nonumber
\\ \hphantom{a'(v)}
{}=c_4\frac{\partial \Psi}{\partial x_4}+\cdots +c_7\frac{\partial \Psi}{\partial x_7}
+ \bigg(c_8\frac{\partial Q'}{\partial x_1}+c_9\frac{\partial Q'}{\partial x_2}+c_{10}\frac{\partial Q'}{\partial x_3}\bigg)(x_4+x_5+x_6+x_7).\label{21}
\end{gather}
We want to show this expression cannot be a non-trivial quadric in $x_1,\dots,x_4$. The monomials which appear fall into $4$ classes:
\begin{gather*}
(123)^2, \qquad (123)(456),\qquad (4)(456),\qquad (567)^2.
\end{gather*}
(Here, e.g., $(123)^2$ refers to monomials $x_ix_j$ with $1\le i, j\le 3$.) The only terms in $(567)^2$ appear in~\eqref{21} with coefficient $c_4$ so we must have $c_4=0$. We have for $i=5,6,7$
\begin{gather*}
\frac{\partial \Psi}{\partial x_i} = \frac{\partial Q}{\partial x_i}(x_1+x_2+x_3+x_4)+Q'+x_4\frac{\partial {\mathcal A}}{\partial x_i}.
\end{gather*}
The terms $Q'$ and $x_4\partial {\mathcal A}/\partial x_i$ lie in $(1234)^2$. For the terms in~\eqref{21} not in $(1234)^2$ to cancel we must have for some constant $K$
\begin{gather}
c_5\frac{\partial Q}{\partial x_5} + \frac{c_6\partial Q}{\partial x_6} +\frac{c_7\partial Q}{\partial x_7}=K(x_5+x_6+x_7),\nonumber
\\
c_8\frac{\partial Q'}{\partial x_8}+c_9\frac{\partial Q'}{\partial x_9}+c_{10}\frac{\partial Q'}{\partial x_{10}}=-K(x_1+x_2+x_3).\label{22}
\end{gather}
It follows from~\eqref{21} and~\eqref{22} that $a'(v)$ in~\eqref{21} will involve a term $K(x_5+x_6+x_7)$ which does not cancel. This proves Lemma~\ref{sublem10}.
\end{proof}
The assertion (needed to prove Lemma~\ref{lem9}) that $\delta_3$ is injective is now just a diagram chase.
\end{proof}
\begin{lem}
\label{lem11}
$H^0\big(\P^6, I({\mathcal S})\cap {\mathcal I}^2\Omega^6_{\P^6}(3X_{3,1,3})\big) \cong {\mathbb C}$.
\end{lem}
\begin{proof}[Proof of lemma]
We will show that $H^0\big(\P^6, {\mathcal I}^2\Omega^6_{\P^6}(3X_{3,1,3})\big) \cong {\mathbb C}$ and that the generating section also vanishes on points of ${\mathcal S}$. In fact, ${\mathcal I}^2\Omega^6_{\P^6}(3X_{3,1,3})\cong {\mathcal I}^2_{\P^6}(2)$. Note ${\mathcal I}^2=(I_C\cap I_{C'})^2$, so~we are looking for homogenous forms of degree $2$ which vanish to order 2 on $C$ and on $C'$. The only such forms lie in ${\mathbb C}\cdot x_4^2$. Finally, note that $x_4$ vanishes on ${\mathcal S}$.
\end{proof}
Theorem \ref{mainthm} follows by combining Lemmas \ref{firstlem}, \ref{lem9}, and \ref{lem11}.
\end{proof}

\section{The Hodge structure in more detail}
In this section, we justify steps $2$ and $3$ from the introduction. A general reference is~\cite{PS}.
\begin{prop
With notation as above, we have $F^4H^5(Z,{\mathbb C}) = (0).$
\end{prop}
\begin{proof} Again from \cite[Chapter~II, Proposition~3.13]{D}, we have
\begin{gather*}
\H^1\big(W,\Omega^5_W(Z) \xrightarrow{d} \Omega^6_W(2Z)\big)\cong F^5H^6(W-Z,{\mathbb C}) \cong F^4H^5(Z,{\mathbb C}).
\end{gather*}
It will suffice to show
\begin{gather*}
H^0\big(W,\Omega^6_W(2Z)\big) = 0= H^1\big(W,\Omega^5_W(Z)\big).
\end{gather*}
$Z$ is birational with the cubic $X_{3,1,3}\subset \P^6$, and a section of $\Omega^6_W$ with only a pole of order $2$ along $Z$ would give rise to a section of $\Omega^6_{\P^6}$ with only a pole of order $2$ along $X_{3,1,3}$. Since $\Omega^6_{\P^6}\cong {\mathcal O}_{\P^6}(-7)$ there are no such sections.

It remains to show $H^1\big(W, \Omega^5_W(Z)\big)=(0)$. By duality, it suffices to show $H^5\big(W, \Omega^1_W(-Z)\big)$ $=(0)$. Recall $W$ is obtained from $\P^6$ by blowing up $C\amalg C' \amalg {\mathcal S}$. The exceptional divisors are~$E$,~$E'$, $\coprod_i \P^5 \subset W$. Define $\Xi=\Omega^1_W/\pi^*\Omega^1_{\P^6}$. We claim
\begin{gather*}
\Xi \cong \Omega^1_{E/C}\oplus \Omega^1_{E'/C'}\oplus \bigoplus_i\Omega^1_{\P^5}.
\end{gather*}

Indeed there is a natural surjective map just by restricting $1$-forms from $W$ to the exceptional divisors. Consider the local structure along $E$. Locally $C\colon z_1=\cdots =z_5=0$ in $\P^6$. Locally upstairs, we have $E\colon z_1=0$ and $\Omega^1_W=\pi^*\Omega^1_{\P^6}+\sum {\mathcal O}_W{\rm d}(z_i/z_1)$. Note
\begin{gather*}
z_1{\rm d}(z_i/z_1)={\rm d}z_i-(z_i/z_1){\rm d}z_1 \in \pi^*\Omega^1_{\P^6}.
\end{gather*}
This implies that $\pi^*\Omega^1_{\P^6}\supset {\mathcal O}_W(-E)\Omega^1_W$, so $\Omega^1_W|_E \twoheadrightarrow \Xi$. Also the conormal bundle $N^\vee(E/W)= {\mathcal O}_W(-E)/{\mathcal O}_W(-2E)$ of $E\subset W$ lies in $\Omega^1_W|_E$. Similarly, the conormal bundle $N^\vee_{C/\P^6}$ lies in~$\Omega^1_{\P^6}|_C$, and the pullback $\pi^*N^\vee_{C/\P^6} \to N^\vee(E/W)$ is surjective. It follows that
\begin{gather*}
\Omega^1_E = \big(\Omega^1_W|_E\big)/N^\vee(E/W) \twoheadrightarrow \Xi.
\end{gather*}
The proof of~\eqref{24} is now straightforward.

We have the exact sequence
\begin{gather} \label{24}
0 \to \pi^*\Omega^1_{\P^6}(-Z) \to \Omega^1_W(-Z) \to \Xi(-Z) \to 0.
\end{gather}
In the Picard group of $W$ we have $-Z=-\pi^*(X_{3,1,3})+2E+2E'+2\sum_{s\in {\mathcal S}}\P^5_s$. We compute the direct images
\begin{gather}\label{25}
{\mathbf R}\pi_*\big(\pi^*\Omega^1_{\P^6}(-Z)\big) = \Omega^1_{\P^6}(-3)\otimes {\mathbf R}\pi_*\bigg({\mathcal O}_W\bigg(2E+2E'+2\sum_{s\in {\mathcal S}}\P^5_s\bigg)\!\bigg).
\end{gather}
The calculus of intersection theory on blowups \cite[Section~II.7]{Ha}, yields $E\cdot E = [{\mathcal O}_E(-1)]$, the dual of the tautological relatively ample bundle on the projective bundle $E$ over $C$ (and similarly for~$E'$ and the $\P^5_s$). Applying $\mathbf R\pi_*$ to the exact sequences
\begin{gather*}
0 \to {\mathcal O}_W\Big(E+E'+\sum \P^5_s\Big) \to {\mathcal O}_W\Big(2E+2E'+2\sum \P^5_s\Big)
\\ \phantom{0}
 \to {\mathcal O}_E(-2)\oplus {\mathcal O}_{E'}(-2) \oplus \bigoplus_{{\mathcal S}}{\mathcal O}_{\P^5}(-2) \to 0
\end{gather*}
and the similar sequence
\begin{gather*}
0 \to {\mathcal O}_W \to {\mathcal O}_W\Big(E+E'+\sum \P^5_s\Big) \to {\mathcal O}_E(-1)\oplus {\mathcal O}_{E'}(-1)+\sum {\mathcal O}_{\P^5_s}(-1) \to 0
\end{gather*}
and recalling that for a projective space bundle $\P \xrightarrow{\pi}S$ with fibre $\P^m$ we have ${\mathbf R}\pi_*{\mathcal O}_{\P}(-n)=(0)$ for $1\le n\le m$, we can see that
\begin{gather}\label{27}
{\mathbf R}\pi_*\bigg({\mathcal O}_W\bigg(2E+2E'+2\sum_{s\in {\mathcal S}}\P^5_s\bigg)\!\bigg) \cong {\mathbf R}\pi_*({\mathcal O}_W).
\end{gather}
We want to use these results to prove $H^5\big(W, \Omega_W^1(-Z)\big) = (0)$. From~\eqref{24}, it will suffice to show
\begin{gather*}
H^5\big(W, \pi^*\Omega^1_{\P^6}(-Z)\big) = (0) = H^5(W,\Xi(-Z)).
\end{gather*}
From~\eqref{25} and~\eqref{27} we get
\begin{gather}\label{28}
H^5\big(W, \pi^*\Omega^1_{\P^6}(-Z)\big) =H^5\big(\P^6,\Omega^1_{\P^6}(-3)\otimes {\mathbf R}\pi_*{\mathcal O}_W\big).
\end{gather}
We claim $\mathbf R\pi_*({\mathcal O}_W) \cong {\mathcal O}_{\P^6}$, i.e., $R^i\pi_*{\mathcal O}_W=(0)$ for $i\ge 1$. This can be checked locally on $\P^6$, so for notational simplicity we assume $W$ is $\P^6$ blown up along $C$. (Of course, $R^0\pi_*{\mathcal O}_W={\mathcal O}_{\P^6}$. Also $R^i\pi_*({\mathcal O}_E)=(0)$ for $i\ge 1$.) The ideal sheaf ${\mathcal O}_W(-E)$ is relatively ample (a basic fact about blowing up \cite[Section~II.7]{Ha}) so $R^i\pi_*({\mathcal O}_W(-nE))=(0)$ for $n\gg0$ and $i\ge 1$. We have $\big($note $E\cdot E = [{\mathcal O}_E(-1)]\big)$
\begin{gather*}
R^i\pi_*\big({\mathcal O}_W(-nE)\big) \to R^i\pi_*\big({\mathcal O}_W(-(n-1)E)\big) \to R^i\pi_*\big({\mathcal O}_E(n-1)\big).
\end{gather*}
By descending induction we conclude $R^i\pi_*({\mathcal O}_W(-nE))=(0)$ for $n\ge 0$ and $i\ge 1$.

Finally, from~\eqref{28} we get
\begin{gather*}
H^5\big(W, \pi^*\Omega^1_{\P^6}(-Z)\big) = H^5\big(\P^6,\Omega^1_{\P^6}(-3)\big).
\end{gather*}
Dualizing and twisting the Euler sequence yields
\begin{gather*}
0 \to \Omega^1_{\P^6}(-3) \to \bigoplus_7 {\mathcal O}_{\P^6}(-4) \to {\mathcal O}_{\P^6}(-3) \to 0,
\end{gather*}
from which we conclude the desired vanishing for $H^5\big(W, \pi^*\Omega^1_{\P^6}(-Z)\big)$.

We need finally to check vanishing for $H^5(W, \Xi(-Z))$, with $\Xi$ as in~\eqref{24}. It suffices to show for example ${\mathbf R}\pi_*\big(\Omega^1_{E/C}(-2)\big)=(0)$. We have $0 \to \Omega^1_{E/C} \to \bigoplus_5 {\mathcal O}_E(-1) \to {\mathcal O}_E \to 0$, and the sheaves ${\mathcal O}_E(-2)$ and ${\mathcal O}_E(-3)$ are fibrewise ${\mathcal O}_{\P^4}(-k), k=2, 3$ and have vanishing cohomology along the fibres in all degrees.
\end{proof}
\begin{prop}[verification of Step $3$ at the end of Section~\ref{S1}]
We have
\begin{gather*}
H^5(Z,{\mathbb Q}) \cong {\rm gr}^W_5H^5(X_{3,1,3},{\mathbb Q}).
\end{gather*}
\end{prop}
\begin{proof}
We consider the Leray spectral sequence (writing $\pi_Z=\pi|_Z\colon Z \to X_{3,1,3}$).
\begin{gather*}
E_2^{p,q} = H^p\big(X_{3,1,3},R^q\pi_{Z,*}{\mathbb Q}_Z\big) \Rightarrow H^{p+q}(Z,{\mathbb Q}).
\end{gather*}
The fibres of $\pi_Z$ are connected, so $\pi_*{\mathbb Q}_Z \cong {\mathbb Q}_{X_{3,1,3}}$ and $E_2^{5,0}=H^5(X_{3,1,3},{\mathbb Q})$. We can stratify $X_{3,1,3}$ so the fibres consist either of one point or a (possibly singular) $3$-dimensional complex quadric (over $C\amalg C'$) or a $4$-dimensional complex quadric (over the isolated singular set ${\mathcal S}$). The only odd $q\le 5$ representing a possible non-zero cohomology of a fibre is $H^3$ which can occur in isolated singular fibres for the $3$-dimensional quadric over $C\amalg C'$. But then $H^2\big(R^3\big)=(0)$. Thus, the only $E_2^{p,q}$ which contributes to $H^5(F)$ is $E_2^{5,0} = H^5(X_{3,1,3},{\mathbb Q}) \to H^5(F,{\mathbb Q})$. This map is necessarily surjective. It factors through ${\rm gr}^W_5H^5(X_{3,1,3},{\mathbb Q})$, proving Step~3.
\end{proof}


\section{Introduction}

\IEEEPARstart {A}{s} an economical and convenient alternative to depth sensors, monocular depth estimation
 has wide applications in obstacle avoidance \cite{Mancini2016FastRM}, simultaneous localization and mapping (SLAM) \cite{Tateno2017CNNSLAMRD,Guo2020SemanticHB}, robot navigation \cite{mendes2020deep}, etc., and thus has captured much attention for a long time \cite{Hu2019VisualizationOC,hu2019analysis,laina2016deeper,Eigen2014depth}.
In the past few years, it has further gained significant progress due to the fast development of deep learning.

\begin{figure}[t]
\vspace{-5mm}
\centering
\includegraphics[width=0.53\textwidth]{figs/para-accu.pdf}
\caption{The total parameters and accuracy of different methods for depth estimation. As seen that, there is a trade-off between accuracy and model's capacity. Our method achieves competitive performance compared with state-of-the-art methods with only very a small number of parameters.}
\label{fig_config}
\end{figure}

Most of the previous works focused solely on the improvement of estimation accuracy \cite{fu2018deep,laina2016deeper,hu2019revisiting}.
However, the depth estimation has to be both computationally efficient and accurate.
It is essential for robotic systems with limited computation resources.
Although several prior works have attempted to improve the computational efficiency with light-weight networks \cite{Wofk2019FastDepthFM,Nekrasov2019RealTimeJS}, the cost is a significant degradation of inference accuracy.
To the best of our knowledge, there are no prior works of light-weight depth estimation that can achieve the same level or at least comparable performance as state-of-the-art methods.

\begin{figure*}[t]
\centering
\subfigure {\includegraphics[width=\linewidth]{figs/framework2.pdf}}
\caption{The left side shows the depth histogram of NYU-v2 training set and ScanNet validation set, respectively. Both of the two histograms exhibit a long-tailed distribution and they are highly similar. The right side shows the configuration of knowledge distillation (KD) considered in this paper.
(a) is the standard method that applies KD with the original labeled set. (b) applies KD with only auxiliary unlabeled set. (c) applies KD with the original labeled set and the auxiliary unlabeled set. (d) applies KD with both the original and auxiliary labeled set.
}
\label{fig_frame}
\end{figure*}

In this paper, we delve into monocular depth estimation from the aspects of inference accuracy and computational efficiency. We aim to achieve a maximum improvement of the accuracy with the minimum cost of computation time.
Towards this end, we first introduce a light-weight network with a particular target on model compression.
In contrast to traditional encoder-decoder architecture that gradually up-samples the intermediate features,
we first compress feature maps extracted by multi-layers of the encoder to a fixed small number of channel at each scale, and then upsample them to the same resolution. These feature maps are then concatenated and fed to two convolutional layers to yield a final depth map. Thanks to the above design, the network built on MobileNet-v2 only has 1.7 M parameters, and thus yields the minimum model complexity than previous methods.

We then consider how to tackle the low inference accuracy when using a light-weight network.
A common solution in image classification is the use of knowledge distillation (KD) to boost a small network, i.e. student, with the guidance from a large network, i.e. teacher.
Previous studies tend to introduce complex strategies to improve KD, such as distilling intermediate features \cite{Romero2015FitNetsHF}, multi-teacher distillation \cite{Tarvainen2017MeanTA,Liu2019KnowledgeFI}, adversarial distillation \cite{Wang2020DeceivingIT}.

We consider a more intuitive way to promote KD with auxiliary data.  Our motivations are two-fold: first, as the depth estimation is a non-linear mapping from RGB space to depth space, KD approximates this function in a data driven approach. Therefore, the more valuable data we have, the more accurate we can approximate the mapping. Second, unlike image recognition, data without same categories is considered as out-of-distribution data which is usually harmful to KD. Since depth estimation is a regression task, auxiliary data is easy to collect. For instance, indoor scenarios have similar structures and scene scales, as seen in Fig.~\ref{fig_frame}.

 We particularly study two cases, i.e. labeled and unlabeled data.
In labeled case, ground truths can be obtained by depth sensors, e.g. Kinect. In unlabeled case, auxiliary data can be simply collected by visual camera in scenarios with similar scene scales.
Therefore, it is more practical for depth estimation to boost the performance with auxiliary data in real-world applications.
In this paper, we propose to take the following learning strategies for these two specific cases.
\begin{itemize}
    \item When auxiliary data is unlabeled, we first train the teacher with the original labeled set, and then
    apply KD with the original labeled set and the auxiliary unlabeled set. \vspace{2mm}
    \item When auxiliary data is labeled, we first train the teacher on the combination of the original set and auxiliary set. It will provide us a more discriminative teacher. KD is then applied to further boost the student with the mixed dataset.
\end{itemize}

The use of auxiliary data can actually improve light-weight depth estimation.
As shown in Fig.~\ref{fig_config}, our method can achieve comparable performance against state-of-the-art methods while only using $1\%$ parameters and outperforms other light-weight approaches by a large margin.
We conduct a number of experiments to evaluate the effectiveness of our approach, and confirm through the experiments that:
\begin{itemize}
    \item  Even without access to the original training set, we can still successfully apply KD with enough auxiliary unlabeled samples as long as they have similar scene scales to the original training samples. \vspace{2mm}
    \item  When applying KD with the original trained set and auxiliary unlabeled set together, the performance improvement is significant as the gap between the teacher and student is better filled.
    \vspace{2mm}
    \item  The improvement of the light-weight network is limited if we directly train it with the mixed dataset of both original and auxiliary labeled data due to its low capacity. On the other hand, the two-stage learning strategy, i.e. training a larger teacher and then apply KD is more effective.

\end{itemize}

\section{Related Work}

 \begin{figure*}[t]
\centering
\subfigure {\includegraphics[width=\linewidth]{figs/network.png}}
\caption{Diagram of the proposed network. The core components are the four feature fusion and compression layers (FFC). The FFC layer consists of a channel-wise attention layer and a convolutional layer, such that it can automatically attribute more weights to more important features and then fuse them to fewer channels.
}
\label{fig_arch}
\vspace{-5mm}
\end{figure*}

\subsection{Monocular Depth Estimation}
In previous studies, monocular depth estimation was solved in a supervised learning fashion by penalizing pixel-wise loss between prediction and ground truth depth \cite{Eigen2014depth,hu2019revisiting}. Similar to other vision tasks, e.g. semantic segmentation, different networks have been continuously proposed. Starting from the basic encoder-decoder network \cite{laina2016deeper}, networks assembled with skip connections \cite{hu2019revisiting}, dilated convolution \cite{fu2018deep}, pyramid pooling \cite{Mendes2021OnDL}, etc. have shown advanced performance.
It can be also formulated as an unsupervised learning problem complying with the geometry consistency of multi-view images \cite{Zhou2017UnsupervisedLO,Zhan2018UnsupervisedLO}.
Currently, the performance of unsupervised learning approaches is still inferior to supervised learning methods.

Several approaches have attempted to estimate depth in real time.  In \cite{Wofk2019FastDepthFM,Nekrasov2019RealTimeJS}, light-weight networks built on MobileNet and MobileNet-v2 were introduced for fast depth estimation. The networks are trained in a traditional supervised fashion. Also, a light-weight network with  recurrent modules was proposed to solve the unsupervised depth estimation \cite{Liu2020MiniNetAE}. The methods using small networks demonstrate superior performance in terms of computation speed, however, the accuracy tends to drop significantly compared to those built upon large networks.

\subsection{Learning with Auxiliary Data}
To boost the performance of learning based methods, a recent tendency is to leverage extra labeled training datasets. As demonstrated by image recognition on ImageNet \cite{deng2009imagenet}, this strategy is indeed effective. The top1 accuracy demonstrated by those approaches \cite{pham2020meta,foret2020sharpness} is greater than $5\%$ compared with methods without utilizing extra data.

This strategy is also employed for depth estimation. In \cite{Chen2020ImprovingMD}, six auxiliary datasets are employed to handle hard cases including low light, reflecting surface, spurious edges, etc., which demonstrates a superior performance for indoor depth estimation.
Also, several methods employ multi-domain datasets, e.g. indoor and outdoor scenes, synthesized and real-world images to learn a universal network \cite{romanov2020towards,Lasinger2020TowardsRM,yin2020diversedepth,Aleotti2021RealTimeSI}. However, these methods omit the scale difference across datasets and can only estimate a normalized depth map. In this paper, we focus on indoor depth estimation and aim to reconstruct the real scale of a scene.

 Note that auxiliary unlabeled data is also commonly leveraged in image recognition under the setting of semi-supervised learning \cite{Wang2021KnowledgeDA,Xie2020SelfTrainingWN}.
In image recognition, auxiliary data has to be carefully collected such that its semantic attribute falls into categories of model's prediction.
As depth estimation is formulated as a regression task, the auxiliary data is easier to be collected in practical scenarios.

\subsection{Knowledge Distillation}
Knowledge distillation has been extensively studied in recent years. it is originally proposed to transfer the knowledge from a teacher to a student on image recognition \cite{Hinton2015DistillingTK}. Usually, the teacher is large and the student is relatively small.
Recent studies have attempted to improve KD more effectively.
Mirzadeh et al. proposed to use an assistant network between the teacher and the student  \cite{Mirzadeh2020ImprovedKD}. Some works have employed intermediate features to guide the learning of the student \cite{Huang2017LikeWY,Liu2020StructuredKD}. In \cite{Tarvainen2017MeanTA,Liu2019KnowledgeFI}, the strategy of distillation from multiple teachers has been proposed.
Another type of method to improve KD is to augment training set, such as applying GAN to generate more data \cite{Snelson2005SparseGP}, or employing a large amount of extra data on the cloud \cite{Xu2019PositiveUnlabeledCO}.

A few works also apply KD to boost depth estimation \cite{Pilzer2019RefineAD,Aleotti2021RealTimeSI}.
Unlike these methods that simply apply existing methods of KD developed for image recognition, we propose to utilize auxiliary unlabeled/labeled data to boost KD based on an observation that many scenarios have similar scene scales in real-world.

\section{Methodology}
 In this section, we first give the details of the light-weight network. Then, we present the learning strategy of applying KD with auxiliary data.

\subsection{Light-weight Network}

Most of the previous works employ a symmetric encoder-decoder to deal with pixel to pixel regression tasks \cite{laina2016deeper,ma2017sparse,Hu2020ATU}. Such network yields low inference efficiency as it requires lots of GPU memories during the computation procedure.
As shown in \cite{netdissect2017}, there is a high redundancy inside CNNs, multiple filters tend to capture same or similar feature representations.
To push the envelope of light-weight depth estimation, in this paper, we introduce an extremely compact design of network architecture.

The proposed network is shown in Fig.~\ref{fig_arch}. The core components are
four feature fusion and compression layers (FFC). The FFC layer is employed to efficiently fuse feature maps. It consists of a channel-wise attention mechanism and a convolutional layer.
 To be specific,
given a set of feature maps
extracted by a convolutional block, e.g. residual block,  we first apply channel-wise attention \cite{hu2018senet} to automatically attribute weights to them. Then, they are fused by a $5 \times 5$ convolutional layer. To reduce the model's complexity, these feature maps are compressed to a fixed number of channels (16 channels).
For features extracted with an encoder at multi-scales, we apply FFC at each scale.
Then, the outputted feature maps are upsampled to have the same size by $\times2,4,8$ and 16, respectively. Finally, they are concatenated and fed into two  $5 \times 5$ convolutional layers (denoted as \textbf{convs} in Fig.~\ref{fig_arch}) to yield a final depth map.

We use ResNet-34 and MobileNet-v2 as backbone networks for the teacher and student, respectively. As a result, the teacher and student have 21.9 M and 1.7 M parameters, respectively. The detailed information of model parameters is given in Table.~\ref{net_paras}.

\begin{table}[!t]
\begin{center}
\caption{The details of model parameters for the teacher and student.}
\label{net_paras}
\begin{tabular}{|l|c|c|c|c|}
\hline
Network  & Backbone & Encoder (M) & Decoder (M) & Total (M)\\
\hline
Teacher & ResNet-34 & 21.3 &0.6 &21.9 \\
Student & MobileNet-v2 & 1.4 &0.3 &1.7 \\
\hline
\end{tabular}
\end{center}
\end{table}

\subsection{Applying KD with Auxiliary Data}

\subsubsection{Standard KD}

We consider a classical knowledge distillation with a well pre-trained teacher on a labeled set  $\mathcal{X}$.
The supervision for training the student comes from the loss against ground truth depths and estimation from the teacher net.
Here, we use $N_t$ and $N_s$ to represent the teacher and student net, respectively.
Then, the loss for training the student is:
\begin{equation}
    \mathcal{L} = \frac{1}{\mathcal{X}} \sum_{x_i, g_i \in \mathcal{X}} (\lambda  L (N_s(x_i), N_t(x_i))  + (1- \lambda) L (N_s(x_i), g_i) )
\end{equation}
where $g_i$ denotes the ground truth for $x_i$, $L$ is an error measure between two depth maps.

We employ the error measure proposed in \cite{hu2019revisiting}, where the sum of three losses are used:
\begin{equation}
 L  = l_{\rm depth} +  l_{\rm grad} + l_{\rm normal}
\end{equation}
where $l_{\rm depth}=\frac{1}{n}\sum_{i=1}^n F(e_i)$,
$l_{\rm grad}=\frac{1}{n}\sum_{i=1}^n (F(\nabla_{x}(e_i))+F(\nabla_{y}(e_i)))$, and $ l_{\rm normal} = \frac{1}{n}\sum_{i=1}^n
    \left(1-\cos\theta_i\right),$
where $F(e_i) = \ln(e_i+0.5)$; $e_i = \|y_i - \hat{y_i}\|_1$; $y_i$ and $\hat{y_i}$ are two depth maps; and $\theta_i$ is the angle between the surface normals computed from them.

\subsubsection{Learning with Auxiliary Data}

\begin{figure}[t]
\centering
\subfigure {\includegraphics[width=0.98\linewidth]{figs/ax_res2.pdf}}
\caption{Results of the student network on NYU-v2 test set. From left to right: result of the student that learned via KD with the training set of ImageNet, result with supervised learning on NYU-v2 training set, result of the student that learned via KD with training set of ScanNet.}
\label{fig_ax_res}
\vspace{-5mm}
\end{figure}

 We assume auxiliary data is effective to KD for depth estimation as long as it has similar scene scales.
 We conduct a preliminary experiment to verify our assumption. Specifically,
 given a teacher network trained on the NYU-v2 dataset, we perform KD with cross-domain datasets.
 \begin{equation}
    \mathcal{L} =  \frac{1}{\mathcal{U}} \sum_{u_j \in \mathcal{U}} (\lambda  L (N_s(u_j), N_t(u_j))
\end{equation}
We choose ImageNet and ScanNet for the purpose, the former is an out-of-distribution dataset and the latter is also an indoor dataset.
As seen in Fig.~\ref{fig_ax_res}, we can see that the results of KD with ScanNet is slightly better than the original result and there is $15.5\%$ accuracy drop if we perform KD with ImageNet. The experimental details are given in Sec.~\ref{experimental_details}.

 The result of this preliminary experiment supports our assumption very well that even without access to the original training data, using unlabeled data with similar scene scales can still successfully perform KD for depth estimation.
It also reveals that applying KD with original set as well as auxiliary data together, the gap between the teacher and student can be better filled. Thereby, the performance of the light-weight network can be further improved.
Specifically, we consider two situations of using auxiliary data.
\paragraph{The use of auxiliary unlabeled data}
The teacher is trained with the original labeled set $\mathcal{X}$, which is denoted as $N_t$. We can access $\mathcal{X}$ as well as an auxiliary unlabeled set $\mathcal{U}$ during the procedure of KD. Then the loss for the student is formulated as:
\begin{equation}
\begin{split}
    \mathcal{L} = &
    \frac{1}{\mathcal{X}} \sum_{x_i, g_i \in \mathcal{X}} (\lambda  L (N_s(x_i), N_t(x_i))  + (1- \lambda) L (N_s(x_i),g_i) ) + \\
    &  \frac{1}{\mathcal{U}} \sum_{u_j \in \mathcal{U}}  (L (N_s(u_j), N_t(u_j))
\end{split}
\end{equation}

\paragraph{The use of auxiliary labeled data}
In this case, the auxiliary data $\mathcal{U}^{'}$ is fully labeled.
The teacher is trained on a mixed dataset, i.e. $\mathcal{X} \cup \mathcal{U}^{'}$, and it will be highly discriminative than the one trained on $\mathcal{X}$ only.
The student is then learned via KD  with mixed data:
\begin{equation}
\begin{split}
    \mathcal{L} = &
    \frac{1}{\mathcal{X}} \sum_{x_i, g_i \in \mathcal{X}} (\lambda  L (N_s(x_i), N_t(x_i))  + (1- \lambda) L (N_s(x_i),g_i) ) + \\
    &  \frac{1}{\mathcal{U}^{'}} \sum_{u_j, g_j \in \mathcal{U}^{'}}  (\lambda L (N_s(u_j), N_t(u_j))  + (1- \lambda) L (N_s(u_j),g_j) )
\end{split}
\end{equation}

\section{Experiments}

\subsection{Experimental Setting}

We use the NYU-v2 dataset \cite{Silberman2012IndoorSA} for all the experiments. The dataset consists of a variety of indoor scenes and is used in most of the previous studies. We use the standard  procedure  for preprocessing \cite{Eigen2014depth,laina2016deeper,ma2017sparse}. To be specific, the official splits of 464 scenes are used, i.e., 249 scenes for training and 215 scenes for testing.
This results in approximately 50K unique pairs of an image and a depth map of 640$\times$480 pixel size.
The images are then resized down to 320$\times$240 pixels using bilinear interpolation, and then crop their central parts of 304$\times$228 pixels, which are used as inputs to networks. The depth maps are resized to
$152\times 114$ pixels.
For testing, following the previous studies, we use the same small subset of 654 samples.
The auxiliary data is taken from the ScanNet dataset \cite{dai2017scannet}. We randomly select 204K images from 1513 scenarios.

\subsubsection{Implementation Details}
\label{experimental_details}
For the teacher net, we use the network built on ResNet-34.
For the student net, we use MobileNet-v2 as the encoder.
Both the teacher and student are trained for 20 epochs. The parameter $\lambda$ for calculating $\mathcal{L}$ is set to 0.1.
The encoder module in the network is initialized by a model pretrained with the ImageNet dataset \cite{deng2009imagenet}. The other layers in the network are randomly initialized.
We use Adam optimizer with an initial learning rate of 0.0001, and reduce it to 10$\%$ for every 5 epochs. We set $\beta_{1}=0.9$, $\beta_{2}=0.999$, and use weight decay of 0.0001.

\begin{figure*}[!t]
\centering
\subfigure []{\includegraphics[width=0.45\linewidth]{figs/res_wo_KD.pdf}}\quad
\subfigure []{ \includegraphics[width=0.45\linewidth]{figs/res_w_kd.pdf}}
\caption{(a) Results of the teacher and student network trained with supervised learning. The blue color denotes results trained on $\mathcal{X}$ and the green color denotes results with $\mathcal{X}\cup \mathcal{U'}$.  (b) Results of the student network learned with KD. The blue color denotes results of  $N_s(\mathcal{X})$, $N_s(\mathcal{U})$ and $N_s(\mathcal{X \cup U})$, respectively, and the green color denotes results of $N_s(\mathcal{X \cup U'})$.}
\label{w_wo_kd}
\end{figure*}

\begin{table*}[!t]
\begin{center}
\caption{Quantitative comparisons between our method and other approaches built on large networks on the NYU-v2 dataset.}
\label{nyu_depth_acc}
\begin{tabular}{|l|l|c|c|c|c|c|c|c|}
\hline
Method & Backbone & Original set & Auxiliary set & Params (M) $\downarrow$  & RMSE $\downarrow$ & REL $\downarrow$ &$\log10$ $\downarrow$ & $\delta_1$ $\uparrow$ \\
\hline
Laina et al. \cite{laina2016deeper}&ResNet-50 &NYU-v2 &-- &60.6 &0.573  &0.127 &0.055 &0.811 \\
Hu et al. \cite{hu2019revisiting}&ResNet-50 &NYU-v2 &-- &63.6 &0.555 &0.126 &0.054 &0.843 \\
Zhang et al. \cite{Zhang2019PatternAffinitivePA} &ResNet-50 &NYU-v2 &-- &95.4 &0.497 &0.121 &- &0.846  \\
Fu et al. \cite{fu2018deep}&ResNet-101 &NYU-v2 &-- &110.0 &0.509 &0.115 &0.051 &0.828 	 \\

Hu et al.  \cite{hu2019revisiting}&SeNet-154 &NYU-v2 &-- &149.8	&0.530 &0.115 &0.051 &0.866 	 \\
Chen et al. \cite{Chen2019StructureAwareRP}&SeNet-154 &NYU-v2 &-- &210.3 &0.514 & 0.111 &0.048 & 0.878  \\
Chen et al.\cite{Chen2020ImprovingMD}&ResNet-101  &NYU-v2 & HC  labeled data &163.4 &\textbf{0.376} &\textbf{0.098} &\textbf{0.042} & \textbf{0.899} \\ \hline
Ours $N_s(\mathcal{X} \cup \mathcal{U})$ &MobileNet-V2 &NYU-v2 & Unlabeled ScanNet &1.7 &0.482 &0.131 &0.056  &0.837\\
Ours $N_s(\mathcal{X} \cup \mathcal{U}^{'})$  &MobileNet-V2 &NYU-v2 & Labeled ScanNet &1.7 &\textbf{0.461} &\textbf{0.121} &\textbf{0.052}  &\textbf{0.855} \\ \hline
\end{tabular}
\end{center}
\end{table*}

\begin{figure*}[t]
\centering
\begin{tabular}
{p{0.15\textwidth}<{\centering}p{0.15\textwidth}<{\centering}p{0.15\textwidth}<{\centering}p{0.15\textwidth}<{\centering}p{0.15\textwidth}<{\centering}}
\IncG[ width=1.1in]{./figs/imgs/20.png}
&\IncG[ width=1.1in]{./figs/gt/gt20.png}
&\IncG[ width=1.1in]{./figs/ma/20.png}
&\IncG[ width=1.1in]{./figs/netra/20.png}
&\IncG[ width=1.1in]{./figs/ours/out20.png}
\\
\IncG[ width=1.1in]{./figs/imgs/29.png}
&\IncG[ width=1.1in]{./figs/gt/gt29.png}
&\IncG[ width=1.1in]{./figs/ma/29.png}
&\IncG[ width=1.1in]{./figs/netra/29.png}
&\IncG[ width=1.1in]{./figs/ours/out29.png}
\\
\IncG[ width=1.1in]{./figs/imgs/36.png}
&\IncG[ width=1.1in]{./figs/gt/gt36.png}
&\IncG[ width=1.1in]{./figs/ma/36.png}
&\IncG[ width=1.1in]{./figs/netra/36.png}
&\IncG[ width=1.1in]{./figs/ours/out36.png}
\\
\IncG[ width=1.1in]{./figs/imgs/40.png}
&\IncG[ width=1.1in]{./figs/gt/gt40.png}
&\IncG[ width=1.1in]{./figs/ma/40.png}
&\IncG[ width=1.1in]{./figs/netra/40.png}
&\IncG[ width=1.1in]{./figs/ours/out40.png}
\\
\IncG[ width=1.1in]{./figs/imgs/54.png}
&\IncG[ width=1.1in]{./figs/gt/gt54.png}
&\IncG[ width=1.1in]{./figs/ma/54.png}
&\IncG[ width=1.1in]{./figs/netra/54.png}
&\IncG[ width=1.1in]{./figs/ours/out54.png}
\\
\IncG[ width=1.1in]{./figs/imgs/67.png}
&\IncG[ width=1.1in]{./figs/gt/gt67.png}
&\IncG[ width=1.1in]{./figs/ma/67.png}
&\IncG[ width=1.1in]{./figs/netra/67.png}
&\IncG[ width=1.1in]{./figs/ours/out67.png}
\\
\footnotesize{(a) Input images} & \footnotesize{(b) Ground truth} & \footnotesize{(c) Fast-depth \cite{Wofk2019FastDepthFM}}& \footnotesize{(d) Joint-depth \cite{Nekrasov2019RealTimeJS}} &(e) \footnotesize{Ours} \\
\end{tabular}
\caption{Qualitative comparison of different methods for light-weight depth estimation. }
\label{fig1}
\end{figure*}

\subsection{Quantitative Evaluation}
For simplicity, we use $\mathcal{X}$, $\mathcal{U}$ and $\mathcal{U'}$ to denote the NYU-v2, unlabeled ScanNet, and labeled ScanNet, respectively.
The teacher trained on $\mathcal{X}$ and $\mathcal{X} \cup \mathcal{U}$ are denoted as $N_t(\mathcal{X})$ and $N_s(\mathcal{X} \cup \mathcal{U})$, respectively. Similarly, the student trained on  $\mathcal{X}$, $\mathcal{X} \cup \mathcal{U}$ and $\mathcal{X} \cup \mathcal{U}^{'}$ are denoted as
$N_s(\mathcal{X}), N_s(\mathcal{X} \cup \mathcal{U})$, $N_s(\mathcal{X} \cup \mathcal{U}^{'})$, respectively, throughout the paper.

\subsubsection{Performance without KD}
We first quantify the teacher and student network with pure supervised learning. We perform experiments on  $\mathcal{X}$ and the mixed dataset $\mathcal{X}\cup\mathcal{U'}$, respectively. The results are shown in Fig.~\ref{w_wo_kd} (a).
It is clear that the performance can be improved by training with more labeled samples, as the teacher is improved from o.845 to 0.874 and the student is improved from 0.802 to 0.825, respectively.
On the other hand, the performance gap between the teacher and student is significant, e.g. 0.845 vs 0.802 and 0.874 vs 0.825.

\subsubsection{Performance with KD}

We conduct several experiments to validate the proposed method. We first train the teacher net $N_t(\mathcal{X})$ and $N_t(\mathcal{X} \cup \mathcal{U}^{'})$ on the $\mathcal{X}$ and $\mathcal{X} \cup \mathcal{U}^{'}$, respectively. Then, we train the student in four different settings as follows:
\begin{enumerate}
    \item Given a trained teacher on the original set, we apply KD with the original training set, i.e $N_t(\mathcal{X}) \rightarrow N_s(\mathcal{X})$.
    \item Given a trained teacher on the original set, we apply KD with the auxiliary unlabeled set, i.e $N_t(\mathcal{X}) \rightarrow N_s(\mathcal{U})$.
    \item Given a trained teacher on the original set, we apply KD with the original training set and auxiliary unlabeled set, i.e $N_t(\mathcal{X}) \rightarrow N_s(\mathcal{X}\cup \mathcal{U})$.
    \item  Given a trained teacher on the original training set and auxiliary labeled set, we apply KD with the mixed labeled set, i.e $N_t(\mathcal{X}\cup \mathcal{U'}) \rightarrow N_s(\mathcal{X}\cup \mathcal{U'})$.
\end{enumerate}

The results are shown in Fig.~\ref{w_wo_kd} (b), we can see a large performance gap between the teacher and the student when applying standard KD as the setting of 1), the performance dropped from 0.845 to 0.808. An intriguing
observation is that the performance is even better when using only auxiliary unlabeled data, i.e. setting 2), compared to standard KD. As expected, the combination of them, i.e. setting 3), brings a significant performance boost.

As seen in Fig.~\ref{w_wo_kd} (a) $N_s(\mathcal{X}\cup\mathcal{U'})$,  when the auxiliary data is labeled, the light-weight network can be improved by supervised learning though, the improvement is limited due to low capacity of the small network. Its performance is inferior to the one trained with KD with auxiliary unlabeled data, i.e. result of $N_s(\mathcal{X}\cup\mathcal{U})$ in Fig.~\ref{w_wo_kd} (b).
On the other hand, we can learn a more accurate teacher, thus the light-weight network can also be further improved by applying KD, as seen in $N_s(\mathcal{X}\cup\mathcal{U'})$ of Fig.~\ref{w_wo_kd} (b).

\subsection{Comparison with Previous Methods.}
\paragraph{Comparison with Large Networks }
The comparison against methods built on large networks is shown in Table~\ref{nyu_depth_acc}.
A clear tendency of accuracy improvement is demonstrated by previous methods built on different backbone networks, i.e. from ResNet-50 to SeNet-154.
As seen that when using auxiliary unlabeled data,
our method is comparable to \cite{Zhang2019PatternAffinitivePA, hu2019revisiting} and even better than \cite{fu2018deep,laina2016deeper} with only 1.7 M parameters.

When comparing methods using extra labeled data, the best performance shown in Table~\ref{nyu_depth_acc} is \cite{Chen2020ImprovingMD} where the method is trained with six auxiliary datasets. The number of extra training data is 120K in total. Note that the auxiliary data used in \cite{Chen2020ImprovingMD} is carefully collected to handle the hard cases for depth estimation, such as spurious edges and reflecting surface. The extra data of our method is randomly selected from the ScanNet dataset \footnote{The auxiliary data used in \cite{Chen2020ImprovingMD} is not publicly available. We believe the performance of our method can be further improved when utilizing similar data for handling hard cases.}.

\paragraph{Comparison with Light-weight Networks }

\setlength{\tabcolsep}{1.0pt}
\begin{table}[!t]
\begin{center}
\caption{Quantitative comparison of light-weight approaches on the NYU-v2 dataset. The best and the second best results are highlighted in red and blue, respectively.}
\label{nyu_depth_acc_light}
\begin{tabular}{|l|l|c|c|c|c|c|c|c|c|}
\hline
Method & Backbone & Params (M) & GPU [ms]  & REL & $\delta_1$  \\
\hline
Fast-depth \cite{Wofk2019FastDepthFM} & MobileNet &3.9 &7 &- &0.775\\
Joint-depth \cite{Nekrasov2019RealTimeJS} &MobileNet-V2 &3.1 &21 &0.149 &0.790\\
Ours $N_s(\mathcal{X} \cup \mathcal{U})$     &MobileNet-V2 &1.7 &11  &{\color{blue}\textbf{0.131}} &{\color{blue}\textbf{0.837}}\\
Ours $N_s(\mathcal{X} \cup \mathcal{U}^{'})$   &MobileNet-V2 &1.7 &11  &{\color{red}\textbf{0.121}}  &{\color{red}\textbf{0.855}}\\
\hline
\end{tabular}
\end{center}
\end{table}

\setlength{\tabcolsep}{2.5pt}
\begin{figure*}[t]
\centering
\begin{tabular}
{p{0.05\textwidth}<{\centering}p{0.16\textwidth}<{\centering}p{0.16\textwidth}<{\centering}p{0.16\textwidth}<{\centering}p{0.16\textwidth}<{\centering}p{0.16\textwidth}<{\centering}}
\footnotesize{RGB}
&\IncG[ width=1.03in]{./figs/tum/rgb/1305031809.649447.png}
&\IncG[ width=1.03in]{./figs/tum/rgb/1305031455.491637.png}
&\IncG[ width=1.03in]{./figs/tum/rgb/1305031533.275405.png}
&\IncG[ width=1.03in]{./figs/tum/rgb/1305031245.461734.png}
&\IncG[ width=1.03in]{./figs/tum/rgb/1305031104.943262.png}
\\
\footnotesize{Ground truth}
&\IncG[ width=1.15in]{./figs/tum/gt/360_gt.png}
&\IncG[ width=1.15in]{./figs/tum/gt/desk_gt.png}
&\IncG[ width=1.15in]{./figs/tum/gt/desk2_gt.png}
&\IncG[ width=1.2in]{./figs/tum/gt/rpy_gt.png}
&\IncG[ width=1.15in]{./figs/tum/gt/xyz_gt.png}
\\
\vspace{-5mm}
\footnotesize{Ours}
&\IncG[ width=1.15in]{./figs/tum/ours/360_pred.png}
&\IncG[ width=1.15in]{./figs/tum/ours/desk_pred.png}
&\IncG[ width=1.15in]{./figs/tum/ours/desk2_pred.png}
&\IncG[ width=1.2in]{./figs/tum/ours/rpy_pred.png}
&\IncG[ width=1.15in]{./figs/tum/ours/xyz_pred.png}
\\
&\footnotesize{(a) fr1/360} & \footnotesize{(b) fr1/desk } & \footnotesize{(c) fr1/desk2} & \footnotesize{(d) fr1/rpy}  & \footnotesize{(e) fr1/xyz} \\
\end{tabular}
\caption{Qualitative comparison of predicted point clouds on the five sequences from TUM dataset. }
\label{fig_tum}
\end{figure*}

\begin{figure}[t]
\centering
\subfigure {\includegraphics[width=\linewidth]{figs/num.pdf}}
\caption{Results of KD that applies  $N_t^\mathcal{X} \rightarrow N_s(\mathcal{X \cup U})$ with different number of training samples from $\mathcal{U}$.}
\label{fig_num}
\vspace{-5mm}
\end{figure}

We compare the proposed method with two previous approaches for light-weight depth estimation, including a traditional encoder-decoder net, i.e. Fast-depth \cite{Wofk2019FastDepthFM}, joint learning of semantic and depth, i.e. Joint-depth \cite{Nekrasov2019RealTimeJS}.
The quantitative results are given in Table~\ref{nyu_depth_acc_light}. As seen that our method outperforms other two methods by a significant margin even with only about half parameters. The $\delta_1$ accuracy of our method $N_s(\mathcal{X} \cup \mathcal{U})$ is $83.7\%$ which outperforms Joint-depth and Fast-depth by $4.7\%$ and $6.2\%$, respectively. When the auxiliary data is labeled, the improvement is more significant, as the accuracy of $N_s(\mathcal{X} \cup \mathcal{U'})$ is 85.5\%, yielding $6.5\%$ and $8\%$ improvement.
The qualitative comparisons are shown in Fig.~\ref{fig1} where the estimated depth maps of our method are more accurate and have finer details.

We also compare the GPU time for inferring a depth map from an input image. We use a  computer  with  Intel(R)  Xeon(R)  CPU  E5-2690  v3  and a  GT1080Ti  GPU  card.
For the other two methods, we use the official implementations to calculate the computation time. The experimental results show our method infers a depth map using only 11 ms of GPU time, which is much faster than Joint-depth. On the other hand, Fast-depth achieves the smallest inference speed at the expense of  degradation of accuracy. It demonstrates the worst accuracy among the three methods.

\subsection{Effect of Varying the number of Auxiliary Data}
We conduct an ablation study to quantify the light-weight network while varying the number of auxiliary data. Specifically, we use the teacher trained on the original labeled set and apply KD with different number of auxiliary unlabeled samples taken from $\mathcal{U}$. In our experiments, we evaluate on the 11.6K, 22.0K, 40.2K, 67.6K,153.0K and 204.2K samples.

The results are shown in Fig.~\ref{fig_num}. Generally, the more auxiliary data we use, the better KD we can apply, and there will be no more improvement when the auxiliary samples reach a certain number.
\subsection{Cross-dataset Evaluation}
To measure the generalization performance of the light-weight model, we conduct a cross-dataset evaluation on the SUNRGBD \cite{Song2015SUNRA} and TUM \cite{sturm12iros} datasets.
We directly apply $N_s(\mathcal{X}\cup\mathcal{U})$ and  $N_s(\mathcal{X}\cup\mathcal{U'})$ without any fine-tuning to evaluate on these two datasets.
Note that the comparison between $N_s(\mathcal{X}\cup\mathcal{U'})$ and other methods is not fair as it employs auxiliary labeled data.
Nevertheless, we still show it here to provide an experimental support to the community that it is probably the most effective and reliable way to boost KD in a data-driven fashion.
Results for each dataset are shown as follows.

\subsubsection{Results on SUNRGBD}
The SUNRGBD dataset is commonly employed to test the generalization performance in previous works \cite{huynh2020guiding}. The results are shown in Table.~\ref{sunrgbd_evaluation} where the best and second best results are shown in red and blue, respectively.  Our method demonstrates the lowest RMSE and REL error. Joint-depth is best in $\delta_1$ and also is second best in REL.

\begin{table}[t]
\begin{center}
\caption{The results of different methods on the SUNRGBD dataset. The best and the second best results are highlighted in red and blue, respectively.}
\label{sunrgbd_evaluation}
\begin{tabular}
{|l|ccccccccccc|}
\hline
Method  & RMSE && && & REL & && && $\delta_1$\\ \hline
Fast-depth \cite{Wofk2019FastDepthFM}  &0.662  &&&&&0.376 &&& &&0.404 \\
Joint-depth \cite{Nekrasov2019RealTimeJS} &0.634&&  &&&{\color{blue} \textbf{0.338}}  &&&&&{\color{red} \textbf{0.454}} \\
Ours $N_s(\mathcal{X} \cup \mathcal{U})$ & {\color{blue} \textbf{0.577}} &&& & &{\color{blue} \textbf{0.338}}& & && &0.430\\
Ours $N_s(\mathcal{X} \cup \mathcal{U'})$  &{\color{red} \textbf{0.531}}&&  &&&{\color{red} \textbf{0.306}} &  &&&&{\color{blue} \textbf{0.446}}\\
\hline
\end{tabular}
\end{center}
\end{table}

\begin{table}[t]
\begin{center}
\caption{The $\delta_1$ accuracy of different methods on the five sequences from TUM dataset. The best and the second best results are highlighted in red and blue, respectively.}
\label{cross_evaluation}
\begin{tabular}
{|l|ccccc|}
\hline
Method &fr1/360 &fr1/desk &fr1/desk2 &fr1/rpy &fr1/xyz \\ \hline
Fast-depth \cite{Wofk2019FastDepthFM}   &0.548 &0.308 &0.358 &0.333 &0.287 \\
Joint-depth \cite{Nekrasov2019RealTimeJS} &0.512 &0.410 &0.441 &0.552 &{\color{blue} \textbf{0.583}}\\
Ours $N_s(\mathcal{X} \cup \mathcal{U})$  &{\color{blue} \textbf{0.615}}  &{\color{blue} \textbf{0.442}} &{\color{blue}\textbf{0.498}} &{\color{blue}\textbf{ 0.611}} &0.486 \\
Ours $N_s(\mathcal{X} \cup \mathcal{U'})$  &{\color{red} \textbf{0.854}} &{\color{red} \textbf{0.695}} &{\color{red} \textbf{0.772}}  &{\color{red} \textbf{0.679}} &{\color{red} \textbf{0.905}}\\
\hline
\end{tabular}
\end{center}
\end{table}

\subsubsection{Results on TUM}
We select five sequences including fr1/360, fr1/desk, fr1/desk2, fr1/rpy and fr1/xyz as \cite{Czarnowski2020DeepFactorsRP}.
The experimental results are shown in Table~\ref{cross_evaluation}. The depth accuracy is measured by $\delta_1$.
As seen that our method outperforms the other methods by a large margin, showing a satisfactory generalization performance.
The average accuracy (the mean of $\delta_1$) of Joint-depth and Fast-depth is 0.494 and 0.369, respectively, while our method $N_s(\mathcal{X} \cup \mathcal{U})$ and $N_s(\mathcal{X} \cup \mathcal{U'})$ achieve an average of 0.530 and 0.781, respectively.
The qualitative comparison between predicted point clouds and ground truths is shown in Fig.~\ref{fig_tum}.

\section{Conclusion}
In this paper,
we revisit monocular depth estimation from the aspects of inference  accuracy and computational efficiency. We point out the inherent difficulty is the trade-off between the accuracy and model's capacity. Our method first introduces a light-weight network with a goal of using much fewer parameters. Then, we quantitatively show that employing auxiliary training data with similar scene scales is effective to improve the light-weight network, through the study of two specific cases, i.e. auxiliary unlabeled/labeled data when applying knowledge distillation.
As a result, our method  achieves a comparable performance against methods built on large networks with about only 1\% parameters and outperforms other light-weight methods by a large margin.

\bibliographystyle{IEEEtranS}


\section{Experimental methods}
Single crystals of TTF-QBr$_3$I were grown by slow evaporation of cold ($\sim$5~$^{\circ}$C) mixed acetonitrile solution of QBr$_3$I and TTF and harvested as the minor products with the main products of (TTF)$_2$QBr$_3$I.
While TTF-QBr$_3$I crystallized as black rectangular plates, (TTF)$_2$QBr$_3$I crystallized as dark brown blocks.
The dielectric permittivity was measured by means of the typical two-terminal method along the b-axis parallel to the one-dimensional columns in the frequency range of 100~Hz-100~kHz.
The magnetization measurement for the $M$-$H$ curve was carried out in a pulse magnet with polycrystalline samples (total weight:$\sim$30~mg).
We performed the static magnetic susceptibility measurement at 1~T by using a magnetic property measurement system (Quantum Design) with polycrystalline samples weighing about 30~mg.
By using the pulse-echo and the phase comparison methods, the ultrasonic properties were measured. 32.0~MHz longitudinal ultrasonic waves along the $b$-axis are generated and detected by LiNbO$_3$ piezoelectric transducers glued on the (0~1~0) surfaces.

\section{Frequency and temperature dependence of $\epsilon$ }
\begin{figure}[h]
\begin{center}
\includegraphics[width=\hsize,clip]{FigS1.pdf}
\end{center}
\caption{Temperature dependence of $\epsilon_{\rm r}$ (left axis) and $\epsilon_{\rm i}$ (right axis) at each frequency in a linear plot (a) and in a semi-logarithmic plot (b).
}
\label{figS1}
\end{figure}
\clearpage

\section{Heat capacity}
\begin{figure}[b]
\begin{center}
\includegraphics[width=0.7\linewidth,clip]{FigS2.pdf}
\end{center}
\caption{
(a)The heat capacity data in the $C_p$/$T$ vs. $T^2$ plot at 0~T and 7~T.
The purple dotted curve denotes the lattice heat capacity mentioned in the text.
(b) The enlarged plot of (a) in the low temperature region.
(c) The electronic part of the heat capacity in a logarithmic plot of  $C_{\rm ele}$/$T$ vs. $T$.
The black dotted curve indicates a fit composed of the constant value $\gamma$ and the two-level Schottky anomaly at 7~T.
The black dashed line is a guide showing the relation  $C_{\rm ele}$/$T$$\sim$$T^{-1}$.
(d) The enlarged plot of (c) around $T_{\rm FSP}$.
}
\label{figS2}
\end{figure}
 Employing a physical property measurement system (Quantum Design), the heat capacity was measured with several single crystals whose total weight is about 5~mg.
We show the total heat capacity plotted as $C_p$/$T$ vs. $T^2$ in Figs.~\ref{figS2}(a),(b).
In the case of the typical organic conductors, the low-temperature heat capacity is described by the following formula:
 \begin{equation}
C_p/T=\gamma+\beta T^{2}+cR(T_{\rm E}/T)^{2}{\rm exp}(T_{\rm E}/T)/T[{\rm exp}(T_{\rm E}/T)-1]^{2},
\end{equation}
where $\gamma$ and $\beta$ denote the electronic and lattice heat capacity coefficients, respectively.
The third term, the Einstein mode, represents the optical phonon contribution, coming from the libration motion of the planar molecules\cite{34} like TTF and QBr$_3$I in the present case.
This component is typically negligible at low temperatures below 5~K$^2$.
For the present case, we need to take account of the additional term originating from the spin solitons, which shows the two-level Schottky anomaly $C_{\rm Sch}$ in magnetic fields.
A best fit for the data at 7~T below 5~K$^2$ is given by the parameters, 49.8~mJK$^{-2}$mol$^{-1}$ for $\gamma$, 10.2~mJK$^{-4}$mol$^{-1}$ for $\beta$ and 29.4~K for $T_{\rm E}$.
The Schottky heat capacity yields the density of the spin solitons of $\sim$9$\%$, consistent with the value observed in the magnetization measurement.
Note, however, that there are many parameters to fit the data and we assumed that the contribution of the transition is smaller than these components.
The density of the spin solitons may be overestimated because of the anomalous heat capacity of the transition.
The values of $\beta$ and TE are comparable with those of typical organic charge-transfer salts\cite{34,35}.
To obtain the electronic contributions, the lattice contributions (the dashed curve displayed in Figs.~\ref{figS2}(a),(b)), the second and third terms in the formula, are subtracted from the total heat capacity.
Figure~\ref{figS2}(c) shows the temperature profile of the obtained electronic heat capacity $C_{\rm ele}$ in the logarithmic plot of  $C_{\rm ele}$/$T$ vs. $T$.
At 0~T, the broad hump at 4-5~K is observed.
As identified in the main text, the thermodynamic anomaly reflects the ferroelectric SP transition.
Applying a magnetic field of 7~T, the anomaly becomes hard to distinguish due to the emergence of the large Schottky anomaly.
Nevertheless, the size of the anomaly does not show large field dependence, as shown in Figs.~\ref{figS2}(d).
Apart from the anomaly, we should notice the change of the low-energy excitations.
At low temperatures,  $C_{\rm ele}$/T varies as $\sim$$T^{-1}$ at 0~T while it is constant value $\gamma$ at 7~T.
The heat capacity is expected to exhibit gapped behavior\cite{36} because the ground state is the SP state.
However, in the present case the transition can be no longer treated in the mean-field due to the strong quantum fluctuation.
That's why the thermodynamic anomaly is strongly broadened or smeared.
Since TTF-QBr$_3$I is composed of the one-dimensional antiferromagnetic chains, the Nambu-Goldstone mode has the relation $C_p$$\sim$$T$ in the low temperature region, which is consistent with the constant behavior of  $C_{\rm ele}$/$T$ at 7~T.
Since the coefficient $\gamma$ is scaled with the inverse of $J$/$k_{\rm B}$ in such case, the obtained $\gamma$ give when $J$/$k_{\rm B}$$\sim$200~K, which is roughly comparable to the value determined in the susceptibility measurement.
In any case, the observed $\gamma$ at 7~T can be attributed to the low-energy excitation of the one-dimensional antiferromagnetism.
By contrast, the origin of the diverging heat capacity  $C_{\rm ele}$/$T$$\sim$$T^{-1}$ in zero field is still unclear because the Schottky heat capacity does not appear at zero field.
The disappearance of the component at 7~T indicates that it should be related to magnetic degrees of freedom, such as the spin soliton, the one-dimensional antiferromagnetism and nuclear spin.
Even though internal or remnant fields exist in the sample, the typical Schottky heat capacity exhibits  $C_{\rm ele}$/$T$$\sim$$T^{-3}$ dependence.
Thus, characteristic behavior originating from temperature-dependent relaxation time of the spin solitons and/or nuclear spins or critical behavior of magnetic degrees of freedom are considered as possible candidates.
We should again notice that the present fits have large ambiguity, and therefore, the evaluation of these components should not be accurate quantitatively.
Although decomposing the total heat capacity into each part is quite hard, the anomalous heat capacity must contain all the degrees of freedom including the contribution of the electric dipoles forming the ferroelectricity.

\section{Raw magnetic susceptibility}
\begin{figure}[hh]
\begin{center}
\includegraphics[width=0.7\linewidth,clip]{FigS3.pdf}
\end{center}
\caption{
Temperature dependence of magnetic susceptibility of TTF-QBr$_3$I in the plot (a) $\chi$ vs. $T$ and (b) $\chi$$T$ vs. $T$.
The red curve presents the raw data while the orange denotes the data plotted in Fig. 2(b).
The dotted lines in (b) are extrapolation of the data in the range of 7-10~K to 0~K.
}
\label{figS3}
\end{figure}
 As discussed in the main text and shown in Fig.~2(d), TTF-QBr$_3$I has the paramagnetic component of the spin solitons described by the S=1/2 Brillouin function.
While Fig.~2(e) presents the extracted magnetic susceptibility to emphasize the transition, Fig.~\ref{figS3} shows the total magnetic susceptibility (red) without the subtraction of the Curie-type paramagnetic component with the subtracted data.
We should notice that the number of the spin solitons should depend on temperature.
The temperature dependence of the subtracted susceptibility should be improper at higher temperatures far from 4.2~K.
However, the pulsed-field magnetization measurements of the present salt were difficult above 4.2~K, namely the liquid $^4$He temperature, because the small sample signal was undetectable precisely without a highly stable heat bath.
Above the FSP transition, the number of spin solitons is smaller, but not zero, because local domain formation exists as a dimerization fluctuation as evidenced by the elastic data and dielectric data.
This means that the accurate values of $\chi$ above $T_{\rm FSP}$ is between the total $\chi$ data and subtracted $\chi$ data.
Nevertheless, we can evaluate whether the present subtraction is reasonable or not.
By employing the typical manner for one-dimensional systems, the temperature dependence above the FSP transition should be described by the uniform one-dimensional antiferromagnetic Heisenberg model (1D-AFHM).
The absolute value of the subtracted $\chi$ at 4.2~K, which is almost constant below 10~K, is about 1.1~emu/mol.
This value gives the exchange interaction as $J$/$k_{\rm B}$$\sim$150 K\cite{37,38}, roughly consistent with the value 200~K estimated by the linear term of heat capacity even though the analyses of $C_p$ have some fitting ambiguity.
Therefore, the abrupt decrease of $\chi$ below 4.2 K should be reasonable although the accurate values above $T_{\rm FSP}$ are unclear.

\section{Ultrasonic echoes}
 In this study, we determine the elastic constant $C$ and the relative change of the ultrasonic attenuation $\Delta$$\alpha$ from the detected ultrasonic echoes.
Figure~\ref{figS4} shows the ultrasonic echoes at various temperatures. In each echo measurement, we give a 300~nsec pulse of 32~MHz to one of the attached transducers at 0~$\mu$sec as described by the shaded area.
The applied ultrasound passes through the sample and reach the opposite side of the sample, which has another attached transducer to detect the echoes as presented in the Fig.~\ref{figS4}.
Thus, we derived the absolute value of the sound velocity $V$ from the sample size and the time span to deliver the 0th echo as 1600$\pm$200~m/sec.
The elastic constant $C$ is given as $\rho$$V^2$, where $\rho$ denotes the density.
For $\Delta$$\alpha$, we evaluate the change of the echo amplitude from that at the lowest temperature.
In order to precisely determine $V$ and $\Delta$$\alpha$, we measured the change of the amplitude and phase of the 0th echo at 1~$\mu$sec presented by the dashed line because the other echoes may include additional components coming from other echoes passing through other routes.
\begin{figure}[hh]
\begin{center}
\includegraphics[width=0.4\linewidth,clip]{FigS4.pdf}
\end{center}
\caption{
In this measurement, we apply a pulsed ultrasound in the gray area.
The solid lines are guides to estimate the beginning position of the echoes.
The dashed line indicates the position that we analyze the ultrasonic properties of the 0th echo.
The right panel is the enlarged plot of the left panel at 1~$\mu$sec.
}
\label{figS4}
\end{figure}

\end{document}