\section{Introduction}
One of the first results in the development of neural networks is
the {\it Universal Approximation Theorem}
\cite{DBLP:journals/mcss/Cybenko89, Hornik:1991:ACM:109691.109700}. This classical result shows that any  continuous function on a compact set in ${\mathbb R}^n$ can be approximated by a multi-layer feed-forward network with only one hidden layer and non-poly\-nomial activation function (like the sigmoid function).
It is well-known that
this result has
two important drawbacks for
its practical use: firstly,
the width of the hidden layer grows exponentially and,
secondly,
the proofs developed in \cite{DBLP:journals/mcss/Cybenko89, Hornik:1991:ACM:109691.109700}
do not provide a practical algorithm for building such a network.
Bearing these results in mind, many researchers are paying attention to theoretical aspects of the current success of neural network architectures and searching
for bounds for the depth and width of such networks and the possibility
that they act
as universal approximators (see, e.g., \cite{DBLP:journals/corr/LiangS16,pmlr-v70-safran17a,DBLP:journals/corr/Telgarsky16} among many others).

Undoubtedly,
the use of many hidden layers is a big contribution to the success of deep learning architectures \cite{DBLP:conf/aaai/SunCWLL16}, but instead of exploring the power of depth, recently several studies have made interesting contributions
about
the power of width \cite{1710.11278,DBLP:conf/nips/LuPWH017,DBLP:journals/corr/abs-1803-00094}. To sum up,
these works
show that there exist continuous functions on compact sets that
cannot be approximated by
any
neural network if the width of the layers is not larger than a bound, regardless of the depth of the network. In \cite{GULIYEV2018296}, the authors constructively proved  that single-hidden-layer feed-forward networks with fixed weights are universal approximators for univariate functions, and they provided a step-by-step construction. However, as the authors claimed,
not all continuous multivariate functions can be approximated by such neural networks. Their case of study could be considered somehow a particular case of our approach for the 1-dimensional case.

Other interesting research line on the expressive power of neural networks follows an algebraic approach. In \cite{DBLP:conf/nips/DelalleauB11,MartensM14, poon2012sumproduct}, sum product networks are explored and tensor properties are studied in \cite{DBLP:conf/colt/CohenSS16,DBLP:conf/icml/CohenS16}. A different perspective was introduced in \cite{DBLP:conf/nips/KileelTB19} where "deep polynomial neural networks" (where the activation function is a polynomial exponentiation) are considered.

In this paper, we provide an effective method for finding the weights of a two-hidden-layer
feed-forward network which approximates
a given continuous
function between two triangulable
metric spaces.
Let us remark that the method is constructive, and it only depends on the desired level of approximation to the
given function.
Our approach is based on a classical result from algebraic topology. Roughly speaking, our result is
based on
two observations:
Firstly, triangulable
spaces
can be ``modelled''
using
simplicial complexes, and a continuous
function between two triangulable spaces can be approximated by a simplicial map
between simplicial complexes. Secondly,
a simplicial map between simplicial complexes can be ``modelled'' as a two-hidden-layer feed-forward network.
Let us remark that the classical result Universal Approximaton Theorem is valid for all compact set on ${\mathbb R}^n$ and our
results presented here are valid for triangulable
 spaces.
However, triangulable
 spaces
are
common in real-world problems.
Furthermore, we would like to highlight that an advantage of our approach from a theoretical and practical point of view is that it can be useful to solve real-world problems that can be modeled by triangulable spaces.

The paper is organized as follows: In Section \ref{sec:background}, the preliminary notions about multi-layer feed-forward networks and simplicial complexes are provided. Then, in Section \ref{sec:MLPandSM}, a concrete architecture
of such networks
that acts equivalently to a given simplicial map is given. In Sections \ref{sec:SimpAprox} and \ref{sec:UnivApprox}, we extend the
Simplicial Approximation Theorem and Universal Approximation Theorem, respectively. The complexity of the architecture is studied in Section 6.
A specific example is described in Section \ref{sec:example}. Finally, conclusions are given in Section \ref{sec:conclusions}.

\section{Background}\label{sec:background}

In this section, we recall some notions about artificial neural networks and simplicial complexes.

\subsection{Multi-layer
Feed-forward Networks }

Artificial neural networks are
inspired
by
biological networks of alive neurons in a brain. The number of different architectures, algorithms, and areas of application has recently grown in many directions. In general, a neural network can be formalized as a
function
${\cal N}_{\omega,\Theta}: {\mathbb R}^n \to {\mathbb R}^m$
that
depends on a set of weights
$\omega$ and a set of parameters $\Theta$
which involves the description of activation functions, layers, synapses between nodes (neurons), and whatever other consideration in its architecture. A good introduction to artificial neural networks can be found in \cite{haykin99a}.

In this paper, we focus on one of the simplest classes of artificial neural networks:  multi-layer feed\-for\-ward networks. They
consist of three or more  fully connected layers of nodes:
an input layer, an output layer, and one or more hidden layers. Each node in one layer has an
activation function and it
is connected with
every node in the following layer. The next definition formalizes this idea.

\begin{definition}[adapted from \cite{Hornik:1991:ACM:109691.109700}]\label{def:ANN}
A multi-layer feed-forward network defined on a real-valued $n$-dimensional space is a
function
${\cal N}: {\mathbb R}^n \to {\mathbb R}^m$  such that, for each $x \in {\mathbb R}^n$, ${\cal N}(x)$ is
the composition of
$k+1$
functions
$${\cal N}(x) =
f_{k+1}
\circ f_k\circ \dots
 \circ   f_1  (x)$$
where $k\in\mathbb{Z}$ is the number of hidden layers,
$k\geq 1$, and, for  $1\leq i\leq k+1$,
 $f_i:  {\mathbb R}^{d_{i-1}} \to {\mathbb R}^{d_i}$
 is defined as
  $$f_i(y) =\phi_i(W^{(i)}; y;b_i)$$ being
 $W^{(i)}$
 a real-valued $d_{i-1}\times d_i$ matrix,
 (that is, $W^{(i)}\in {\cal M}_{d_{i-1}\times d_i}$),
 $b_i\in \mathbb{R}^{d_i}$  the bias term, and $\phi_i$
 a bounded,  continuous, and non-constant function (called activation function).
 Notice that $d_0=n$, $d_{k+1}=m$ and $d_i\in {\mathbb Z}$, $1\leq i\leq k$, is called the width of the $i$-th  hidden layer.
\end{definition}

Next, we rewrite one of the most important theoretical results of multi-layer feed-forward networks adapted to our notation.

\begin{theorem}[Universal Approximation Theorem, \cite{Hornik:1991:ACM:109691.109700}]\label{th:unaprox}
Let $A$ be any compact subset of $\mathbb{R}^n$
and let $C(A)$ be the space of real-valued continuous functions on $A$.
Then, given any $\epsilon>0$ and any function $g\in C(A)$,
there exists a multi-layer feed-forward network ${\cal N}:\mathbb{R}^n\to\mathbb{R}$
approximating $g$,  that is,
$||g-{\cal N}||<\epsilon $.
\end{theorem}

As far as we know, the existing
proofs
of
this theorem are non-constructive. See \cite{hornik89a,Hornik:1991:ACM:109691.109700,DBLP:journals/mcss/Cybenko89}
where it is claimed
that
there exists a one-hidden-layer feed-forward network ${\cal N}:\mathbb{R}^n\to\mathbb{R}$, defined as ${\cal N}(x)=f_2\circ f_1 (x)$ with $f_1(y)= \phi_1 (W^{(1)};$ $y;b_1)$ and $f_2(y)=W^{(2)} y$,
approximating
$g$, that is,
$||g-{\cal N}||<\epsilon $,
but  no general algorithm to build ${\cal N}$ is given.
In this paper, we will provide a constructive approach to
Theorem \ref{th:unaprox} through a two-hidden-layer feed-forward network.
The most important restriction to our approach is that our result is valid for triangulable spaces
instead of compact ones but, as pointed out above,
triangulable spaces cover
most of the real-world problems.

\subsection{Simplicial Complexes}
In this subsection, we recall the main result used in this paper based on a
classical theorem from algebraic topology
known as the {\it Simplicial Approximation Theorem}.
For a further comprehension of the field, \cite{ayala2002elementos,boissonnat2018geometric,edelsbrunner2010,Hatcher,munkres} can be consulted.

Simplicial complexes are a data structure widely used to represent topological spaces. They are
versatile mathematical objects
that decompose a given
topological space into pieces
called simplices.

\begin{definition}
Let $v_0,v_1,\dots,v_i$ be  $i+1$ affinely independent points in $\mathbb{R}^n$
(being $i\leq n$).
An $i$-simplex, $\sigma = (v_0,v_1, \dots, v_i)$, is the convex set
$$\Big\{x\in \mathbb{R}^n \ \big | \ x= \sum_{j=0}^i \lambda_j v_j \ \mbox{ with } \ \lambda_j\ge 0
\mbox{ and } \sum_{j=0}^i \lambda_j=1
\Big\}. $$
The points $v_0,v_1,\dots,v_i$ are the vertices of $\sigma$.
The dimension of $\sigma$
is $i$.
We say that
$\sigma'$ is a face of $\sigma$
(denoted as
$\sigma'\preceq \sigma$)
if $\sigma'$ is an $i'$-simplex, with $i'\leq i$, and whose vertices are also vertices of $\sigma$.
\end{definition}

When several simplices are joint together, a more complex structure called simplicial complex is built. The following definition exhibits the way simplices can be glued together to obtain such a simplicial complex.

\begin{definition}
    A     simplicial complex $K$ is a
    collection of simplices  such that:
    \begin{enumerate}
        \item $\sigma\in K$ and
        $\sigma'\preceq \sigma$
        implies $\sigma'\in K$;
        \item
        $\sigma$, $\mu\in K$ implies $\sigma\cap \mu$ is either empty or a face of both.
    \end{enumerate}
If such collection is finite, then $K$ is a finite simplicial complex.
\end{definition}

  The  underlying space of $K$, denoted by $|K|$, is the union of the simplices of $K$ together with the topology inherited from the ambient Euclidean space where the simplices are placed.

  \begin{definition}
    A simplex $\sigma \in K$ is maximal if it is not a face of any other simplex in $K$.
The dimension of $K$, denoted by $\dim(K)$  is the largest  of the dimensions of its maximal simplices. The simplicial complex $K$ is pure if
    all the maximal simplices have  dimension equal to $\dim(K)$.
    \end{definition}

  A subcomplex $L$ of $K$ is a simplicial complex such that $L\subseteq K$.
  The skeleton of $K$ is a particular subcomplex of $K$. Let us remark that the 0-skeleton of $K$ is its vertex set.

  \begin{definition}
  The subcomplex of $K$ consisting of all the simplices of $K$ of dimension $j$ or less is called the $j$-skeleton of $K$ and it is denoted by $K^{(j)}$.
    \end{definition}

Next, we recall
the concept of the star of a simplex $\sigma$ in a simplicial complex $K$. Intuitively, it is the subcomplex of $K$ whose maximal simplices share $\sigma$ as a face.

\begin{definition}
    Let $K$ be a simplicial complex and $\sigma$ a simplex
    of $K$. The star of $\sigma$ in $K$, denoted by $\st (\sigma)$, is defined as:
   $$\st (\sigma) = \big\{\mu \in K \ \big| \ \exists \ \xi \in K \mbox{ such that } \sigma \preceq \xi \mbox{ and } \mu\preceq \xi \big\}.
   $$
\end{definition}

As said above, simplicial complexes are combinatorial data structures used to model topological spaces. A way to obtain a refined model from an existing one is to subdivide  it into small pieces so
that the result is  topologically equivalent to the
former.

\begin{definition}
Let $K$ and $K'$ be simplicial complexes. It is said that $K'$ is a subdivision of $K$ if:
\begin{enumerate}
    \item $|K|=|K'|$;
    \item     $\sigma' \in K'$ implies that there exists $\sigma\in K$ such that $\sigma'$
    is contained in
    $\sigma$.
\end{enumerate}
\end{definition}

The barycentric subdivision is a concrete example of subdivision of simplicial complexes.
Using that the barycenter of an
 $i$-simplex $\sigma= (v_0\dots v_i)$ is $$b(\sigma) = \sum_{j=0}^i\frac{1}{j+1}v_j,$$
the definition
of barycentric subdivision of a simplicial complex arises in a natural way.

\begin{definition}
Let $K$ be a simplicial complex. The
barycentric subdivision of the $0$-skeleton of $K$ is defined as the set of vertices of $K$, that is, $\Sd K^{(0)} = K^{(0)}$. Assuming
we have $\Sd K^{(i-1)}$, which denotes the barycentric subdivision of the $(i-1)$-skeleton of $K$, $\Sd K^{(i)}$ is built by adding the barycenter of every $i$-simplex as a new vertex and connecting it to the simplices that subdivide the boundary of such $i$-simplex.
The barycentric subdivision of $K$, denoted by $\Sd K$, is $\Sd K^{(u)}$ where $u$ is the dimension of $K$.
The iterated application of barycentric subdivisions is denoted by $\Sd^t K$ where $t$ is the number of iterations.
\end{definition}

\begin{figure}
    \centering
    \includegraphics[width =0.8 \linewidth]{figs/simplicial_complex.pdf}
    \caption{Example of a simplicial complex. $(d)$ is a $0$-simplex, $(o,p)$ is a $1$-simplex , $(a,b,c)$ is a $2$-simplex, and $(e,f,g,h)$ is a $3$-simplex. The $1$-simplex $(a,b)$ is a face of
    $(a,b,c)$. The maximal simplices of the simplicial complex are
    $(e,f,g,h)$, $(a,b,c)$, $(i,h,k)$, $(i,j,k)$, $(o,p)$, $(o,q)$, $(p,q)$, and $(d)$. The dimension of    the simplicial complex
    is 3.}
    \label{fig:simplicial_complex}
\end{figure}

\begin{figure}
    \centering
    \includegraphics[width =0.8 \linewidth]{figs/barycentric_subd.pdf}
    \caption{On the left, a simplicial complex with just one maximal simplex, the $2$-simplex $(a,b,c)$.
    On the right, its first barycentric subdivision.
    }
    \label{fig:barycentric_sub}
\end{figure}

Let us see now how the ``geometric size'' of the simplices of a simplicial complex can be measured.

\begin{definition}
    Let $K$ be a  finite simplicial complex. The diameter of a simplex $\sigma$ in $K$ is defined as
    $$\delta(\sigma) =    \max
    \big\{ ||x-y|| \ \mbox{ such that } \ x,y
    \mbox{ are vertices of }
    \sigma\big\}$$
        and  the  mesh of $K$ is defined as
    $$m(K) = \max
    \big\{\delta(\sigma) \ \mbox{ such that } \ \sigma \in K \big\}. $$
\end{definition}

\begin{theorem}[{\cite[p. 86]{munkres}}]\label{th:barycentricepsilon}
 Given a simplicial complex $K$ and  a real number $\epsilon>0$, there exists an integer  $t>0$ such that $m(\Sd^t K)\le \epsilon$.
\end{theorem}

Let us now think about maps between simplicial complexes. These maps can be considered as extensions of simpler maps defined
between the corresponding vertices of two given simplicial complexes. Interestingly, such maps can be considered as approximations of continuous functions defined on the underlying topological spaces that the simplicial complexes are modeling. Let us formalize these notions.

\begin{definition}
    Let $K$ and $L$ be two simplicial complexes. A vertex map is a
    map
     $\varphi:K^{(0)}\to L^{(0)}$
     with the property that
     for every simplex $\sigma$ in $K$ there exists a simplex $\mu$ in $L$ such that the vertices of $\sigma$ map to vertices of $\mu$.
\end{definition}

A vertex map $\varphi$ can be extended to a continuous function $\varphi_c:|K|\to |L|$ in the following way.

\begin{definition}
Let $K$ and $L$ be two  simplicial complexes and let  $\varphi:K^{(0)}\to L^{(0)}$ be a vertex map.
The simplicial map $\varphi_c$  induced by $\varphi$ is defined as follows.
Let $x\in |K|$. Then there exists an $i$-simplex  $\sigma=(v_0,\dots,v_i)$ in $K$ and numbers
    $\lambda_j\ge 0$
    such that
     $$\sum_{j=0}^i \lambda_j=1\,\mbox{ and }\,
 x= \sum_{j=0}^i
 \lambda_j v_j.$$ Then
$$ \varphi_c(x) = \sum_{j=0}^i  \lambda_j  \varphi(v_j).$$
\end{definition}

Any vertex map $\varphi$ induces a simplicial map $\varphi_c$, but if we want that
map to be a simplicial
approximation of a continuous function
between the underlying spaces of
two simplicial complexes $K$ and $L$, a restriction
on the star of each vertex of $K$ must
be added.

\begin{definition}
    Let $K$ and $L$ be simplicial complexes and  $g:|K|\to |L|$ a continuous
    function.
    A simplicial map $\varphi_c: |K|\to |L|$ induced by a vertex map $\varphi: K^{(0)} \to L^{(0)}$ is a simplicial approximation of $g$ if $$g(|\st (v)\ |)\subset |\st (\varphi(v))\ | $$
    for each vertex $v$ of $K$.
\end{definition}

The Simplicial Approximation Theorem
ensures the existence of simplicial maps that approximate continuous functions as close as we want.

\begin{theorem}
[Simplicial Approximation Theorem {\cite[p.  56]{edelsbrunner2010}}]
\label{th:simaprox}
If $g:|K|\to |L|$ is continuous then there is a sufficiently large integer $t>0$ such that $\varphi_c:|\Sd^t K|\to |L|$ is a simplicial approximation of $g$.
\end{theorem}

Theorems \ref{th:unaprox} and \ref{th:simaprox}
are key results in their respective fields. Our aim in this paper is to consider Theorem \ref{th:simaprox} as a pillar to obtain a
constructive approach to
Theorem \ref{th:unaprox}. Roughly speaking, the idea is to consider a simplicial approximation
of a continuous function
between two simplicial complexes.
Such a simplicial approach is characterized through
a vertex map which can be expressed as a neural network.
Besides,
the simplicial approximation can be chosen in such a way that it approximates the continuous function
between the underlying spaces of two simplicial complexes. Since the parameters of the neural network can be effectively obtained from the vertex map, this method provides a constructive way to find a neural network
that
approximates a continuous function between
the underlying spaces of two simplicial complexes.
Following that aim, let us formalize the relationship between topological spaces and simplicial complexes.

\begin{definition}
    A triangulation of a topological space $X$ consists of a simplicial complex $K$ and a homeomorphism $\tau:X\to |K|$.
    We say that the triangulation is finite if $K$ is finite. We say that $X$ is (finitely) triangulable if such (finite) triangulation exists.
\end{definition}

The spaces that can be triangulated by simplicial complexes (see \cite[Theorem A.7, p. 525]{Hatcher}) are
compact, locally contractible spaces
that can be embedded in $\mathbb{R}^n$ for some $n$. Let us remark that the Universal Approximation Theorem (Theorem \ref{th:unaprox}) is valid for any compact subset of ${\mathbb R}^n$, regardless
of whether
they are locally contractible or not. Not all compact sets in a metric space are locally contractible (see \cite[Chapter 17.7, p. 426]{geoghegan2010topological}).
Nevertheless, as far as we know, non-locally contractible spaces are very odd in ${\mathbb R}^n$ and this technical topological property has no practical application in real-world problems. Therefore, we can say that the results proved on triangulable spaces are true on a large amount of current neural network problems. Besides,
we will restrict ourselves to finite pure
triangulations.
The other cases could be obtained using homotopies.
For example, given a sphere and a circumference that intersect transversely, the sphere can be triangulated  using $2$-simplices, and the circumference can be covered by $2$-simplices obtaining a finite pure triangulation of a space homotopic to the initial one (see Figure \ref{fig:triangulation_maximal_simplices}).

\begin{figure}
    \centering
    \includegraphics[width = 0.4 \linewidth]{figs/circ_sphere.PNG}
    \includegraphics[width = 0.4 \linewidth]{figs/triangulation_circ_sphere.PNG}
    \includegraphics[width = 0.4 \linewidth]{figs/triangulation.PNG}
    \caption{From left to right and from top to bottom: (1) A sphere and a loop that intersect; (2) the sphere and the loop with a possible triangulation (the loop is covered by $2$-simplices); (3) the triangulation of the loop and the sphere.}
    \label{fig:triangulation_maximal_simplices}
\end{figure}

Next, we  extend the definition of a mesh of a simplicial complex in the following way.

\begin{definition}
    Let $X$ be a  triangulable
    metric
    space and $(K,\tau)$ a finite triangulation of $X$. The mesh of $X$ induced by $(K,\tau)$ is defined as
    $$\tilde{m}_{(K,\tau)}(X) = \max \big\{\tilde{\delta}(\sigma) \ | \ \sigma \in K \big\} $$
    where
  $$\tilde{\delta}(\sigma) = \max \big\{   d_X(x,y)

  \ | \ x = \tau^{-1}(a), \ y = \tau^{-1}(b);
  a,b\in \sigma
  \big\}
  $$
    is the extended diameter of a simplex.
\end{definition}

\section{Multi-layer Feed-forward Networks and Simplicial Maps}\label{sec:MLPandSM}

In this section, we will show that
simplicial maps can be modelled via multi-layer feed-forward networks in a straightforward way.

In the following theorem, we will compute a two-hidden-layer feed-forward network to
model a simplicial map $\varphi_c: |K|\to |L|$ where
$K$ and $L$ are
finite pure simplicial complexes.
This is not an important constraint in our case, since our final aim is to design a multi-layer feed-forward network that approximates a continuous function between finitely triangulable spaces.

\begin{theorem}\label{th:simplicialmap_nn}
Let us consider
 a simplicial map $\varphi_c:|K|\to |L|$
 between the underlying space of two finite pure simplicial complexes $K$ and $L$.
 Then a two-hidden-layer feed-forward network ${\cal N}_{\varphi}$
  such that $\varphi_c(x) = {\cal N}_\varphi(x) $ for all $x\in |K|$ can be explicitly defined.
\end{theorem}

\begin{pf}
Let us assume that $\dim(K)=n$ and $\dim(L)=m$.
Besides,
let $\{\sigma_1,\dots \sigma_k\}$ be the maximal $n$-simplices of $K$,  where $\sigma_i = \big[v_0^i,\dots,v_n^i\big]$ for all $i$; and let $\{\mu_1,\dots,\mu_{\ell} \}$ be the maximal $m$-simplices of $L$,
 where $\mu_j = \big[u_0^j$, $\dots$, $u_m^j\big]$ for all $j$.
Let us consider a multi-layer feed-forward network ${\cal N}_\varphi$ with the following architecture:
\\
(1) An input layer composed by $d_0=n$ neurons;
\\
(2) a first hidden layer composed by $d_1=k\cdot (n+1)$ neurons;
\\
(3) a second hidden layer composed by $d_2=\ell \cdot (m+1)$ neurons; and
\\
(4) an output layer with $d_3=m$ neurons.
\\
Then, let ${\cal N}_\varphi = f_3\circ f_2\circ f_1$ being $f_i (y)= \phi_i (W^{(i)};y;b_i)$, $i=1,2,3$.
Now, the idea  is to encode
the simplicial complexes involved in the mapping in the hidden layers of the multi-layer feed-forward network. Firstly, a point $x$ in ${\mathbb R}^n$ is transformed into a $k \cdot (n+1)$ vector. This vector can be seen as the juxtaposition of $k$ vectors of dimension $n+1$, one for each of the $k$ simplices in $K$. Each vector of dimension $n+1$ represents the barycentric coordinates of $x$ with respect to the corresponding simplex.
The matrix $W^{(1)}\in {\cal M}_{k(n+1)\times n }$ and the bias term $b_1$ can be obtained from the barycentric coordinates relations
as follows.
Firstly,
$$
W^{(1)} =
\begin{pmatrix}
W^{(1)}_1 \\ \vdots \\ W^{(1)}_k
\end{pmatrix}
$$
where $W^{(1)}_i\in {\cal M}_{(n+1)\times n}$
is:
$$
\begin{pmatrix}
v_0^i & \cdots & v_n^i \\
1 & \cdots & 1
\end{pmatrix}^{-1}
=
\begin{pmatrix}
W^{(1)}_i & \big| &
B_i
\end{pmatrix}
$$
being $\{v_0^i, \dots, v_n\}$ the set of vertices of the maximal simplex $\sigma_i$ of $K$, and second, $b_1 \in \mathbb{R}^{n+1}$ is:
$$
b_1 = \begin{pmatrix}
B_1 \\ \vdots \\ B_k
\end{pmatrix}.
$$
The function $\phi_1$ is then defined as:
 $$\phi_1 (W^{(1)};y;b_1)=W^{(1)}y+b_1.$$

The matrix of weights $W^{(2)}\in {\cal M}_{\ell(m+1)\times k(n+1)}$ between the first and the second hidden layer of ${\cal N}_\varphi$
encodes the vertex map
$\varphi$.
The first hidden layer is composed by $k \cdot (n+1)$ neurons that correspond to the vertices of the maximal simplices of $K$.
The second hidden layer is composed by  $\ell \cdot (m+1)$ neurons that correspond to the vertices of the maximal simplices of $L$.
The matrix $W^{(2)}$ is composed by values zeros and ones. On the one hand, an element of $W^{(2)}$ has value 1 if
the corresponding vertices
in
$K$ and $L$ are related by
the vertex map $\varphi$, and, on the other hand, it has  value 0 if
they are not related by $\varphi$. Then,
$$
W^{(2)} = \big(W^{(2)}_{s_1,s_2}
\big)
$$
where$$    W^{(2)}_{s_1,s_2} = \left\{ \begin{array}{l} 1 \mbox{ if } \varphi (v^i_t)=u^j_r ,
\\ 0   \mbox{ otherwise;} \end{array}\right.$$
being $s_1 = j(r+1)$ and $s_2 = i(t+1)$ for $i=1,\dots, k$; $j = 1,\dots, \ell$; $t =0,\dots,n $; and  $r = 0,\dots, m$.
The bias term $b_2$ is the null vector.
Then, the function $\phi_2$ is defined as:
$$\phi_2 (W^{(2)};y;b_2)=W^{(2)}y.$$
The output of the second hidden layer can be seen as the juxtaposition of $\ell$ vectors of dimension $m+1$, one vector for each simplex in the simplicial complex $L$. Each of these vectors represents the barycentric coordinates of $\varphi_c(x)$ with respect to the corresponding simplex in $L$.
In the next step, only vectors whose all
coordinates are greater than or equal to zero are considered. This condition encodes the simplices of $L$ to which $\varphi_c(x)$ belongs.
Then, $\phi_3(W^{(3)};y;b_3)$ maps
the barycentric coordinates of $\varphi_c(x)$ with respect to each maximal simplex of $L$ to which $\varphi_c(x)$ belongs, to the Cartesian coordinates of $\varphi_c(x)$. Specifically,
$$
W^{(3)} =
\begin{pmatrix}
W^{(3)}_1& \cdots & W^{(3)}_\ell
\end{pmatrix}\in {\cal M}_{m\times \ell(m+1) },
$$
being $W^{(3)}_j=
\begin{pmatrix}
u_0^j & \cdots & u_m^j
\end{pmatrix}$; and
 $b_3$ is the null vector.
Finally,
$\phi_3$ is defined as:
$$\phi_3 (W^{(3)};y;b_3)=\frac{\sum_{j=1}^{\ell}z^j\psi(y^j)}{\sum_{j=1}^{\ell}\psi{(y^j)}}$$
for
$y =
\begin{pmatrix}
y^1 \\
\vdots \\
y^{\ell}
\end{pmatrix}\in {\cal M}^{\ell\cdot (m+1)}
$,
with
$z^j = W^{(3)}_j y^j$ and  $\psi(y^j) = 1$ if all the coordinates of $y^j$ are greater than or equal to $0$ and $\psi(y^j)=0$ otherwise.

The
particular choice
of $\phi_3$ and $\psi$ is motivated by the use of the barycentric coordinates.
Let us observe that the barycentric coordinates vary with respect to the maximal simplex considered to compute them and that the barycentric coordinates are computed from the  coordinates of the vertices of the maximal simplex considered. Besides, maximal simplices can share common vertices. Then, the map
$\psi$ is used to determine if a given input is located in a specific simplex. The map $\phi_3$ is used to normalize the result in case that a point belongs to more than one simplex.
\hfill \qed
\end{pf}

Summing up, Theorem \ref{th:simplicialmap_nn} establishes that a two-hidden-layer feed-forward network can act equivalently to a simplicial map.
The  architecture and the specific computation of the parameters of the network are  provided in the proof of the theorem.

\section{Simplicial Approximation Theorem Extension}\label{sec:SimpAprox}

In this section,
we provide an extension of the Simplicial Approximation Theorem together with an explicit algorithm to compute a simplicial approximation ``as close as desired'' to a given continuous function $g:|K|\to|L|$  between the underlying spaces of
two simplicial complexes $K$ and $L$.
The first observation is that the Simplicial Approximation Theorem (Theorem \ref{th:simaprox}) refers to any continuous function.
Nevertheless, continuity is a property of functions between  topological spaces, not necessarily metric spaces. The next result introduces the concept of closeness between simplicial approximations and continuous functions.

\begin{proposition}\label{th:simaprox1}
 Given $\epsilon>0$ and a continuous function $g:|K|\to |L|$ between the underlying spaces of two simplicial complexes $K$ and $L$, there exist $t_1,t_2>0$ such that $\varphi_c: |\Sd^{t_1}K|\to |\Sd^{t_2}L|$ is a simplicial approximation of $g$ and $||g-\varphi_c||\le \epsilon$.
\end{proposition}

\begin{pf}
By Theorem \ref{th:barycentricepsilon}, there exists $t_2$ such that $m(\Sd^{t_2}L)\le \epsilon$. Then, by Theorem \ref{th:simaprox}, there exists $t_1$ such that $\varphi_c:|\Sd^{t_1}K|$ $\to$  $|\Sd^{t_2}L|$ is a simplicial approximation of $g$:

    $$
\begin{tikzcd}
\vert K\vert \arrow[r, "g"]       & \vert L \vert                     \\
K \arrow[u, hook] \arrow[d, "\Sd"] & L \arrow[u, hook] \arrow[d, "\Sd"] \\
\Sd^{t_1}K  \arrow[d,hook] & \Sd^{t_2}L \arrow[d,hook] \\                      \vert \Sd^{t_1}K \vert \arrow[r, "\varphi_c"]               & \vert \Sd^{t_2}L \vert
\end{tikzcd}
$$

\noindent Besides, $||g-\varphi_c||\le \epsilon$ because $m(\Sd^{t_2}L)\le \epsilon$.
\hfill \qed
\end{pf}

\begin{algorithm2e}
  \KwIn {
  A continuous function $g:|K|\to |L|$  between the underlying spaces of two simplicial complexes $K$ and $L$, and an integer $t$ where $\Sd^t k$ satisfies the {\it star condition}:
  for each $v\in
  \Sd^t K^{(0)}$ there exists $w\in L^{(0)}$ such that
   $g(|st(v)|)\subseteq |st(w)|$.
 }
  \KwOut{A vertex map $\varphi$ that induces simplicial approximation $\varphi_c$ of $g$.}
  \BlankLine
  \ForEach{
  vertex $v\in \Sd^t K^{(0)}$
  }
  {
  Choose $w\in L^{(0)}$ such that $g(|st(v)|)\subseteq |st(w)|$ and define
  $\varphi(v):=w$.
   }
 \caption{Computing a vertex map that induces a simplicial approximation}
  %
  \label{algo-aprox}
\end{algorithm2e}

Algorithm \ref{algo-aprox} is inspired in the proof of the Simplicial Approximation Theorem given in \cite[p. 56]{edelsbrunner2010} and computes a
vertex map $\varphi: \Sd^t K^{(0)}\to
L^{(0)}
$ from which we can define
the
simplicial approximation
 $\varphi_c:$ $|\Sd^t K|$ $\to$ $|L|$
of a continuous function $g:|K|\to |L| $.

\begin{theorem}
 Given a continuous function $g:|K|\to |L|$ and $\epsilon>0$, a two-hidden-layer feed-forward network ${\cal N}$ such that $||g-{\cal N}||\leq \epsilon$ can be explicitly defined.
\end{theorem}

\begin{pf}
By Proposition \ref{th:simaprox1}, there exists a simplicial approximation $\varphi_c$ of $g$ such that $||g-\varphi_c||\le \epsilon$, that can be computed using Algorithm~\ref{algo-aprox}. Then,
by Theorem \ref{th:simplicialmap_nn} there exists ${\cal N}_\varphi$ such that $\varphi_c={\cal N}_\varphi$.
\hfill \qed
\end{pf}

\section{Universal Approximation Theorem Extension}\label{sec:UnivApprox}

In the previous sections, we have proved that a continuous function between triangulable
spaces can be approximated by using the Simplicial Approximation Theorem. In this section, using the extension of the Simplicial Approximation Theorem given in Proposition \ref{th:simapproxextension}, we provide a constructive version of the Universal Approximation Theorem that approximates any continuous function (under some specific conditions) arbitrarily close.

\begin{proposition}\label{prop:measurebarycentric}
 Let
 $(K,\tau)$ be a finite triangulation of a metric space $X$.
  %
 For all $\epsilon>0$ there exists $t>0$ and $\gamma>0$ such that if $m(\Sd^t K)\le \gamma$ then $\tilde{m}_{(\Sd^t K,\tau)}(X)\le \epsilon$.
\end{proposition}
\begin{pf}
Let us consider
$a,b_0\in |K|$.
Then, $a\in\sigma_0$
for some maximal simplex $\sigma_0\in K$.
If $b_0$ belongs to $\sigma_0$ then $||a-b_0||\le \delta(\sigma_0)$ and
$d_X(x,y_0)\le \tilde{\delta}(\sigma_0)$, being
$x=\tau^{-1}(a)$ and $y_0=\tau^{-1}(b_0)$.
Otherwise, we repeat the reasoning with $\Sd K$.
Now,
$a$ and $b_0$
can belong to the same simplex  in $\Sd K$ or not. If they belong to  the same simplex in $\Sd K$,
write
$b_1=b_0$.
If not, take a new point
$b_1$
such that
$a,b_1
\in\sigma_1$ and $\sigma_1\in \Sd \sigma_0$.
Therefore, $
||a-b_1||
\le \delta(\sigma_1)\le \delta (\sigma_0)$. Besides,  $
d_X(x,y_1)
\le \tilde{\delta}(\sigma_1)\le \tilde{\delta}(\sigma_0)$
being $x=\tau^{-1}(a)$ and $y_1=\tau^{-1}(b_1)$
This process can be iterated:  $
||a-b_t||
\le \delta(\sigma_t) \le \dots \le \delta(\sigma_1)\le \delta (\sigma_0)$ and $
d_X(x,y_t)\le \tilde{\delta}(\sigma_t)\le \dots \le \tilde{\delta}(\sigma_1)\le \tilde{\delta}(\sigma_0)$,
being $x=\tau^{-1}(a)$ and $y_t=\tau^{-1}(b_t)$. By this, we have defined a sequence $\{b_i\}_{i=0}^t$ that converges to $a$.
Therefore, given $\epsilon>0$, there exists $t$ such that $d_X(x,y_t)\le \epsilon$, being
$x=\tau^{-1}(a)$ and $y_t=\tau^{-1}(b_t)$.
Let us suppose, without loss of generality, that $\tilde{\delta}(\sigma_t)=\tilde{m}_{(\Sd^t K,\tau)}(X)$. Then, we can consider $\gamma = m(\Sd^t K)$.
\qed
\end{pf}

\begin{corollary}\label{cor:barycentricmeasure}
Given $\epsilon>0$ and a finite triangulation $(K,\tau)$ of a metric space $X$,
there exists $t$ such that $$\tilde{m}_{(\Sd^t K,\tau)}(X)\le \epsilon.$$
\end{corollary}
\begin{pf}
By Theorem \ref{th:barycentricepsilon} there exists $t'$ such that $m(\Sd^{t'}K)$ $\le \gamma$. Then, by Proposition \ref{prop:measurebarycentric}, there exists $t$ such that $\tilde{m}_{(\Sd^t K,\tau)}(X)$ $\le \epsilon$.
\hfill \qed
\end{pf}

Given two continuous function $g_1$ and $g_2$ between two metric spaces $X$ and $Y$, we denote $\sup\{d_Y(g_1(x),g_2(x))\ | \ x\in X\}$ also by $||g_1-g_2||$.
Now, given a continuous function $g$ between two finitely triangulable metric spaces $X$ and $Y$, there exists
a  simplicial approximation $\varphi_c$
``arbitrarily close'' to $g$.

\begin{proposition}\label{th:simapproxextension}
Let $X$ and $Y$ be two finitely triangulable
metric
spaces, $g:X\to Y$ a continuous
function,
and $\epsilon>0$. Then, there exist two finite triangulations $(K,\tau_K)$ and $(L,\tau_L)$ of $X$ and $Y$, respectively,
and a simplicial approximation $\varphi_c:$ $|\Sd^{t_1}K|\to |\Sd^{t_2}L|$
such that
$||g-\tilde{\varphi_c}||\le\epsilon$
being $\tilde{\varphi_c}=\tau^{-1}_L\circ \varphi_c\circ \tau_K$.
\end{proposition}

\begin{pf}
 First, by Corollary \ref{cor:barycentricmeasure}, there exists $t_2$ such that \newline
 $\tilde{m}_{(\Sd^{t_2}L,\tau_L)}(Y)\le \epsilon$. Next, by Theorem \ref{th:simaprox}, there exists $t_1>0$ and a vertex map $\varphi: (\Sd^{t_1}K)^{(0)}\to (\Sd^{t_2}L)^{(0)}$ such that $\varphi_c:|\Sd^{t_1}K|\to |\Sd^{t_2}L|$ is a simplicial approximation of
 $\tau_{L}\circ g\circ \tau^{-1}_{K}$.
 Take
 into account that $|\Sd^{t_1}K|= |K|$ and $|\Sd^{t_2}L|= |L|$.
Finally,
since
$\tilde{m}_{(L,\tau_L)}(Y)\le \epsilon$ then
$||g-\tilde{\varphi_c}||\le\epsilon $.
Below, a diagram that schematizes the proof:
$$
\begin{tikzcd}
X \arrow[d, "\tau_{K}"] \arrow[r, "g"]   & Y \arrow[d, "\tau_{L}"]            \\
\vert K \vert \arrow[r, "\varphi_c"] & \vert L \vert                      \\
K \arrow[u, hook] \arrow[d, "\Sd"]      & L  \arrow[u, hook] \arrow[d, "\Sd"] \\
\Sd^{t_1}K   & \Sd^{t_2}L    \\
(\Sd^{t_1}K)^{(0)} \arrow[u, hook] \arrow[r, "\varphi"]      & (\Sd^{t_2}L)^{(0)}  \arrow[u, hook]
\end{tikzcd}
$$
\hfill \qed
\end{pf}

Finally, we reach the main result of this section: Given a continuous function
$g$ between two finitely triangulable spaces $X$ and $Y$, we can obtain two finite simplicial complexes $K$ and $L$ associated to them, and a
multi-layer feed-forward network
between the underlying spaces of $K$ and $L$  which ``approximates'' $g$.

\begin{theorem}\label{th:hidden-layer-extension}
 Given a continuous function $g:X\to Y$ between two finitely triangulable metric spaces $X$ and $Y$ and finite triangulations $(K,\tau_K)$ and $(L,\tau_L)$ of, respectively, $X$ and $Y$,
a two-hidden-layer feed-forward network $\mathcal{N}$ such that $||g-\tilde{\cal N}||\le \epsilon$, being $\tilde{\cal N}=\tau_L^{-1}\circ {\cal N} \circ \tau_K$, can be explicitly defined.
\end{theorem}
\begin{pf}
By Proposition \ref{th:simapproxextension}, there exists a simplicial approximation $\varphi_c$
such that
$||g-\tilde{\varphi_c}||\leq \epsilon$.
Finally, by Theorem \ref{th:simplicialmap_nn}, there exists $\mathcal{N}$ such that $\mathcal{N} = \varphi_c$ in all the domain. \hfill \qed
\end{pf}

\section{Complexity of the Architecture of the Network}

In previous sections, we have provided a constructive approach to build neural networks to approximate continuous functions as close as desired. Now, let us study
how the ``complexity'' of the
architecture of the neural network
increases in terms of the number of neurons in each hidden layer.

\begin{definition}
    The complexity of a
    neural network is the maximum of the widths of its hidden layers.
\end{definition}

First, let us observe that we can infer an upper bound for the amount of barycentric subdivisions of a simplicial complex needed to reach an specific
mesh.

\begin{proposition}
Let us consider a finite pure simplicial complex $K$. Let $\dim(K)=n$ and let $0<\varepsilon<m(K)$.
If
$$t \ge \frac{\log(m(K))-\log(\varepsilon)}{\log(n+1)-\log(n)} $$
then $m(\Sd^t(K))\leq  \varepsilon$.
\end{proposition}

\begin{pf}
Let us observe that $m(\Sd^t(K))\le m(K)\cdot \left(\frac{n}{n+1}\right)^t$. Then,
$$m(\Sd^t(K))\leq\varepsilon
\Leftarrow
m(K)\cdot \left(\frac{n}{n+1}\right)^t\leq\varepsilon
\Leftrightarrow
t\geq
\frac{\log \left(\frac{\varepsilon}{m(K)}\right)}{\log \left(\frac{n}{n+1}\right)}.
$$
\qed
\end{pf}

Let us recall that, given two finite pure simplicial complexes
$K$ and $L$ with, respectively, $k$ and $\ell$ maximal simplices, being, respectively, $\dim(K)=n$ and $\dim(L)=m$,
the width of  the first  and  the second hidden layer of the neural network $\mathcal{N}$ described in Theorem \ref{th:simplicialmap_nn}
is, respectively,
$k\cdot(n+1)$ and
$\ell\cdot(m+1)$.
Let us describe how the complexity of $\mathcal{N}$ increases with the iterated applications of the barycentric subdivisions. Let us consider, without loss of generality, that we apply one
barycentric subdivision to $K$.
Then, the width of the first hidden layer increases from $k\cdot(n+1)$ to $k \cdot(n+1)!\cdot(n+1)$.

\begin{rmk}
 Let us consider a simplicial approximation $\psi_c:|\Sd^{t_1} K|\to
|\Sd^{t_2} L|$ of a continuous function,  being $K$ and $L$ two finite pure simplicial complexes of dimension $n$ and $m$, and with $k$ and $\ell$ maximal simplices, respectively.
The complexity of the two-hidden-layer feed-forward network ${\cal N}_{\psi} $
is
$$C(t_1,t_2)=\max\Big\{k\big((n+1)!\big)^{t_1}(n+1),\;\ell\big((m+1)!\big)^{t_2}(m+1)\Big\}.$$

\end{rmk}

We can relate the modulus of continuity of the
input function $g$ and the modulus of continuity of the simplicial approximation
$\varphi_c$ between triangulations of the input spaces. Let us recall the definition of modulus of continuity of a function.

\begin{definition}
The modulus of continuity of a continuous function $g:X\to Y$
between two metric spaces $X$ and $Y$
is given by:
$$\rho(\delta,g)=
 \sup\big\{d_Y(g(x),g(y))\ | \ d_X(x,y)\le \delta \big\}.
$$
In particular we have that:
$$d_X(x,y)\le \delta \Rightarrow d_Y(g(x),g(y))\le \rho(\delta,g).$$
\end{definition}

Now, let us study the complexity of the resulting network in terms of the mesh
of the triangulation  and on the modulus of continuity of the considered function.

\begin{theorem}
Let $g: X\to Y$ be a continuous function between two triangulable metric spaces with modulus of continuity $\rho(\delta, g)\geq 0$. Let us suppose that there exist finite triangulations $(K,\tau_K)$ and $(L,\tau_L)$ of $X$ and $Y$, respectively. Then, there exists a two-hidden-layer feed-forward network ${\cal N}$

such that
$\rho(\delta,\tilde{\cal N})\leq 2\rho(\delta,g)$ and $||g-\tilde{\cal N}||\leq \frac{\rho(\delta,g)}{2}$
being $\tilde{\cal N}=\tau^{-1}_L\circ {\cal N}\circ \tau_K$.

\end{theorem}

\begin{pf}
Let $\epsilon=\frac{\rho(\delta,g)}{2}$.
By Theorem~\ref{th:hidden-layer-extension}, there exists a two-hidden-layer feed-forward network ${\cal N}$ such that
$||g-\tilde{\cal N}||\leq \epsilon$.
Consider $x,y\in X$ such that $d_X(x,y)\leq \delta$.
then,
$$\begin{array}{l}
d_Y(\tilde{\cal N}(x),\tilde{\cal N}(y))\\
\leq  d_Y(g(x),\tilde{\cal N}(x))+
d_Y(g(x),g(y))+d_Y(g(y),\tilde{\cal N}(y))\\
\leq  2||g-\tilde{\cal N}||+\rho(\delta,g)
\leq 2\rho(\delta,g).
\end{array}
$$
\hfill \qed
\end{pf}

\section{Example}\label{sec:example}

In this section,
we show a concrete example
of a multi-layer feed-forward network approximating a continuous function between two triangulable spaces.The following diagram illustrates the example:
$$
\begin{tikzcd}
B^3  \arrow[r, "g"] \arrow[d, "\tau_{K}"]  & B^2   \arrow[d, "\tau_{L}"]          \\
\includegraphics[scale = 0.1]{figs/tetrahedron2.png}  \arrow[r, "\varphi_c"] & \includegraphics[scale = 0.1]{figs/triangle.png}   \\
K \arrow[u, hook]  & L \arrow[u, hook] \\
K^{(0)} \arrow[u, hook] \arrow[r,"\varphi"]  & L^{(0)} \arrow[u, hook]
\end{tikzcd}
$$

Let us consider the $n$-dimensional ball $B^n=\big\{x\in \mathbb{R}^n$ $| \ 1\geq ||x||\big\}$.
Then, a triangulation of $B^3$ is the simplicial complex $K$ whose maximal simplex is a a tetrahedron with set of vertices $K^{(0)}=\big\{(0,0,0)$, $(1,0,0)$, $(0,1,0)$, $(0,0,1)\big\}$
and a homeomorphism $\tau_K:B^3\rightarrow |K|$
whose inverse
is defined
for any point
$P\in |K|$
as follows:
$$P'=\tau^{-1}_K(P)=\lambda(P)\cdot \Big(P-\big(\frac{1}{4},\frac{1}{4},\frac{1}{4}\big)\Big)$$
where $\lambda(P)$
is
the quotient $|\overline{OB}|/|\overline{OA}|$
being point $O=(0,0,0)$, point $A$ the intersection
of the boundary of the tetrahedron with the
line that goes through $O$ and $P$, and point $B$ the intersection of such line with the sphere
$S^2=\big\{x\in \mathbb{R}^3$ $| \ 1=||x||\big\}$.
See Figure~\ref{fig:lambda} for a similar homeomorphism  in the 2-dimensional case.

Besides, let us consider the continuous function
$g:B^3\rightarrow B^2$ where
$g:B^3\rightarrow B^2$ is the projection
given by $g:(P_x,P_y,P_z)\to (P_x,P_y)$
being $P=(P_x,P_y,P_z)\in B^3$.
Besides,
$B^2$ can be triangulated by  the simplicial complex $L$ whose maximal simplex is the triangle
with set of vertices $L^{(0)}=\{(0,0)$, $(1,0)$, $(0,1)\}$ and the homeomorphism $\tau_L:B^2\to |L|$, whose inverse is
$$P'=\tau^{-1}_L(P)=\lambda(P)\cdot \Big(P-\big(\frac{1}{4},\frac{1}{4}\big)\Big)$$
where $P\in |L|$ and $\lambda(P)$ is computed in a similar way than above.  See Figure~\ref{fig:lambda}.

\begin{figure}
    \centering
    \includegraphics[width = 0.8 \linewidth]{figs/homotecia_2.pdf}
    \caption{Geometric visualization of the computation of the parameter $\lambda$ in the homeomorphism $\tau_L$ by which $P$ is mapped to $P'$.}
    \label{fig:lambda}
\end{figure}

Now, let us approximate $g$ with a two-hidden-layer neural network. First,
let us observe that a simplicial approximation $\varphi_c$ of
$\tau_L\circ g\circ \tau^{-1}_K$ is given by the vertex set
$\varphi:K^{(0)}\to L^{(0)}$
defined as $\varphi((0,0,0)) = (0,0)$, $\varphi((1,0,0))=(1,0)$, $\varphi((0,1,0))=(0,1)$, and $\varphi((0,0,1))=(0,1)$. Once the simplicial approximation is computed, we can determine the specific two-hidden-layer neural network ${\cal N}_{\varphi}$
that acts equivalently to $\varphi_c$. Concretely, the architecture of ${\cal N}_{\varphi}$ is composed by an input layer with $3$ neurons, a first hidden layer with $4$ neurons, a second hidden layer with $3$ neurons, and an output layer with $2$ neurons. The weights and bias can be computed following the proof of Theorem \ref{th:simplicialmap_nn}:
$$
\begin{pmatrix}
{0} & {1} & {0} & {0} \\ {0} & {0} & {1} & {0} \\ {0} & {0} & {0} & {1} \\ {1} & {1} & {1} & {1}
\end{pmatrix}^{-1}=
\begin{pmatrix}
{-1} & {-1} & {-1} & {1} & {1} \\ {1} & {0} & {0} & {1} & {0} \\ {0} & {1} & {0} & {1} & {0} \\ {0} & {0} & {1} & {1} & {0}
\end{pmatrix}
$$
Then,
$
W^{(1)}=\begin{pmatrix}
{-1} & {-1} & {-1}  \\
{1} & {0} & {0}   \\
{0} & {1} & {0}   \\
{0} & {0} & {1}
\end{pmatrix}
$
and
$b_1 = \begin{pmatrix}
1 \\
0 \\
0\\
0
\end{pmatrix}$.

$$
W^{(2)}=
\begin{pmatrix}
{1} & {0} & {0} & {0} \\ {0} & {1} & {0} & {0} \\ {0} & {0} & {1} & {1}
\end{pmatrix}
$$
$$
W^{(3)} =
\begin{pmatrix}
0&1&0 \\
0&0&1
\end{pmatrix}
$$

In this
straightforward
example, the spaces
$B^3$ and  $B^2$ are approximated by just one $3$-simplex and one $2$-simplex, respectively. Therefore, $\tilde{m}_{(K,\tau_K)}(B^3)$ and $\tilde{m}_{(L,\tau_L)}(B^2)$ are upper bounded by the size of $B^3$ and $B^2$, respectively. If we want  a better approximation  to $g$, we should apply the barycentric subdivision to $K$ and $L$, respectively. Doing that,
 we would  obtain six maximal $2$-simplices in $L$, and  sixteen maximal $3$-simplices in $K$. Hence, the architecture of the neural network will consist of $64$ neurons in the first hidden layer, and $18$ neurons in the second hidden layer.

\section{Conclusion}\label{sec:conclusions}
In this paper, we have provided an effective method to build a multi-layer feed-forward network which approximates a continuous function between triangulable spaces. The main contribution of the paper is the proof that the weights can be exactly
computed without any training process.
Although the homeomorphisms between the triangulable spaces and the simplicial complexes can be hard to find and  the classic theorem for approximations through
neural networks is valid for compact sets and our result is only valid for triangulable spaces,
most of the real-world problems are covered by our result
and, therefore, approximations to continuous functions
through
neural networks can  effectively be built. Besides,
our method can be considered a suitable and powerful tool for the approximation
of continuous functions on triangulable
spaces. Two of the
main advantages of the proposed method are: (1)
knowing
a priori how many hidden neurons are needed
to reach the desired accuracy; and (2)
no need for a training process.

\printcredits

\bibliographystyle{cas-model2-names}


\section{\label{}}

Transition-edge sensors (TESs) are the core sensor technology for an increasing variety of instruments at cutting-edge scientific facilities \cite{barret2018athena, doriese2017practical, galitzki2018simons} and measurements in a variety of different applications including cosmology \cite{spilker2018fast, miller2018massive},  quantum information \cite{giustina2015significant} \cite{shen2018randomness}, neutrino mass measurements \cite{alpert2015holmes}, exotic atom experiments \cite{okada2014high}, and x-ray metrology \cite{fowler2017reassessment}. A TES is a two-dimensional superconducting film that is voltage-biased in the phase transition between the normal and superconducting states.  A TES can be used to very precisely measure small changes in temperature due to the absorption of single photons and particles or the average power deposited by many photons. TESs can be optimized to achieve excellent energy resolution or noise equivalent power. As demanding applications such as x-ray spectroscopy at the LCLS-II \cite{li2018tes} push transition-edge sensors (TESs) to even better energy resolution, it is critical to understand all their potential noise sources. Noise that is commonly observed in TESs in excess of the noise predicted by known mechanisms limits the optimization of TESs for challenging applications. In this letter, we propose a mechanism for the unexplained excess noise, derive an expression for its amplitude, and compare the predictions of the expression to measured noise values.

As soon as TES were carefully compared to noise theories for bolometers\cite{mather1982bolometer} and microcalorimeters \cite{moseley1984thermal}, electrical noise in excess of these models was consistently observed \cite{ullom2004suppression, kinnunen2008reducing, smith2013implications}. For a more detailed discussion on the observation of excess noise see reviews by Irwin and Hilton \cite{irwin2005transition} and Galeazzi\cite{galeazzi2010fundamental}. Noise sources in a TES can be divided into (1) noise sources internal to the TES, (2) noise sources from the circuit in which the TES is embedded such as Johnson noise in the bias resistor and noise in the readout amplifier, and (3) noise from the external environment such as RF-pickup, stray photon arrivals, and fluctuations in the temperature bath. The second and third categories of noise from this list are well understood and are not the source of the excess noise discussed here.

\begin{figure}
\centering
\includegraphics[width=3.4in]{noise_v_freq_03}
\caption{Current noise $(A/\sqrt{Hz})$ as a function of frequency in TES. Blue points are the measured data. The lines are predictions for the thermal fluctuation noise (orange dotted line), SQUID noise (cyan dashed-dotted line), Johnson noise including the $(1+2\beta)$ term of \cite{irwin2006thermodynamics} (red dashed line), and the quadrature sum of the noise components (green solid line). }
\label{fig:m_noise}
\end{figure}

The main internal noise sources that have been observed in TESs are the thermal fluctuation noise (TFN) between the TES and the heat bath, and the voltage fluctuations due to Johnson noise from the resistance of the TES \cite{irwin2005transition}. The power fluctuations due to thermal fluctuation noise are expressed as $S_{P_{TFN}} = 4 k_B T^2 G  F_L$ where the $T$ is the temperature of the TES, $G$ is the thermal conductance to the bath, and $F_L$ is a unitless function that depends on the thermal conductance exponent and on whether phonon transport to the TES is specular or diffuse. $S_{P_{TFN}}$ is frequency independent. The current in a TES is usually measured by a SQUID ammeter, and it is convenient to convert all types of noise into current noise as measured by the SQUID. To convert the thermal power fluctuations to current fluctuations in the TES, we multiply by the the power-to-current responsivity of the TES. Since the power-to-current responsivity is frequency dependent, the resulting current fluctuations are also frequency dependent. The voltage fluctuations due the Johnson noise of the TES are expressed as  $S_V = 4 k_B T R$ where $R$ is the resistance of the TES at its operating point. $S_V$ is independent of frequency. To convert the voltage noise to a current noise as seen in the measured current, we multiply by the internal admittance of the TES circuit.

Figure \ref{fig:m_noise} shows predicted current noise components due to TFN and the TES Johnson noise while the points are measured data for a typical TES. The measured data shows noise in excess of the noise predictions with the same frequency dependence as the TES Johnson noise. This excess noise, often referred to as ``unexplained noise'', is the noise that is not understood and the focus of this letter. It is usually parameterized as an additional factor of $(1+M^2)$ multiplying the expected Johnson noise\cite{ullom2004suppression}.

This noise should not be confused with an additional noise source sometimes observed due to internal thermal fluctuations between distributed heat capacities inside the TES \cite{hoevers2000thermal, gildemeister2001model}. The frequency dependence of internal thermal fluctuation noise (ITFN) can sometimes mimic that of the Johnson noise. However, IFTN is well understood, such that when the TES parameters are carefully measured, it can usually be definitively separated from the unexplained excess noise \cite{wakeham2019thermal}. Also IFTN can usually be suppressed below all other noise sources by use of low resistance TESs.

In 2006, Irwin  \cite{irwin2006thermodynamics} predicted an enhancement of the Johnson noise based on an analysis of a simple nonlinear resistive calorimeter operated out of equilibrium. This analysis predicted that the noise level in a TES is equal to the Johnson noise due to the TES resistance at its operating point multiplied by a factor of $(1+2\beta_I)$ where $\beta_I$ is the logarithmic current sensitivity of the TES defined as
\begin{equation}
\beta_I =  \left. \frac{\partial \log R}{\partial \log I} \right|_{T} = \frac{I}{R} \left. \frac{\partial R}{\partial I} \right|_{T} .
\end{equation}
The parameters $R$, $\beta_I$, and $\alpha_I$, the logarithmic temperature sensitivity defined as
\begin{equation}
\alpha_I =  \left. \frac{\partial \log R}{\partial \log T} \right|_{I} = \frac{T}{R} \left. \frac{\partial R}{\partial t} \right|_{I},
\end{equation}
are commonly used to describe the $R(I,T)$ surface in the small signal limit\cite{irwin2005transition}.

In some scenarios the  $(1+2\beta_I)$ term has reasonably predicted measured noise. In many other scenarios, especially low in the transition and for devices with high $\alpha_I$and $\beta_I$, the $(1+2\beta_I)$ expression dramatically underpredicts the observed noise \cite{jethava2009dependence}. The factor of $(1+2\beta)$ is included in the predicted Johnson noise in Fig.\ \ref{fig:m_noise}.  It is now usual to refer to the excess noise as the noise above the predictions of the $(1 + 2 \beta_I)$ term.  The $M$ parameter is then defined by
 \begin{equation}
 S_V (0) = 4 k_B T R (1 + 2 \beta_I) (1+M^2).
 \end{equation}
In Fig.\ \ref{fig:m_noise}, an additional $M=2.7$ is required to make the predicted noise consistent with the measured data.

A number of explanations for the excess Johnson noise have been proposed including fluctuations due to vortex dynamics \cite{fraser2004nature}, fluctuations in the superconducting phase boundary \cite{luukanen2003fluctuation}, fluctuations in the superconducting order parameter \cite{seidel2004intrinsic}, and by percolation models \cite{bagliani2009study}. However none of these mechanisms give quantitative predictions consistent with the measured dependencies of the excess electrical noise.
Despite more than a decade of theoretical and experimental effort, the magnitude of the unexplained noise in TESs is still not understood.

One obstacle to understanding the excess noise is that there is no consensus model to predict the resistance of TESs biased in the superconducting transition. As implied by a non-zero $\beta_I$, the resistance in a TES is not only a function of temperature, but also a function of current in the film. Any model that attempts to describe the resistance of the TES must include a mechanism that describes the current dependence. Two such models have been proposed, one based on the observation of Josephson junction-like weak-link behavior in TESs \cite{sadleir2010longitudinal} and the other based on intermediate resistance states formed by phase-slips lines  \cite{irwin1998thermal, bennett2014phase}. Based on observations of weak link behavior, Kozorezov et al. \cite{kozorezov2011modelling} proposed modeling TESs using the resistively-shunted junction (RSJ) model. Despite reasonably describing some qualitative features of TESs, the model is not able to successfully predict measured values of $\alpha_I$ and $\beta_I$. An empirical model based on the phase-slip line mechanism, known as the two-fluid model  \cite{irwin1998thermal, bennett2012two}, has been used to predict $\alpha_I$ and $\beta_I$ over some fraction of the transition \cite{morgan2017dependence, pappas2014optical}, but relies on two empirical parameters.

Interestingly, both of these proposed resistance mechanisms predict an induced oscillating supercurrent at the Josephson frequency ($\omega_J = 2 e V / \hbar$). Additionally, both mechanisms imply a non-linear resistance for the TES at least over some range of parameter space. Together, the high frequency oscillations and the non-linear nature of the resistance are suggestive that noise could be mixed down from higher frequencies as is commonly observed in dc-SQUIDs and was studied extensively in weak-link Josephson junctions.

 The amplitude the of Johnson noise mixed down to low frequencies in weak-link junctions was first calculated by Likharev and Semenov  \cite{likharev1972fluctuation} within the context of the RSJ model. In this formulation the only intrinsic source of fluctuations is the normal current. The supercurrent is not a source of fluctuations since the superconducting condensate is an ordered set of electron pairs, which cannot fluctuate independently. Therefore, the normal current passing through the resistive shunt is the source of all the Johnson noise within this model. In the theory of Likharev and Semenov, the spectral density of the voltage noise at frequency $\omega$ is determined by both the voltage noise at that frequency and the noise mixed-down to $\omega$ by noise near the Josephson frequency ($\omega_J$) and its harmonics as
 \begin{equation}
 S_V (\omega) =  \sum_k \left|  Z_k \right|^2  S_I \left( k \omega_J - \omega  \right) \label{psd_z}
 \end{equation}
 where $S_I$ is the spectral density of the current noise and $\left|  Z_k \right|$ are the Fourier coefficients of the junction impedance, defined as $V(\omega) = \sum_k Z_k I (\omega - k \omega_J )$. For frequencies small compared to $\omega_J$ only the $k=-1,0,1$ harmonics of the Josephson current are relevant \cite{likharev1972fluctuation} and the spectral density of the voltage fluctuations at zero frequency is given by
\begin{equation}
S_V (0) = R_d^2 S_I(0) + 2 \left| Z_1 \right|^2 S_I(\omega_J)
\end{equation}
 where $R_d$ is the dynamic resistance of the junction defined as $R_d = \partial V / \partial I$.

By calculating the Fourier coefficients of the junction impedance from differential equations of the RSJ model, Likharev and Semenov were able to derive the power spectral density of the voltage noise across the weak link at frequencies well below the Josephson frequency as
 \begin{equation}
 S_V (0) =   4 k_B T \frac{R_d^2}{R_n} \left[ 1 + \frac{1}{2} \left( \frac{I_c}{I}  \right)^2  \right]. \label{psd_rd_rsj}
 \end{equation}
where $R_n$ is the shunt resistance across the junction, $I$ is the total current in the junction, and $I_c$ is the critical current of the junction. The second term is largest at $I=I_c$ and is reduced as the current is increased above $I_c$. Koch et al.\ \cite{koch1980quantum} later generalized this theory to include quantum effects due to noise from zero-point current fluctuations in the shunt resistor. However, at the voltage biases used in TESs, the quantum effects are obscured by thermal fluctuations and the quantum treatment reduces to Eq.\ \ref{psd_rd_rsj}. Under the assumption that a TES can be described by the RSJ model, we can substitute $R_d = R (1+ \beta_I)$ to rewrite the spectral density of the voltage fluctuations into parameters more usual for TESs as
 \begin{equation}
 S_V (0) =   4 k_B T \frac{R^2}{R_n} \left(1 + \beta_I \right)^2 \left[ 1 + \frac{1}{2} \left( \frac{I_c}{I}  \right)^2  \right]. \label{psd_beta_rsj}
\end{equation}

Inspired by the weak-link effects observed in TESs, Kozorezov et al.\ \cite{kozorezov2012electrical} applied the Smoluchowski equation approach of Coffey et al.\ \cite{coffey2008smoluchowski} in an attempt to explain the excess noise in TESs. The Smoluchowski equation approach is similar to the RSJ model except it takes into consideration thermal fluctuations.  However, in order to reach a calculable expression, Kozorezov et al.\ were forced to use an approximation and the resulting spectral density was the the same as Eq. \ref{psd_beta_rsj}. They then went on to use the RSJ form of the resistance $(R/R_n)^2 =1 - (I/I_c)^2 $ to write the power spectral density of the noise as
\begin{equation}
 S_V (0) \approx 4 k_B T R (1 + \beta_I)^2 \left[ \frac{3}{2} \frac{R}{R_n} \left(1 - \frac{1}{3} \left( \frac{R}{R_n} \right)^2 \right) \right]. \label{psd_alex}
 \end{equation}
Later in the text we compare Eq.\ \ref{psd_alex} to measured data and show that it dramatically underpredicts the observed noise.

Since the transition shape in most TESs is not simply predicted by either the RSJ model or the two-fluid model, we would like a theory for the mixed-down noise that is independent of how closely the dynamic resistance of the device is described by one of the transition models. The first term of Eq.\ \ref{psd_rd_rsj} describes the power spectral density of the current fluctuations at low frequencies that is transformed into voltage fluctuations by multiplying by the local slope of the IV curve. The second term describes how Johnson noise at frequencies near the Josephson frequency interacts with the oscillations at the Josephson frequency to mix noise to low frequencies via the non-linearity of the resistance.

What is required for TESs is a general expression for the second term of Eq.\ \ref{psd_rd_rsj} that does not rely on the specific form of the RSJ model. Kogan and Nagaev  \cite{kogan1988fluctuation} derived a general form for this expression. More recently the expression they derived was compared to experimental data for a weak-link junction by Lhotel et al.\  \cite{lhotel2007divergence}. The relevant Fourier coefficients of the junction impedance are
\begin{equation}
\left| Z_1 \right|^2 = - \frac{V}{4} \frac{\partial R_d}{\partial I}.
\end{equation}
giving the spectral density of the low frequency noise for arbitrary current-phase relationship as
\begin{equation}
S_V (0) = R_d^2 \left[ S_I(0) -  \frac{V}{2 R_d^2} \frac{\partial R_d}{\partial I} S_I(\omega_J) \right] \label{psd_general}.
\end{equation}
In the context of a TES, $\partial R_d / \partial I$ must be evaluated at a constant temperature.

By substituting the dynamic resistance from the RSJ model and its derivative with respect to current into Eq. \ref{psd_general}, we recover Eq.\ \ref{psd_rd_rsj}. For the two-fluid model the dynamic resistance is a constant ($\partial R_d / \partial I=0$), and the second term of Eq.\ \ref{psd_general} is zero. Even though the phase-slip mechanism implies oscillations at the Josephson frequency, in the simplified form of two-fluid model, no noise is mixed down to low frequencies.

Using the definition of $\beta_I$ we can rewrite Eq.\ \ref{psd_general} to get
 \begin{equation}
S_V (0) = R_d^2 \left[ S_I(0) -  \frac{1}{2} \frac{\beta_I}{(1+\beta_I)^2} \frac{\partial R_d}{\partial R} S_I(\omega_J) \right]. \label{noise_noass}
\end{equation}
Equation \ref{noise_noass} makes no assumptions about the transition model except that there are Josephson oscillations being mixed with a device with some dynamic resistance. In order to compare Eq.\ \ref{noise_noass} to data we need to make an assumption about the form of $S_I$. If we take the RSJ model literally then we would assume that we have a normal resistance $R_n$ that shunts the junction and $S_I(0) = 4 k_B T / R_n$. In the TES there is no shunt; the resistance is intrinsic to the TES. When $S_I(0) = 4 k_B T / R_n$ is used, Eq.\ \ref{noise_noass} obviously under predicts the Johnson noise. As we will see in the remainder of this letter, assuming $S_I(0) = 4  k_B T / R$ does a significantly better job fitting the data. This same observation was made by Lhotel et al.\ \cite{lhotel2007divergence} in their comparison with long diffusive superconductor -- normal-metal -- superconductor junctions.

Taking $S_I(0) = 4  k_B T / R$, Eq.\ \ref{noise_noass} becomes
 \begin{equation}
S_V (0) = 4 k_B T R \left(1+\beta_I \right)^2 \left[ 1 -  \frac{1}{2} \frac{\beta_I}{(1+\beta_I)^2} \frac{\partial R_d}{\partial R} \right]. \label{noise_R}
\end{equation}
For comparison with data, we also write down the results of applying the RSJ model,
\begin{equation}
S_V (0) = 4 k_B T R \left(1+ \frac{5}{2} \beta_I + \frac{3}{2} \beta_I^2 \right), \label{noise_RSJ}
\end{equation}
and for the two-fluid model
\begin{equation}
S_V (0) = 4 k_B T R \left(1+ 2 \beta_I +  \beta_I^2 \right) \label{noise_TF}.
\end{equation}

\begin{figure}
\centering
\includegraphics[width=3.4in]{beta01}
\caption{Measured $\beta_I$ for no bar TES (blue points) and 3-bar T-stem TES (orange crosses) from Wakeham et al.\ \cite{wakeham2018effects}. The dashed lines are the two-fluid model predictions of $\beta_I$ for $c_R=1,0.7,0.45$.}
\label{fig:data_beta}
\end{figure}

We want to compare Eqs.\  \ref{noise_R}, \ref{noise_RSJ}, \ref{noise_TF} with data sets where $\beta_I$ and $M$ are known. The ideal data set would have  $\beta_I$ densely sampled around the $R$ values where we compare theory with measured noise.  Wakeham et al.\ \cite{wakeham2018effects} performed careful measurements of the $M$ parameter on well characterized 140 $\mu$m square TESs based on MoAu bilayers with gold banks parallel to the direction of current flow. One of the devices is a so-called T-stem with 3 normal metal stripes. The other TES has no stripes. The measurements of $\beta_I$ and $M$ as function of bias point were performed at 55 mK.

Figure \ref{fig:data_beta} shows the measured $\beta_I$ values for the no bar and 3-bar TESs along with lines corresponding to the predictions of $\beta_I$ versus bias point for the two-fluid model with $c_r = 1,0.7,0.45$. $c_r$ is one of the two empirical parameters in the two fluid model \cite{bennett2012two}. The dashed lines are useful in understanding how $R_d$ is changing with bias point expressed as \% $R_n$. For example, the no bar device follows the $c_r=0.7$ lines above 20\% $R_n$ implying a constant $R_d$ over this region. For a decrease in bias point, if the increase in $\beta_I$ is larger than the predictions of the two-fluid model, Eq.\ \ref{noise_R} predicts additional noise.

 \begin{figure}
\centering
\includegraphics[width=3.4in]{no_bar_noise2}
\caption{The measured noise (blue points) as a function of bias point for the no bar TES. The orange dot-dashed line is the prediction of the usual $(1+2 \beta_I)$ term.  The other lines are the predicted Johnson noise using the two-fluid model (red) Eq.\  \ref{noise_TF},  RSJ model (green long dashed) Eq.\  \ref{noise_RSJ},  Eq.\ \ref{noise_R} (purple dashed) and Eq. \ref{psd_alex} (brown dotted).}
\label{fig:nobar_noise}
\end{figure}

 We use the measured values of $M^2$ and $\beta_I$ from \cite{wakeham2018effects} to calculate the total measured noise with frequency dependence consistent with Johnson nose. This gives the power spectral density of this component of the current noise in units of $4 k_B T R$. A value of one would be the usual Johnson noise spectral density for a resistance $R$ at temperature $T$.  For the no bar device, the extracted noise as a function of bias point in \% $R_n$ is shown as the blue circles in Fig.\ \ref{fig:nobar_noise}. The orange line shows what the prediction of the theory of \cite{irwin2006thermodynamics} where the Johnson noise is enhanced by $(1+2 \beta_I)$. It clearly under predicts the noise at all bias points. This trend is often observed in data with higher $\beta_I$. Eq.\ \ref{psd_alex}, shown as the brown dotted line, also under predicts all bias points. The green line is the prediction of Eq.\ \ref{noise_RSJ} and the red line is the prediction of  Eq.\ \ref{noise_TF}. Both do a better job of predicting the noise than the $1+2 \beta_I$ expression. Finally, the purple dashed line is the prediction from Eq.\ \ref{noise_R}.

The $\partial R_d / \partial R $ term in Eq.\ \ref{noise_R} is estimated using the measured $\beta_I$ data from neighboring points in Fig. \ref{fig:data_beta} to calculate the rate of change of $\beta_I$. Since the neighboring points are at slightly different temperatures, this is only an estimate of $\partial R_d / \partial R $. This approximation gets worse for bias points that are further apart or in regions of especially high $\alpha_I$. The ideal data set would include $\beta_I$ measured at slightly different bath temperatures. Then the thermal model of the TES could be used to find which resistances are at the same temperature for the different bath temperatures and calculate the change in $\beta_I$ for a small change in $R$ while at fixed temperature.

The $R_d$ of the no bar TES is roughly flat above 25 \% $R_n$ as expected from  how well the two-fluid model matched the data over this region. Below 25 \% $R_n$, the $R_d$ is dropping. The resulting positive slope to $R_d$ causes the second term of Eq.\ \ref{noise_R} to be negative actually decreasing the noise compared to the prediction of a linear TES. We are currently exploring if an absolute value of this term was left out in the derivation of Kogan and Nagaev\cite{kogan1988fluctuation}.

\begin{figure}
\centering
\includegraphics[width=3.4in]{three_bar_noise2}
\caption{ (a) The measured noise (blue points) as a function of bias point for the 3-bar T-stem TES. The orange dot-dashed line is the prediction of the usual $(1+2 \beta_I)$ term.  The other lines are the predicted Johnson noise using the two-fluid model (red) Eq.\  \ref{noise_TF},  RSJ model (green) Eq.\  \ref{noise_RSJ}, Eq.\ \ref{noise_R} (purple), and Eq. \ref{psd_alex} (brown dotted). (b) A zoomed version of (a) to compare noise away from the peak of the spiked noise feature.
}
\label{fig:3bar_noise}
\end{figure}

Figure \ref{fig:3bar_noise}a shows the measured noise (blue point) as a function of bias point for the 3-bar T-stem TES. Figure \ref{fig:3bar_noise}b is a zoomed in version of Fig.\ \ref{fig:3bar_noise}a to better show the data away from the peak of the noise spike. The orange line is the prediction of the usual $(1+2 \beta_I)$ term. The green line is the prediction of Eq.\ \ref{noise_RSJ} and the red line is the prediction of  Eq.\ \ref{noise_TF}. Both over predict the noise in some regions, i.e. around 22 \% $R_n$, but both do a better job near the spike in $\beta_I$. The purple dashed line is once again the prediction from Eq.\ \ref{noise_R}. Equation \ref{noise_R} appears to do an excellent job matching the measured noise. The only region it appears to not be a very good match is below 10 \% $R_n$

\begin{figure}
\centering
\includegraphics[width=3.4in]{r11_noise.pdf}
\caption{The measured noise (blue points) as a function of bias point for a 124 $\mu$m 2-bar MoCu TES. The orange dot-dashed line is the prediction of the usual $(1+2 \beta_I)$ term.  The other lines are the predicted Johnson noise using the two-fluid model (red) Eq.\  \ref{noise_TF},  RSJ model (green) Eq.\  \ref{noise_RSJ}, and Eq.\ \ref{noise_R} (purple).
}
\label{fig:r11_noise}
\end{figure}

To compare Eqs.\ \ref{noise_R}-\ref{noise_TF} to a TES of a different bilayer material, we measured noise data for a 124 $\mu$m two bar MoCu TES fabricated by our group. The parameters for the noise fits were measured in the usual way, with the thermal conductance parameters extracted from power law fits, and the heat capacity and $\alpha_I$ and $\beta_I$ extracted from fits to complex impedance. The factor multiplied by voltage noise $(4 k_B T R)$ to fit the measured data is shown in Fig.\ \ref{fig:r11_noise} as function of bias point. Eq.\ \ref{noise_R} once again does a good job predicting the excess noise, despite only having a limited number bias points to estimate $\partial R_d / \partial R$. The need for more densely sampled data is especially apparent at the top of noise spike between 19 \% $R_n$ and 20.5 \% $R_n$ where both the measured $\beta_I$ and excess noise peak.

Equation \ref{noise_R} needs to be carefully compared to a much larger variety of TESs. However, the two TESs discussed here are excellent examples of the types of devices where $(1+2 \beta_I)$ term fails to predict the observed noise. Although Eq.\ \ref{noise_R} does not perfectly predict the measured excess noise, the agreement is impressive given the approximations used and the uncertainty in the measurements.  In the literature, there are a few examples of TESs that show $M=0$ across most of the transition. However, these are the exception rather then the rule. They usually correspond to devices that have $\beta_I$ less than one. TESs with low $\beta_I$ are usually consistent with a dynamic resistance that is approximately linear as a function of current at a fixed temperature and well described by the two-fluid model. Then, for small values of $\beta_I$ Eq.\ \ref{noise_TF} is consistent with $(1+2 \beta_I)$. Eq.\ \ref{noise_R} could also explain measurements where dramatically different levels of excess noise are observed for the same $\beta_I$ \cite{jethava2009dependence} since the amount of noise mixed down from higher frequencies depends not just on $\beta_I$ but also on how $\beta_I$ is changing across the transition.

Unexplained excess noise has been a major impediment to designing TESs that achieve the best possible energy resolution and noise equivalent power. Using the measured dynamic resistance for a given TES and including the noise mixed down from frequencies near the Josephson frequency into the noise model for TESs are important steps to explaining the excess noise and realizing the full potential of TESs.

\begin{acknowledgments}
\end{acknowledgments}