text:
\section{Introduction}

With the advent of deep learning, deep neural networks (DNNs) have gained popularity, and they have been applied to a wide variety of problems~\cite{haykin2009neural,LeCun2015}.
When it comes to sequence modeling and prediction, Recurrent Neural Networks (RNNs) have proved to be the most suitable ones~\cite{LeCun2015}.
Essentially, RNNs are feedforward networks with feedback connections. This feature allows them to capture long-term dependencies among the input variables.
Despite their good performance, they are very sensitive to their \emph{hyperparameter} configuration and hard to train~\cite{bengio1994learning,haykin2009neural,Ojha2017,Pascanu2013}.

Finding an appropriate hyperparameter setting has always been a difficult task. The conventional approach to tackle this problem is to do a trial/error exploration based on expert knowledge. In other words, a human expert defines an architecture, sets up a training method (usually a gradient descent-based algorithm), and performs the training of the network until some criterion is met. Lately, automatic methods based on optimization algorithms, e.g., grid search, evolutionary algorithms or Bayesian optimization (BO), have been proposed to \emph{replace} the human expert. However, due to the immense size and complexity of the search space, and the high computational cost of training a DNN, hyperparameter optimization still poses an open problem~\cite{haykin2009neural,Ojha2017}.

Different approaches have been proposed for improving the performance of  hyperparameter optimization, ranging from evolutionary approaches (a.k.a.~neuroevolution)~\cite{Ojha2017}, to techniques to speed up the evaluation of a DNN~\cite{Camero2018lowcost,Domhan2015}.
Among these approaches, the \emph{Mean Absolute Error Random Sampling} (MRS)~\cite{Camero2018lowcost} poses a promising ``low-cost, training-free, rule of thumb'' alternative to evaluate the performance of an RNN, which drastically reduces the evaluation time.

In this study, we propose to tackle the architecture optimization problem with a hybrid approach. Specifically, we combine BO~\cite{jones1998efficient,movckus1975bayesian} for optimizing the architecture, MRS~\cite{Camero2018lowcost} for evaluating the performance of candidate architectures, and ADAM~\cite{kingma2014adam} (a gradient descent-based algorithm) truncated through time for training the \emph{final} architecture on a given problem. We benchmark our proposal on three problems (the sine wave, the filling level of 217 recycling bins in a metropolitan area, and the load demand forecast of an electricity company in Slovakia) and compare our results against the state-of-the-art.

Therefore, the main contributions of this study are:\vspace{-0.2cm}
\begin{itemize}
    \item We define a method to optimize the architecture of an RNN based on BO and MRS that significantly reduces the time without compromising the performance (error),\vspace{-0.2cm}
    \item We introduce multiple alternatives to cope with the variable-length solution problem. Specifically, we study three encoding schemes and two penalty approaches (i.e., the infeasible representation and the constraint handling), and\vspace{-0.2cm}
    \item We propose a strategy to improve the performance of the surrogate model of BO for variable-length solutions based on the augmentation of the initial set of solutions, i.e., the \emph{warm-start}.
\end{itemize}

The remainder of this article is organized as follows: Section~\ref{sec:related} briefly reviews some of the most relevant works related to our proposal. Section~\ref{sec:approach} introduces our proposed approach. Section~\ref{sec:experiments} presents the experimental study, and Section~\ref{sec:conclusions} provides conclusions and future work.

\section{Related Work}
\label{sec:related}

In this Section, we summarize some of the most relevant works related to our proposal. First, we introduce the architecture optimization problem and some interesting proposals to tackle it in section~\ref{subsec:hyper-opt}. Second, we present the \emph{neuroevolution}, a research line for handling the problem (section~\ref{subsec:neuroevolution}). After briefly reviewing the Mean Absolute Error Random Sampling (MRS) method in section \ref{subsec:MRS} we finally introduce Bayesian Optimization in section~\ref{subsec:bayesopt}.

\subsection{Architecture Optimization}
\label{subsec:hyper-opt}

The existing literature teaches us on the importance of optimizing the architecture of a deep neural network on a particular problem, including, for example, the type of activation functions, the number of hidden layers, and the number of units for each layer~\cite{Bergstra2011,Camero2018lion,Jozefowicz2015}. For DNNs, the architecture optimization task is usually faced by either manual exploration of the search space (that is usually guided by expert knowledge) or by automatic methods based on optimization algorithms, e.g., grid search, evolutionary algorithms or Bayesian optimization~\cite{Ojha2017}.

The challenges here are three-fold: firstly, the search space is typically huge due to the fact that the number of the parameters increases in proportion to the number of layers. Secondly, the search space is usually a mixture of real (e.g., the weights), integer (e.g., the number of units in each layer) and categorical (e.g., the type of activation functions) values, resulting in a demanding optimization task: different types of parameters naturally require different approaches for handling them in optimization. Last, the architecture optimization falls into the family of expensive optimization problems as function evaluations in this case are highly time consuming (which is affected both by the size of training data and the depth of the architecture). In this paper, we shall denote the search space of architecture optimization as $\mathcal{H}$. The specification of $\mathcal{H}$ depends on the choice of encoding schemes of the architecture (see Section~\ref{subsec:encoding}).

To tackle the mentioned issues, many alternatives have been explored, ranging from reducing the evaluation time of a configuration (e.g., early stopping criteria based on the learning curve~\cite{Domhan2015} or MRS~\cite{Camero2018lowcost}) to \emph{evolving} the architecture of the network (neuroevolution).

On the other hand, when it comes to RNN optimization, there are two particular issues: the exploding and the vanishing gradient~\cite{bengio1994learning}. Many alternatives have been proposed to tackle with this problems~\cite{Pascanu2013}. One of the most popular ones is the Long Short Term Memory (LSTM) cell~\cite{hochreiter1997long}. However, in spite of its ability to effectively deal with these issues, the problem still remains open, because the learning process is also affected by the weight initialization strategy~\cite{ramos2017quantitative} and the algorithm parameters~\cite{haykin2009neural}.

\subsection{Neuroevolution}
\label{subsec:neuroevolution}

Neuroevolutionary approaches typically represent the DNN architecture as solution candidates in specifically designed variants of state-of-the-art evolutionary algorithms.
For instance, genetic algorithms (GA) have been applied to evolve increasingly complex neural network topologies and the weights simultaneously, in the so-called NeuroEvolution of Augmenting Topologies (NEAT) method~\cite{larochelle2009exploring,stanley2002evolving}. However, NEAT has some limitations when it comes to evolving RNNs~\cite{miikkulainen2019evolving}, e.g., the fitness landscape is deceptive and a large number of parameters have to be optimized. For RNNs, NEAT-LSTM~\cite{rawal2016evolving} and CoDeepNeat~\cite{Liang:2018:EAS:3205455.3205489} extend NEAT to mitigate its limitations when evolving the topology and weights of the network. Besides NEAT, there are several evolutionary-based approaches to evolve an RNN, such as EXALT~\cite{elsaid2019evolving}, EXAMM~\cite{ororbia2019investigating}, or a method using ant colony optimization (ACO) to improve LSTM RNNs by refining their cellular structures~\cite{ElSaid2018}.

A recent work~\cite{camero2019specialized} suggested to address the issue of huge training costs when evolving the architecture. In that research, the objective function, that is usually evaluated by training the candidate network on the full data set evolved by a complete training of the candidate network, instead it is approximated by the so-called MAE random sampling (MRS) method, in which no actual training is required. In this manner, the time required for a function evaluation is drastically reduced in the architecture optimization process.

\subsection{Mean Absolute Error Random Sampling}
\label{subsec:MRS}

MAE Random Sampling is an approach to evaluate the expected error performance of a given architecture. First, the weights of the network are randomly initialized. Second, the error is calculated (i.e., the real and expected output are compared). This two-step process is repeated, and the errors are accumulated. Then, a probabilistic density function (e.g., a truncated normal distribution) is fitted to the error values. Finally, the probability of finding a set of weights whose error is below a user-defined \emph{threshold} is estimated. In other words, by using a random sampling of the output (error), we are estimating how \emph{easy} (i.e., a high probability) it would be to find a \emph{good} (i.e., small error) set of weights.

Given a training data set $\mathcal{D} = \{(\mathbf{x}_i, y_i)\}_i^N, \mathbf{x}_i\in\mathbb{R}^n$, for a given network architecture $\mathbf{h}\in\mathcal{H}$ and $Q$ i.i.d. random weight matrices $\{\mathbf{W}_i\}_{i=1}^Q, \mathbf{W}_i \sim \bm{\mathcal{N}}(\mathbf{0}, \mathbf{I})$, the \emph{Mean Absolute Error} (MAE) of this RNN is denoted as $\mathcal{E} = \{\operatorname{MAE}(\mathcal{D}, \mathbf{h}, t, \mathbf{W}_{i})\}_{i=1}^Q$, where $t$ is the number of time steps in the past used for the prediction. Let $\mu$ and $\sigma$ denote the sample mean and standard deviation of the error sample $\mathcal{E}$. Then the so-called \emph{Mean Absolute Error Random Sampling} (MRS) measure is defined as the empirical probability of obtaining a better error rate than a user-specified threshold $p_{\text{m}}$:
\begin{equation} \label{eq:MRS}
    \textsc{mrs}(\mathcal{D}, \mathbf{h}, t, p_{\text{m}}, Q) = \frac{\Phi\left(\frac{p_m-\mu}{\sigma}\right) -\Phi\left(-\frac{\mu}{\sigma}\right)}{1- \Phi\left(-\frac{\mu}{\sigma}\right)},
\end{equation}
where $\Phi$ stands for the cumulative distribution function (CDF) of the standard normal distribution. The MRS value is calculated from a truncated normal distribution (on the interval $[0,\infty)$), whose location and scale parameters are set to the sample mean $\mu$ and standard deviation $\sigma$, respectively. Throughout this paper, we shall set $p_{\text{m}}$ to $1\%$. Intuitively, the higher MRS value a network architecture that yields a higher MRS value would be more likely to possess a much smaller (hence better) MAE rate after the backpropagation training. Hence, it seems promising to use MRS as a training-free estimation for the performance of neural networks.

In this paper, we shall adopt MRS as the objective function (that is subject to maximization) for the architecture optimization. For a detailed discussion of MRS, please refer to~\cite{Camero2018lowcost}.

\subsection{Bayesian Optimization}
\label{subsec:bayesopt}

The so-called \emph{Bayesian Optimization} (BO) (a.k.a.~Efficient Global Optimization)~\cite{jones1998efficient,movckus1975bayesian} algorithm has been applied extensively for automated algorithm configuration tasks~\cite{1554761,horn2015model,Hutter2011}. Bayesian optimization is a sequential global optimization strategy that does not require the derivatives of the objective function and is designed to tackle expensive global optimization problems. Given a real-valued \emph{maximization problem} $f\colon \mathcal{H}\rightarrow \mathbb{R}$ (e.g., $f=\textsc{mrs}$ in the following), BO employs a surrogate model, e.g., Gaussian process regression (GPR) or random forests (RF), to approximate the landscape of the objective function, which is trained on an initial data set $(X, Y)$. Here, $X\subset\mathcal{H}$ is typically sampled in the search space $\mathcal{H}$ using the Latin Hypercube Sampling (LHS) method and $Y=\{f(\mathbf{h})\colon \mathbf{h}\in X\}$ is the set of function values of points in $X$. Essentially, the prediction from surrogate models and the estimated prediction uncertainty are considered simultaneously to propose new candidate solutions for the evaluation. Loosely speaking, the model prediction and its uncertainty are taken as input to the so-called \emph{acquisition function} (or \emph{infill criterion}), which can be interpreted as the utility of unseen solutions and hence is subject to maximization when proposing new candidate solutions. An example of commonly used acquisition functions is the Expected Improvement (EI)~\cite{movckus1975bayesian}. Given the predictor $m\colon \mathcal{H}\rightarrow \mathbb{R}$, the uncertainty of predictions $s(\mathbf{h})\coloneqq \mathbb{E} \{(m(\mathbf{h}) - f(\mathbf{h}))^2\}$ of the surrogate model and the current best function value $y_{\text{max}}=\max\{Y\}$, the EI criterion can be expressed for an unknown point $\mathbf{h}\in\mathcal{H}$:
\begin{align}\label{eq:EI}
\scriptsize
	\operatorname{EI}(\mathbf{h}) & =I(\mathbf{h})\Phi\left(\frac{I(\mathbf{h})}{s(\mathbf{h})}\right) + s(\mathbf{h})\phi\left(\frac{I(\mathbf{h})}{s(\mathbf{h})}\right),
\end{align}
where $I(\mathbf{h}) = m(\mathbf{h}) - y_{\max}$ and where $\phi$ stands for the probability density function (PDF) of the standard normal distribution. Note that the new candidate solution is generated by maximizing the EI criterion, namely
\begin{equation}\label{eq:EI-maximization}
	\mathbf{h}^* = \argmax_{\mathbf{h}\in\mathcal{H}}\operatorname{EI}(\mathbf{h}).
\end{equation}

After evaluating the new candidate solution $\mathbf{h}^*$, $\mathbf{h}^*$ and its objective function value are included in the data set $(X,Y)$ and the surrogate model will be re-trained. Please, see~\cite{8122656} for an overview of the acquisition functions.

Despite being a proven technique for automated algorithm configuration tasks~\cite{1554761,horn2015model,Hutter2011}, the state-of-the-art of BO does not reconcile well with variable-length solution problems~\cite{kim2019bayesian,shahriari2015taking}. Therefore, in this study we propose multiple strategies to cope with variable-length solutions (inherent to the architecture search problem).

\subsection{Our contribution}
Herein, we briefly summarize the novelty of the architecture search described in the following sections and compare those to the state-of-the-art works reviewed in this section.
\begin{itemize}
    \item We propose to use the Mean Absolute Error Random Sampling (MRS) procedure as the objective function for the architecture search, which is relatively much inexpensive compared to full training of the same architecture on the same data. In contrast to employing full training, e.g.,~\cite{rawal2016evolving}, our approach could allow for more iterations of the Bayesian optimization algorithm.
    \item We designed three different encoding schemes that turn the neural architecture search that is inherently a variable-dimensional problem(for instance, the NEAT~\cite{larochelle2009exploring} approach operates on the network topology directly) into an optimization problem with fixed dimensions, hence facilitating the application of surrogate modeling and Bayesian optimization accordingly.
    \item We contemplated making the Bayesian optimization more efficient and effective by imposing a penalty on the infeasible solutions or warm-starting the search process with infeasible solutions.
\end{itemize}

\section{The proposed approach}
\label{sec:approach}

In this paper, it is proposed to optimize the architecture of an RNN by a combination of Bayesian optimization (BO) and Mean Absolute Error Random Sample (MRS) to reduce the running time of the architecture search. Specifically, this is to solve the following problem using \emph{Bayesian optimization},
\begin{equation}
\argmax_{\mathbf{h}\in\mathcal{H}}\textsc{mrs}(\mathcal{D}, \mathbf{h}, p_{\text{m}}, Q),
\end{equation}
given a training data set $\mathcal{D}$, a cutoff threshold $p_{\text{m}}$ and the number of random weights used in MRS. Importantly, as the architecture could shrink and expand in the search, its natural representation takes a variable-length form, which does not reconcile well with the state-of-the-art BO algorithm. To resolve this issue, three fixed-length encoding schemes are proposed to represent network architectures with variable sizes. Note that in this paper the search space $\mathcal{H}$ is determined by each encoding scheme (please see below). Also, we only employ the random forest model in the Bayesian optimization procedure (described in Alg.~\ref{alg:bo}) for the following reason: the design space of neural architecture comprises of integer/Boolean variables, which can be dealt with naturally by random forests. Gaussian process regression, which works over Euclidean spaces, is by default not applicable in this scenario. Although there are many recently endeavours in extending GPR's ability to handle the discrete and integer variables (e.g.,~\cite{Nguyen0RSV20}), it is not our major aim herein to compare the performance of Bayesian optimization when coupled with different surrogate models and hence we decided to choose the simplest random forest model to validate the proposed algorithm.

\subsection{Encoding Schemes}
\label{subsec:encoding}

Assuming that the number of neurons per each layer is restricted to the range $[\munderbar{N}..\bar{N}]$, the number of layers is $m\in[\munderbar{M}..\bar{M}]$, and $T$ denotes the maximum number of steps taken in back-propagation throughout time, three encoding schemes are proposed in this paper:
\begin{itemize}
    \item \textbf{Plain}: the total length of this encoding is $m +1$.
    $$\mathbf{h}=\left[h_1,  h_2, \ldots, h_m, l\right] \in \left(\{0\} \cup [\munderbar{N}..\bar{N}]\right)^m \times [1..T],$$
    where $h_i$ is the number of neurons per each layer and $l$ is the number of time steps. Note that $h_i$ can take value zero, meaning there is no neuron in this layer and hence it is effectively dropped in the decoding procedure.
    \item \textbf{Flag}: the total length of this encoding is $2m + 1$.
    $$\mathbf{h}=\left[h_1, b_1, h_2, b_2, \ldots, h_m, b_m, l\right] \in [\munderbar{N}..\bar{N}]^m \times \{0, 1\}^m \times [1..T],$$
    where $b_i \in \{0, 1\}$ is the so-called ``flag'' that disables layer $h_i$ if $b_i = 0$ when decoding such a representation to compute the actual architecture.
    \item \textbf{Size}: the total length of this encoding is $m + 2$.
    $$\mathbf{h}=\left[h_1,  h_2, \ldots, h_m, s, l\right] \in [\munderbar{N}..\bar{N}]^m \times [1..m] \times [1..T],$$
    where $s\leq m$ is the number of layers from the start of the representation that are considered in decoding, namely only $h_1, h_2, \ldots, h_s$ are used to generate the actual architecture.
\end{itemize}

\begin{figure}[!htbp]
 \centering
 \begin{tikzpicture}[scale=.8, every node/.style={scale=.8}]
     \matrix[nodes={draw, fill=blue!20, minimum size=10mm, line width=1pt},
         row sep=1mm,column sep=0cm] {
     \node[draw=none,fill=none] {Plain};&
     \node[square] {$h_1$}; &
     \node[square] {$h_2$}; &
     \node[square] {$\ldots$}; &
     \node[square] {$h_m$}; &
     \node[square] {$l$}; \\
     \node[draw=none,fill=none] {Size};&
     \node[square] {$h_1$}; &
     \node[square] {$h_2$}; &
     \node[square] {$\ldots$}; &
     \node[square] {$h_m$}; &
     \node[square] {$s$}; &
     \node[square] {$l$}; \\
     \node[draw=none,fill=none] {Flag};&
     \node[square] {$h_1$}; &
     \node[square] {$b_1$}; &
     \node[square] {$\ldots$}; &
     \node[square] {$h_m$}; &
     \node[square] {$b_m$}; &
     \node[square] {$l$}; \\
     };
 \end{tikzpicture}
 \caption{Illustrations of the proposed encoding schemes.}
 \label{figure:encodings}
\end{figure}

We shall use the notation $\texttt{code} \in \{\text{plain},\text{flag},\text{size}\}$ for the encoding scheme henceforth. In this manner, a fixed-length representation can be used to optimize variable-size architectures. For each case, in the decoding procedure, an output layer is appended to the RNN structure encoded in the search algorithm, to match the expected output dimension. Note that the activation function of the output layer has to be set according to the type of the task in each problem.

\subsection{Decoding}
\label{subsec:valid}

It is worthwhile to note that the decoding procedure of all three representations is a \emph{many-to-one mapping}. For instance, given a plain representation with a maximum of five layers ($m = 5$), $[h_1,h_2,0,0,h_5,l]$ and $[h_1,h_2,h_5,0,0,l]$ are representing exactly the same architecture. If $[h_1,h_2,h_5,0,0,l]$ has already been evaluated in the optimization process, then assessing the performance of $[h_1,h_2,0,0,h_5,l]$ is purely redundant. To determine the equivalence among representations, it is necessary to apply appropriate decoding functions for each type of representation:
\begin{equation} \label{eq:decoding}
    \texttt{decode}(\mathbf{h}) =
    \begin{cases}
        \text{keep } h_i \text{ if } h_i > 0, i=1,\ldots,m. &\text{if the plain encoding} \\
        \text{keep } h_i \text{ if } b_i =1, i=1,\ldots,m. &\text{if the flag encoding}\\
        \mathbf{h} \mapsto [h_1, h_2, \ldots, h_s, l] &\text{if the size encoding}
    \end{cases}
\end{equation}
As the decoding function is a many-to-one mapping, the BO algorithm could potentially propose the same architecture constantly (even with different representations before decoding), and hence the search efficiency would be drastically affected due to the following facts 1) the convergence of BO would be hampered as such an iteration (where the seen architecture is proposed again) makes no progress and the there is no information gain for the surrogate model therein, and 2) the same network architecture has to be evaluated again by MRS, which is wasteful even if MRS is much more efficient as compared the full network training.
To cope with the former issue, it is important to avoid proposing the same architecture again as much as possible. In this study, we propose two alternative strategies which both rely on the definition of \emph{``infeasibility''} (please see below) for representations:
\begin{itemize}
	\item to set the MRS value of infeasible representations to the worst possible value (zero), which will be learned by the surrogate model underlying BO. Hence, the infeasible ones would not likely to be proposed by the surrogate model, or
	\item to use the original MRS values (as in Eq.~\eqref{eq:MRS}) and add constraints on the EI criterion to screen out infeasible representations. Note that in this case the surrogate model will be built on the original MRS values.
\end{itemize}
For the latter, the simplest solution is to maintain a lookup table to register the architectures (together with objective values) that are evaluated before.

\paragraph{Infeasible representation}
Taking the plain encoding scheme as an example, a representation taking the form $[h_1, \ldots h_q, 0, \ldots,0, l]$ (where $h_i >0$) shall be called \emph{feasible}, e.g., $[h_1,h_2,h_5,0,0,l]$ is an infeasible representation when $m=5$. $[h_1,h_2,h_5,0,0,l]$ represents the same architecture with the other $16$ representations (by inserting two zeros at four different positions, e.g., $[h_1,h_2,0,0,h_5,l]$ and $[h_1,0,h_2,0,h_5,l]$). The other representations shall be called \emph{``infeasible''}, which will be assigned with a fixed objective value that is worse than all the feasible solutions. Particularly, since we are maximizing MRS (which is a probability value), we set the penalized objective function value to be equal to zero. The rationale behind this treatment is that whenever the Bayesian optimization (BO) algorithm proposes an infeasible representation, the penalized objective function value will be learned by the surrogate model of BO and hence the chance of generating such representations will diminish gradually. In this manner, we are guiding the optimization process through the feasible ones and thus the search space is virtually reduced. Note that the BO algorithm still needs to make lots of infeasible trials before it stops proposing the infeasible ones, due to the large combinatorial space. It is conceptually better to directly avoid generating such representations by a constraint handling method (see below). The idea of defining the infeasible representation can be easily extended to the flag encoding scheme by \emph{masking} $h_i$ with $b_i$, i.e., replacing the value of $h_i$ with a zero if $b_i$ is equal to zero. However, this idea can not be applied to the \emph{size} encoding scheme.

\paragraph{Constraint handling} To avoid generating infeasible representations, we propose to assign penalty values to infeasible ones and to use a constraint handling method when proposing new candidate representations in BO.
In addition, representations that are already evaluated will be also be penalized by the length of itself (the maximum penalty at line $4$). For an infeasible representation that has \emph{not} been evaluated (line $5$), the number of zeros located before the last nonzero element is used as the penalty value. In line $7$, the decoded representation is registered in a set $L$ to check whether a representation has been evaluated before. The penalty value will be added to the EI criterion when proposing the candidate representations (see line $13$ of Alg.~\ref{alg:bo}). As for the constraint handling, a \emph{dynamic penalty} method is adopted here, where the penalty value will be scaled up with increasing iterations of BO. We choose the dynamic penalty here because it yields a relatively small penalty in the early phase of the search, allowing for exploring the infeasible regions within the search space, which is particularly critical to move between disconnected feasible regions. Also, as the search iteration increases, the penalty value will be enlarged to ensure a feasible solution as the outcome. In this manner, the following penalized infill criterion is used to propose candidate representations (instead of Eq.~\ref{eq:EI-maximization}):
\begin{equation} \label{eq:constrained-EI}
    \mathbf{h}^* = \argmax_{\mathbf{h}\in \mathcal{H}}\operatorname{EI}(\mathbf{h} ; \mathcal{M}) - Ct\cdot\textsc{penalty}(\mathbf{h}, X),
\end{equation}
where 1) $X$ is a set containing all evaluated solutions (not decoded), 2) $t$ is the iteration counter of BO, and 3) $C=0.5$ is a scaling factor. The intuition of this treatment is that the penalty value would have a large impact on the maximization of EI in the late stage, such that the probability of generating infeasible solutions becomes marginal. Also, the penalty value of $\mathbf{h}$ equals its length when it has been evaluated before, i.e., $\mathbf{h}\in X$, for avoiding proposing duplicated solutions, and otherwise, it is set to penalize $\mathbf{h}$ by the number of zeros preceding non-zero elements thereof, namely,
\begin{equation} \label{eq:penalty}\scriptsize
\textsc{penalty}(\mathbf{h}, X)=
\begin{cases}
  \texttt{length}(\mathbf{h}), &\text{ if }  \mathbf{h} \in X \\
  |\left\{h_i \colon \forall i \in [1..n-1]\left(h_i = 0 \cap\exists j \in [i+1..n](h_j = 1)\right)\right\}|, &\text{ otherwise}.
\end{cases}
\end{equation}

\subsection{A Warm-start Strategy}
\label{subsec:warm-start}

Within the Bayesian optimization algorithm, a surrogate model (e.g., a random forest) is used to learn the mapping from the evaluated solutions to the corresponding objective values. Typically, the Bayesian optimization starts with initializing the surrogate model by some randomly generated solutions. The basic idea of the so-called ``warm-start'' strategy is to augment the initial solutions by a set of infeasible solutions that can be generated before the optimization, such that the optimization process is started with a priori information. The infeasible solutions can be generated by randomly picking some components of a solution and setting them to zero for both the plain and flag encoding. Additionally, the objective value of those infeasible solutions is assigned with some default bad value (it is set to zero here since the MRS measure, which is the objective function of the architecture search, is bounded by zero from below), without the need to execute the MRS procedure. We anticipate that this warm-start strategy will add a bias in proposing the new candidate solutions in BO, steering the optimization process away from the infeasible solutions.

In all, the pseudo-code of the proposed approach is described in Alg.~\ref{alg:bo}. After creating the initial data set of BO $(X,Y)$ using Latin Hypercube Sampling~\cite{mckay1979comparison}, the user can choose to turn on the generation of the warm-data prior to the optimization loop (lines $6$-$9$). A set $X'$ consisting of decoded representations is meant to track all the unique architectures that have been evaluated in MRS (line $11$). In line $16$, the constrained EI maximization is applied if the constraint method is enabled. The newly proposed solution $\mathbf{h}^*$ is decoded (line $20$), after which we check if its decoded counterpart ${\mathbf{h}^*}'$ has been evaluated (line $21$). If ${\mathbf{h}^*}'$ is not evaluated before (line $22$-$28$), the feasibility of $\mathbf{h}$ is then checked and its objective value is set to zero in case of being infeasible (Otherwise, we evaluate its decoded representation ${\mathbf{h}^*}'$ in MRS (line $26$)) If ${\mathbf{h}^*}'$ has been evaluated before, its objective value is looked up in the data set $(X,Y)$ (line $30$ and $31$). The newly proposed candidate representation and its objective value are appended to BO's data set $(X, Y)$ (lines $33$ and $34$). Afterwards, the random forest model is re-trained on the augmented data set (line $35$).

\begin{algorithm}[!htp]
	\caption{Efficient Architecture Optimization for RNNs}\label{alg:bo}
	\begin{algorithmic}[1]
		\State \textbf{input}: A data set $\mathcal{D}$, an encoding scheme $\texttt{code} \in \{\text{plain},\text{flag},\text{size}\}$, the random forests algorithm \textsc{rf}, and the maximal iteration number
		$t_{\text{max}}$.
		\State \textbf{output}: a full training RNN model
		\State{$C \leftarrow 0.5, t \leftarrow 0, p_{\text{m}} \leftarrow 0.01, Q\leftarrow 100$}
		\State{Determine the search space $\mathcal{H}$ according to \texttt{code}}
		\State{Generate $X\subseteq \mathcal{H}$ using Latin Hypercube Sampling}
		\State{$Y \leftarrow \{\textsc{mrs}(\mathcal{D}, \texttt{decode}(\mathbf{h}), t, p_{\text{m}}, Q)\colon \mathbf{h}\in X\}$}\Comment{evaluate $X$}
		\If{``warm-start'' is enabled}
			\State{generate the warm data $(X_{\text{warm}},Y_{\text{warm}})$} \Comment{See sec.~\ref{subsec:warm-start}}
			\State{$X\leftarrow X\cup X_{\text{warm}}, Y\leftarrow Y\cup Y_{\text{warm}}$}
		\EndIf
		\State{$X' \leftarrow \{\texttt{decode}(\mathbf{h})\colon \mathbf{h} \in X\}$} \Comment{set of evaluated architectures}
		\State{$\mathcal{M}\leftarrow\textsc{rf}(X, Y)$}\Comment{surrogate model training}
		\While{$t < t_{\text{max}}$}
			\If{``constraint-handling'' is enabled}
				\State{$\mathbf{h}^* \leftarrow \argmax_{\mathbf{h}\in \mathcal{H}}\operatorname{EI}(\mathbf{h} ; \mathcal{M}) - Ct\cdot\textsc{penalty}(\mathbf{h}, X)$} \Comment{penalized EI}
			\Else
	 			\State{$\mathbf{h}^* \leftarrow \argmax_{\mathbf{h}\in \mathcal{H}}\operatorname{EI}(\mathbf{h} ; \mathcal{M})$} \Comment{unconstrained case}
	 		\EndIf
	 		\State{${\mathbf{h}^*}' \leftarrow \texttt{decode}(\mathbf{h}^*)$}\Comment{solution decoding (Eq.~\eqref{eq:decoding})}
	 		\If{${\mathbf{h}^*}' \notin X'$} \Comment{for unseen architectures}
	 		    \If{``infeasible-solution'' is enabled \textbf{and} \\ \hspace{1.4cm}$\texttt{code} \neq \text{size}$ \textbf{and} $\mathbf{h}^*$ is infeasible}
	 		        \State{$y^* \leftarrow -\textsc{inf}$}  \Comment{penalty value for the infeasible ones}
	 		    \Else
	 		        \State{$y^* \leftarrow \textsc{mrs}(\mathcal{D}, {\mathbf{h}^*}', t, p_{\text{m}}, Q)$} \Comment{evaluate ${\mathbf{h}^*}'$ using $\textsc{mrs}$}
	 		    \EndIf
	 		    \State{$X'\leftarrow X'\cup \{{\mathbf{h}^*}'\}$} \Comment{add to the set of evaluated architectures}
	 		\Else
	 		    \State{$S \leftarrow \left\{y \colon \forall(\mathbf{h}, y) \in (X, Y) \land \texttt{decode}(\mathbf{h}) = {\mathbf{h}^*}'\right\}$} \Comment{the objective value of evaluated solutions that decodes to the same architecture as ${\mathbf{h}^*}'$}
	 		    \State{$y^* \leftarrow$ sample a value from $S$ uniform at random}
	 		\EndIf
	 		\State{$X\leftarrow X\cup \{\mathbf{h}^*\}, Y \leftarrow Y \cup \{y^*\}$}\Comment{augment the data set}
	 		\State{$\mathcal{M}\leftarrow\textsc{rf}(X, Y)$} \Comment{re-train the random forest model}
	 		\State{$t\leftarrow t + 1$}
	 	\EndWhile
	 	\State $y_{\text{best}} \leftarrow \max\{Y\}$ and $\mathbf{h}_{\text{best}}$ is the corresponding solution to $y_{\text{best}}$
	 	\State $\mathbf{h}_{\text{trained}} \leftarrow \operatorname{ADAM}(\mathcal{D}, \mathbf{h}_{\text{best}})$ \Comment{train the final neural architecture}
	 	\State \textbf{return} $\mathbf{h}_{\text{trained}}$
	\end{algorithmic}
\end{algorithm}

\section{Experiments}
\label{sec:experiments}

In this section, we present the experimental study performed to test the proposed approach. First, we present the three prediction problems used to benchmark the method. Second, we present the experimental setup and the results of several combinations of the three strategies presented, i.e., infeasible solution, warm start, constraint handling, and encoding. Later, we compare the time between MRS and (short training) Adam. Finally, we study the error trade-off while changing the number of MRS samples.

\subsection{Data sets}
\label{subsec:dataset}

We tested the approach on three prediction problems: \emph{sine wave}, \emph{waste}~\cite{Ferrer2018}, and \emph{load forecast}~\cite{chen2004load}.

\paragraph{The \emph{sine wave}} is a mathematical curve that represents a periodic oscillation. Despite its simplicity, it is extensively used to analyze systems~\cite{bracewell1986fourier}. It is usually expressed as a function of time (\emph{t}), where $A$ is the peak amplitude, $f$ the frequency, and $\phi$ the phase (Equation~\ref{equation:sine}). Its study is interesting because, by adding sine waves, it is possible to approximate any periodic waveform~\cite{bracewell1986fourier}. We sampled the sine wave described by: $A=1$, $f=1$, and $\phi=0$, in the range $t \in [0,100]$~seconds, and at 10 samples per second. Then, given a truncated part of the time series (i.e., a time steps number of points of the sampled sine wave), the problem consists in predicting the next value.
\begin{align}\label{equation:sine}
y(t) = A \sin(2 \pi f  t + \phi)
\end{align}

\paragraph{The \emph{waste} problem} introduced in~\cite{Ferrer2018}, consists of predicting the filling level of 217 recycling bins located in the metropolitan area of a city in Spain, recorded daily for one year. Thus, given the historical filling levels of all containers (217 input values per day), the task is to predict the next day (i.e., the filling level of all bins). It is important to notice that this problem has been used as a benchmark in several studies~\cite{ferrer2019bin,camero2019waste,camero2019specialized} and that it is a real-world problem.

\paragraph{The \emph{load forecast} problem} provided by the European Network on Intelligent Technologies for Smart Adaptive Systems (EUNITE, \url{http://www.eunite.org}) as part of a competition~\cite{chen2004load,lang2018short}, is a data set consisting of the electricity load demand of the Eastern Slovakian Electricity Corporation. It was recorded every half hour, from January 1, 1997, to January 31, 1999. Also, the temperature (daily mean) and the working calendar for this period are provided. Then, based on this data, the challenge is to predict the next maximum daily load. In other words, given the load demand (52 variables), i.e., the load demand recorded every half hour (48), the max daily load (1), the daily average temperature (1), the weekday (1), and the working day information (1), the task is to predict the max daily load of the next day (1). Note that the last month is used as the test data, thus our results may be compared directly against the competitors.

\subsection{Performance}
\label{subsec:performance}

We implemented our approach\footnote{Code available in \url{https://github.com/acamero/dlopt}} in Python 3, using DLOPT~\cite{camero2018dlopt}, MIP-EGO~\cite{wang2018cooling}, Keras~\cite{chollet2015keras}, and Tensorflow~\cite{abadi2016tensorflow}. We used LSTM cells to build the decoded stacked architectures (as a way to mitigate the exploding and vanishing gradient problems~\cite{hochreiter1997long}), and Adam truncated through time~\cite{werbos1990backpropagation} (i.e., sharing all parameters in the unfolded models) to train the final solutions, with default parameter values~\cite{kingma2014adam}.

We defined the search space (i.e., the constraints to the RNN architectures) of the three problems studied (Table~\ref{table:search-spaces}) according to the datasets and the state-of-the-art. Particularly, the sine wave search space is taken from~\cite{camero2019specialized} and the waste search space is copied from~\cite{camero2019waste} to enable a direct comparison.

\begin{table}
    \caption{Optimization search spaces}
	\centering
    \begin{tabular}{ lrrr }
    \hline

    Parameter & Load Range & Sine Range & Waste Range \\
    \hline
    Hidden layers (M)     & [1,8]     & [1,3]   & [1,8] \\
    Look back (T)         & [2,30]    & [2,30]  & [2,30] \\
    Neurons per layer (N) & [10,100]  & [1,100] & [1,300] \\
    \hline
	\end{tabular} \\
    \label{table:search-spaces}
\end{table}

Also, to ease the visualisation of the results, we defined the following naming scheme to denote different combinations of encoding, warm start, invalid, and the constraint handling method: \\ $\small \texttt{[constraint][warm start][infeasible][encoding]}$.

Specifically, we use a character to denote each variant: Constraint (\texttt{C}), Warm start (\texttt{w}), Infeasible (\texttt{I}), and Encoding (\texttt{F}: flag, \texttt{S}: size, and \texttt{P}: plain). A dash (\texttt{-}) means that the corresponding alternative was not used. For example, \texttt{-W-F} corresponds to the combination of warm data and the flag encoding (i.e., without constraint handling and without invalid solution penalty).

Finally, we execute $30$ independent runs for each combination of encoding, warm start, and the constraint handling method on a heterogeneous Linux cluster with more than 200 cores and 700 GB RAM, managed by HTCondor. In these experiments we used the optimization parameter values presented in Table~\ref{table:optimization-params}. The remainder of this subsection introduces the performance results for the three problems and some insights into the solutions.

\begin{table}[h]
    \caption{BO and MRS parameter values}
	\centering
    \begin{tabular}{ lrclr }
    \cline{1-2} \cline{4-5}
    Parameter & Value &
        & Parameter & Value \\
    \cline{1-2} \cline{4-5}
    No. Samples (Q) & 100 &
        & Threshold ($p_m$)   & 0.01 \\
    Max Evaluations   & 100 &
        & Init Solutions & 10 \\
    Epochs  & 1000  &
        & Dropout   & 0.5 \\
    \cline{1-2} \cline{4-5}
	\end{tabular} \\
    \label{table:optimization-params}
\end{table}

Note that the parameters presented in Table~\ref{table:search-spaces} and~\ref{table:optimization-params} were taken from~\cite{camero2019specialized,camero2019waste}. We decided to chose these values (instead of performing an hyperparameter tuning) to enable a direct comparison with our competitors.

\subsubsection{Sine Wave}
The range of the sine function is $[0,1]$, thus we set the activation function of the dense output layer to be a \texttt{tanh}. Due to the immense number of \emph{invalid} solutions, we implemented a \emph{limited} version of the infeasible solution listing, i.e., instead of enumerating all infeasible solutions, we list a subset of them. Particularly, we listed the infeasible solutions described by the min and max values of each parameter (i.e., the number of neurons per layer and look back). Thus, we added 80 infeasible solutions to the warm-start.

\begin{sidewaystable}
	\centering
    \caption{Sine optimization results (MAE of the best solution). Groups sharing a letter in the Conover row are not significantly different}        \label{table:sin-summary}
    \begin{tabular}{ lrrrrrrrrrrrr }
    \hline
    	& GDET	& MLES	& ---F	& --IF	& -W-F	& -WIF	& C--F	& CWIF	& ---S	& C—S	& ---P	& --IP \\
    \hline
    Mean	& 0.1419	& 0.1047	& 0.0785	& 0.0882	& 0.0816	& 0.1119	& 0.0839	& 0.1452	& 0.0857	& 0.0745	& 0.1198	& 0.1363 \\
    Median	& 0.1489	& 0.0996	& 0.0738	& 0.0882	& 0.0772	& 0.0861	& 0.0789	& 0.0935	& 0.0748	& 0.0721	& 0.1170	& 0.1244 \\
    Max	& 0.2695	& 0.2466	& 0.1172	& 0.1266	& 0.1185	& 0.3677	& 0.1276	& 0.5723	& 0.1794	& 0.0962	& 0.1700	& 0.3290 \\
    Min	& 0.0540	& 0.0631	& 0.0449	& 0.0505	& 0.0518	& 0.0492	& 0.0631	& 0.0577	& 0.0584	& 0.0525	& 0.0922	& 0.0665 \\
    Sd	& 0.0513	& 0.0350	& 0.0194	& 0.0182	& 0.0161	& 0.0695	& 0.0154	& 0.1367	& 0.0274	& 0.0109	& 0.0177	& 0.0558 \\

    \hline
    Conover & a & bc & \textbf{d} & ef & \textbf{d}e & bf & \textbf{d}ef & c & \textbf{d}ef & \textbf{d} & a & a \\
    \hline
	\end{tabular} \\

\bigskip

    \caption{Waste optimization results (MAE of the final solution). Groups sharing a letter in the Conover row are not significantly different}    \label{table:waste-summary}
    \begin{tabular}{ lrrrrrrrrrrrr }
    \hline
    	& Cities	& MLES	& ---F	& --IF	& -W-F	& -WIF	& C--F	& CWIF	& ---S	& C—S	& ---P	& --IP  \\
    \hline
    Mean	  & 0.0728	& 0.0790	& 0.0722	& 0.0821	& 0.0730	& 0.0812	& 0.0728	& 0.0728	& 0.0732	& 0.0725	& 0.0744	& 0.0735 \\
    Median	  & 0.0731	& 0.0728	& 0.0723	& 0.0735	& 0.0725	& 0.0730	& 0.0725	& 0.0725	& 0.0736	& 0.0723	& 0.0737	& 0.0731 \\
    Max	      & 0.0757	& 0.1377	& 0.0791	& 0.1227	& 0.0806	& 0.1231	& 0.0767	& 0.0767	& 0.0756	& 0.0760	& 0.0920	& 0.0883 \\
    Min	      & 0.0709	& 0.0691	& 0.0695	& 0.0691	& 0.0703	& 0.0698	& 0.0692	& 0.0701	& 0.0691	& 0.0688	& 0.0717	& 0.0701 \\
    Sd	      & 0.0012	& 0.0172	& 0.0019	& 0.0156	& 0.0020	& 0.0177	& 0.0018	& 0.0014	& 0.0015	& 0.0016	& 0.0041	& 0.0027 \\

    \hline
    Conover & \textbf{a}bc & \textbf{a}bc & \textbf{a} & bcd & \textbf{a}b & d & \textbf{a}b & \textbf{a}b & bcd & \textbf{a}b & cd & bcd \\
    \hline
	\end{tabular} \\
\bigskip

    \caption{Optimization results (MAPE of the final solution). Groups sharing a letter in the Conover row are not significantly different}     \label{table:eunite-summary}
    \begin{tabular}{ lrrrrrrrrrrrrr }
    \hline
    	& SVM	& RBF	& WK+	& ---F	& --IF	& -W-F	& -WIF	& C--F	& CWIF	& ---S	& C—S	& ---P	& --IP  \\
    \hline
    Mean	& 2.879	& NA	& NA	& 2.726	& 3.148	& 2.595	& 3.066	& 2.158	& 2.844	& 2.321	& 2.235	& 4.823	& 5.287 \\
    Median	& 2.945	& NA	& NA	& 2.466	& 2.933	& 2.368	& 2.814	& 2.099	& 2.846	& 2.125	& 2.050	& 5.040	& 5.213 \\
    Max	& 3.480	& NA	& NA	& 6.271	& 5.207	& 4.594	& 6.031	& 3.364	& 3.901	& 6.271	& 4.605	& 6.999	& 11.004 \\
    Min	& 1.950	& 1.481	& 1.323	& 1.840	& 1.759	& 1.593	& 1.919	& 1.452	& 2.033	& 1.654	& 1.657	& 3.142	& 3.415 \\
    Sd	& 0.004	& NA	& NA	& 0.888	& 1.013	& 0.765	& 1.000	& 0.440	& 0.515	& 0.774	& 0.564	& 0.884	& 1.727 \\

    \hline
    Conover & NA & NA & NA & abc & a & bc & a & \textbf{d} & ab & ce & \textbf{d}e & f & f \\
    \hline
	\end{tabular} \\

\end{sidewaystable}

Table~\ref{table:sin-summary} summarizes the results of the experiments, where MLES and GDET are the results presented in~\cite{camero2019specialized}, and the other results correspond to the tested combinations. Figure~\ref{figure:mae-sin} shows the distribution of the MAE of the solutions of the sine wave problem. The Friedman rank sum test $p$-value is less than 2.2e-16 (chi-squared = 138.17, df = 11). Therefore, we performed a pairwise comparison using the Conover test for a two-way balanced complete block design~\cite{conover1979multiple}, and the Holm $p$-value adjustment method. The results are presented in the row label Conover in Table~\ref{table:sin-summary}. Groups sharing a letter are not significantly different ($\alpha=0.01$).

The results show that using BO and MRS improves the performance of the final solution (error). On the other hand, multiple combinations of the proposed strategies (i.e., the combinations grouped by the letter \texttt{d}) show a similar performance.

\begin{figure}[ht]
  \centering
  \includegraphics[width=0.8\columnwidth]{mae-sin.png}
  \caption{MAE of the sine wave solutions}
  \label{figure:mae-sin}
\end{figure}

\subsubsection{Waste}
The filling level of the bins ranges from 0 to 1. Accordingly, we set the activation function of the output layer to be a \texttt{sigmoid}. In this case, we added 126976 invalid solutions to the warm start.

Table~\ref{table:waste-summary} summarizes the results of the tests on the waste problem. The table also includes the results of \cite{camero2019waste} (Cities) and \cite{camero2019specialized} (MLES). Figure~\ref{figure:mae-waste} shows the distribution of the MAE of the solutions of the waste problem. The Friedman rank sum test $p$-value is equal to 0.02401 (chi-squared = 22.048, df = 11). Therefore, we performed a pairwise comparison using the Conover test for a two-way balanced complete block design~, and the Holm $p$-value adjustment method. The results are presented in the row label Conover in Table~\ref{table:waste-summary}. Groups sharing a letter are not significantly different ($\alpha=0.01$).

In this case, our results are as good as our competitors (the results grouped by the letter \texttt{a}). Nonetheless, it is important to remark that~\cite{camero2019waste} (Cities) trains every candidate solution using Adam, turning out to be more time-consuming.

\begin{figure}[ht]
  \centering
  \includegraphics[width=0.8\columnwidth]{mae-waste.png}
  \caption{MAE waste}
  \label{figure:mae-waste}
\end{figure}

\subsubsection{Load Forecast}
According to the preprocessing performed in~\cite{lang2018short}, we normalized the data to have a mean equal to zero and a standard deviation equal to one. Then, we set the activation function of the output layer to be \texttt{linear}. Besides, we added 126976 invalid solutions to the warm start.

Table~\ref{table:waste-summary} summarizes our results and the ones presented in
\cite{chen2004load} (SVM), and \cite{lang2018short} (RBF and WK+, WKNNRW in the original work). In this case, we present the mean absolute percentage error (MAPE) because it is the performance metric used in the referred studies (\texttt{NA} indicates the corresponding data is not available). Figure~\ref{figure:mae-waste} shows the distribution of the MAPE of the solutions of the waste problem. Unfortunately, in this case, we do not have the detailed results of SVM, RBF, and WK+. Thus, we can not perform a detailed analysis considering all competitors. Nonetheless, we performed a detailed analysis considering exclusively the results of our tests. The Friedman rank sum test $p$-value is less than $2.2\times 10^{-16}$ (chi-squared = 146.38, df = 9). Therefore, we performed a pairwise comparison using the Conover test for a two-way balanced complete block design, and the Holm $p$-value adjustment method. The results are presented in the row label Conover in Table~\ref{table:eunite-summary}. Groups sharing a letter are not significantly different ($\alpha=0.01$).

\begin{figure}[ht]
  \centering
  \includegraphics[width=0.8\columnwidth]{mape-eunite.png}
  \caption{MAPE load forecast}
  \label{figure:mape-eunite}
\end{figure}

\subsubsection{Solutions Overview}

To get insights into the RNN architectures, we analyzed the (best) solutions. Figure~\ref{figure:arch-hl} shows the percentage of solutions that have a specific number of hidden layers (within the search space defined in Table~\ref{table:search-spaces}). Figure~\ref{figure:arch-lb} presents the percentage of solutions that have each of the possible look backs. Figure~\ref{figure:arch-nn} depicts the distribution of the total number of LSTM cells.

It is no surprise that the \emph{plain} encoding produced deeper and bigger (in terms of the total number of neurons) solutions, because of its own encoding limitations. On the other hand, two relatively similar combinations in terms of the error, namely \texttt{C--F} and \texttt{C--S}, present different architecture combinations.

Also, it is quite interesting that there is no clear \emph{architecture trend}. There are some value ranges that seem to be more suitable, e.g, shallower instead of deeper networks, or mid-to-upper look back values for the load forecast problem, but we can not conclude that there is an \emph{all-rounder} architecture.

\begin{figure}[ht]
  \centering
  \includegraphics[width=0.8\columnwidth]{arch-hl-problem-color.png}
  \caption{Number of hidden layers of the solutions}
  \label{figure:arch-hl}
\end{figure}

\begin{figure}[ht]
  \centering
  \includegraphics[width=0.8\columnwidth]{arch-lb-problem-color.png}
  \caption{Look back or time steps of the solutions}
  \label{figure:arch-lb}
\end{figure}

\begin{figure}[ht]
  \centering
  \includegraphics[width=0.8\columnwidth]{arch-nn-problem.png}
  \caption{Distribution of the total number of LSTM cells}
  \label{figure:arch-nn}
\end{figure}

\subsection{Time Analysis}

The results presented in this study (Table~\ref{table:waste-summary}) show that using MRS as a proxy of the performance is as good as using short training results. However, as it is claimed in~\cite{Camero2018lowcost}, MRS is supposed to be a low-cost approach. Therefore, we compared the run time of Adam against MRS. Specifically, we randomly select 16 runs from the previous experiments (i.e., 100 architectures evaluated in 16 runs, totaling 1600 RNNs). Then, for each network we performed a MRS (100 samples) and a 10 epochs training using Adam.

We repeated the experiments because of two reasons. First, the previous experiments were run on a cluster of heterogeneous computers (hence the run times were not fairly comparable). Secondly, the final solutions were trained for 1000 epochs, thus the comparison would not have been fair.

Table~\ref{table:time-comparison} summarizes the time in seconds for both approaches, and Figure~\ref{figure:time-violin} shows the distribution of the time (in seconds). We performed a Wilcoxon rank sum test to compare both approaches. Note that we compare the overall results and the results of each problem independently. The results are presented in the table (\emph{Signif.}) using the following codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1.

On average, MRS is 2.6 times faster than Adam. These results are in line with the ones presented in~\cite{camero2019specialized}. In other words, if we have used the results of 10 epochs training using Adam to compare the architectures during the optimization process (instead of MRS), we will have spent more than twice the time!

\begin{table}
    \caption{Time comparison in seconds: Adam vs MRS. According to the Wilcoxon rank sum test, there is a significant improvement}
	\centering
    \begin{tabular}{ llrrrr }
    \hline
    & [seconds] & Load	& Sin	& Waste & Overall \\
    \hline
    \multirow{5}{*}{Adam}
    & Mean   &  72.1 &  41.8 &  45.3 &  53.1 \\
    & Median &  62.6 &  32.3 &  29.1 &  34.1 \\
    & Max    & 220.9 & 105.8 & 172.3 & 220.9 \\
    & Min    &  21.9 &  23.0 &   7.8 &   7.8 \\
    & Sd     &  48.7 &  19.0 &  40.8 &  40.5 \\
    \hline
    \multirow{5}{*}{MRS}
    & Mean   &  13.8 &  27.9 &  19.3 &  20.3 \\
    & Median &  11.7 &  23.9 &  13.8 &  20.0 \\
    & Max    &  25.3 &  56.8 &  61.6 &  61.6 \\
    & Min    &  10.8 &  20.4 &  10.9 &  10.8 \\
    & Sd     &   4.3 &   8.0 &  10.7 &  10.0 \\
    \hline
    \multicolumn{2}{l}{Signif. (Adam vs MRS)} &  *** & *** & *** & *** \\
    \hline
	\end{tabular} \\
    \label{table:time-comparison}
\end{table}

\begin{figure}[ht]
  \centering
  \includegraphics[width=0.8\columnwidth]{time-violin-color.png}
  \caption{Time comparison: Adam (10 epochs) vs MRS (100 samples)}
  \label{figure:time-violin}
\end{figure}

\subsection{Error Trade-off}

Moreover, we studied how much the outcome of MRS is affected (i.e., error of the final solution) when the number of samples is changed. We repeated the \emph{waste} and \emph{load forecast} experiments using the \texttt{C--S} configuration, and 30, 50, and 200 samples per each solution evaluated (MRS).

Table~\ref{table:tradeoff} summarizes the error trade-off results. The Friedman rank sum test $p$-value is equal to 0.004996 (chi-squared = 12.84, df = 3) in the waste problem, while it is equal to 0.0003184 (chi-squared = 18.68, df = 3) in the load forecast problem. Therefore, we performed a pairwise comparison using the Conover test for a two-way balanced complete block design~\cite{conover1979multiple}, and the Holm $p$-value adjustment method. The results are presented in the row Conover of both tables. Groups sharing a letter are not significantly different ($\alpha=0.01$).

The results show that we might reduce the time (by taking fewer samples) but with an error increase. On the other hand, doubling the number of samples (used in this study), we will have not reduced the error. Nonetheless, it is quite interesting that even with a small number of samples, lets say 30, it is possible to estimate the performance of a network.

\begin{table}
    \caption{Waste and Load trade-off results. Groups sharing a letter in the Conover row are not significantly different}
	\centering
    \begin{tabular}{ clrrrr }
    \hline
    & Samples	& 30	& 50	& 100	& 200 \\
    \hline
    \multirow{5}{1.5cm}{\centering Waste (MAE)}
    & Mean	& 0.0734	& 0.0734	& 0.0725	& 0.0723 \\
    & Median	& 0.0732	& 0.0740	& 0.0723	& 0.0726 \\
    & Max	& 0.0778	& 0.0780	& 0.0760	& 0.0757 \\
    & Min	& 0.0694	& 0.0690	& 0.0688	& 0.0685 \\
    & Sd	& 0.0017	& 0.0020	& 0.0016	& 0.0018 \\

    \hline
    \multirow{5}{1.5cm}{\centering Load (MAPE)}
    & Mean	    & 2.664	& 2.616	& 2.235	& 2.137 \\
    & Median	& 2.510	& 2.555	& 2.050	& 2.073 \\
    & Max   	& 4.436	& 3.750	& 4.605	& 3.146 \\
    & Min	    & 1.930	& 1.884	& 1.657	& 1.521 \\
    & Std	    & 0.597	& 0.492	& 0.564	& 0.405 \\

    \hline
    Conover & & a & a & b & b \\
    \hline
	\end{tabular} \\
    \label{table:tradeoff}
\end{table}

\subsection{Algorithm Convergence}
Finally, we studied the convergence of the proposed algorithm. Particularly, we analyzed the fitness (probability estimated by the MRS) of the solutions as the search was done. Figure~\ref{figure:waste-convergence} and~\ref{figure:eunite-convergence} depict the best-so-far MRS value against the number of candidates evaluated, average over all independent runs for each combination of encoding, warm-start, and constraint handling methods (shown by the bold line).
Also, the standard deviation is illustrated by the shaded areas. It is important to point out that a higher MRS value is correlated with a better performance after training the network using Adam~\cite{Camero2018lowcost}, hence indicating that all combinations are converging.
\begin{figure}[ht]
  \centering
  \includegraphics[width=0.9\columnwidth]{waste-convergence-sd.png}
  \caption{Average convergence of the fitness of the solutions for the waste problem}
  \label{figure:waste-convergence}
\end{figure}
\begin{figure}[ht]
  \centering
  \includegraphics[width=0.9\columnwidth]{eunite-convergence-sd.png}
  \caption{Average convergence of the fitness of the solutions for the load forecast problem}
  \label{figure:eunite-convergence}
\end{figure}

Moreover, to show the impact of the penalty function, we compared the pairs \texttt{C--S} and \texttt{---S}, \texttt{C--F} and \texttt{---F}. Notice that the results present the average value of the MRS and the standard deviation (shaded area) for 30 independent runs (each combination) in the waste prediction problem. Therefore, we assume that the difference in performance (i.e., the convergence) can be explained by the the penalty.

\begin{figure}[!h]
  \centering
  \subfigure[]{
  	\includegraphics[width=0.45\textwidth]{waste-convergence-sd-flag.png}}
  \subfigure[]{
  	\includegraphics[width=0.45\textwidth]{waste-convergence-sd-size.png}}
  \caption{Impact of the penalty on the average convergence (waste problem)}
  \label{figure:sine-logp-mae-1-2}
\end{figure}

\section{Conclusions and Future Work}
\label{sec:conclusions}

In this study, we propose to optimize the architecture of a recurrent neural network with a combination of Bayesian optimization and Mean Absolute Error Random Sampling (MRS). More specifically, we propose three fixed-length encoding schemes to represent variable size architectures (\emph{flag}, \emph{plain}, and \emph{size}), an alternative to deal with the many-to-one problem derived from the fixed-variable-length problem (i.e., the \emph{infeasible}solution), and two strategies to cope with the fixed-variable-length problem, namely \emph{warm-start} and \emph{constraints handling}.

We test our proposal on three prediction problems: the sine wave, the waste filling level of 217 bins in a metropolitan area of a city in Spain, and the maximum daily load forecast of an electricity company in Slovakia. We benchmark our proposal against state-of-the-art techniques, and we performed a time comparison and an error trade-off study. Notice that for each problem a different activation function has been used, namely, \texttt{tanh}, \texttt{sigmoid}, and \texttt{linear}.

The results show that none of the strategies presented outperforms the others in all cases. Nonetheless, using the \emph{size} encoding and the \emph{constraints handling} consistently show to be an \emph{effective} alternative to the problem.

Moreover, the results show that MRS is an \emph{efficient} alternative to optimize the architecture of an RNN. Particularly, we showed that evaluating an architecture using MRS is 2.6 times faster than performing a short training (ten epochs) using Adam, and without losing performance.

Overall, using BO, in combination with MRS, shows to be a competitive approach to optimize the architecture of an RNN. It offers a state-of-the-art error performance, while the time is drastically reduced.

Finally, for the next step, several issues have to be addressed. First, it is necessary to test on more data sets to validate the proposal. Second, MRS has to be further researched because it shows to be a promising alternative, but there is no clear explanation of why it works. Additionally, it will be interesting to use the \emph{warm start} strategy to explore \emph{augmenting restarts}, i.e., iteratively increase the number of hidden layers and feeding the model with the previous results.

\subsection*{Acknowledgments}

This work was supported in part by Universidad de M\'alaga, Andaluc\'ia Tech, Consejer\'ia de Econom\'ia y Conocimiento de la Junta de Andalu\'ia, Ministerio de Econom\'ia, Industria y Competitividad, Gobierno de Espa\~na, and European Regional Development Fund grant numbers TIN2017-88213-R (6city.lcc.uma.es), RTC-2017-6714-5 (ecoiot.lcc.uma.es), and UMA18-FEDERJA-003 (Precog). And by the Helmholtz Association's Initiative and Networking Fund (INF) under the Helmholtz AI platform grant agreement (ID ZT-I-PF-5-1).

\balance

\bibliographystyle{elsarticle-harv}

redpajama_set_name: RedPajamaArXiv


text:
Викто́рия (символ U+270C ✌ «victory hand» в Unicode) — распространённый жест, означающий победу или мир.

Знак «Победа» показывается указательным и средним пальцами руки, направленными вверх в форме латинской буквы «V», на чём-либо изображают (рисуют) в виде V.

В Великобритании (там «приветствие победы» распространилось благодаря Уинстону Черчиллю) и Австралии этот знак приобретает оскорбительное значение, если кисть повернута тыльной стороной к человеку, к которому обращён жест.

Грубая версия жеста
В современных англо-говорящих государствах и странах похожий жест, демонстрируемый внешней стороной кисти к зрителю, является грубым жестом выражения презрения. «Средневековое» происхождение этого жеста якобы привязано к временам Столетней войны: пленённым английским и валлийским лучникам, наводившим трепет на французов, отрубали именно эти два пальца на правой руке, дабы они не могли в дальнейшем держать лук. Лучники, зная об этом, перед битвой дразнили французов, показывая им неповреждённые пальцы — «Бойтесь, враги!».

Вторая мировая война
Во время Второй мировой войны этот жест служил символом победы над оккупантами (оккупанты ненавидели этот знак и наказывали тех, кто его рисовал на стенах). Предложил его в июле 1941 года , бывший министр юстиции Бельгии и руководитель бельгийской секции вещания Би-би-си. «V» — первая буква слов vrijheid, victoire и victory. В голландском, французском и английском эти слова означают «победа». Знак в скором времени перебрался во Францию. Его код появился и в азбуке Морзе • • • —. Его звучание было похоже на начало Пятой симфонии Бетховена. 19 июля 1941 года Уинстон Черчилль одобрительно отозвался об этой идее в своей речи и с тех пор стал изображать знак V пальцами.

Примечания

Жесты
Символы мира

redpajama_set_name: RedPajamaWikipedia


text:
In 1985, while playing on what had been his grandfather's favourite golf course, award-winning Canadian broadcaster and journalist Peter Gzowski had an idea…Why couldn't he organize a fun-filled golf tournament for a few friends and perhaps raise a dollar or two for adult literacy at the same time? A year later, his 'little fundraising tournament' evolved into the Peter Gzowski Invitational (PGI), a national series of locally produced golf tournaments featuring musicians, media and literacy celebrities.
Held annually in every province and territory, over 50,000 people have taken part in PGIs across the country since 1986. In the process, more than $13.9 million has been raised for volunteer-based literacy programs. These funds directly support literacy initiatives in the provinces and territories where the PGIs are held.. The Literacy Coalition of New Brunswick has raised more than $1.6 million dollars for literacy since our inaugural PGI event 15 years ago. Funds raised go directly to youth, adult and family literacy programs around the province, including support for our schools, libraries and community groups.

redpajama_set_name: RedPajamaCommonCrawl


text:
Ceserana ist ein Ortsteil (Fraktion, italienisch frazione) der italienischen Gemeinde Fosciandora in der Provinz Lucca in der Toskana.

Geografie
Der Ort liegt etwa ein Kilometer nördlich des Hauptortes Fosciandora, etwa 30 km nördlich der Provinzhauptstadt Lucca und etwa 75 km nordwestlich der Regionalhauptstadt Florenz. Der Ort liegt in der Landschaft der Garfagnana im oberen Serchiotal etwa 1 km östlich am linken Ufer des Serchio bei  und hatte 2001 158 Einwohner. 2011 waren es 143 Einwohner. Nächstgelegener Ort neben Fosciandora ist Castelnuovo di Garfagnana, etwa 4 km westlich gelegen.

Geschichte
Der Ort wurde 996 als Finibus Cicerani von Papst Johannes XV. oder Papst Gregor V. den Ronaldinghi zugewiesen. Durch diese Verbindung blieb der Ort in mehreren Konflikten zwischen Lucca und Pisa auf der päpstlichen Seite der Pisaner. 1170 wurde der Ort von Lucca angegriffen und erlitt schwere Schäden an der Burg und den Wehrmauern. Zwei Jahre später trat man auf Grund des Druckes aus der Lega Pisana aus und begab sich unter die Herrschaft der Republik Lucca. Von 1185 bis 1197 unterstand der Ort Barbarossa und dann seinem Sohn, kam dann zu Lucca zurück und begab sich 1227 erneut unter päpstliche Herrschaft. Die Konflikte um die Herrschaft in der Garfagnana entschied Lucca für sich und teilte Ceserana 1272 ins Vicaria di Castiglione ein. Aufgrund der weiteren Konflikte um die Vorherrschaft in der Garfagnana bat der Ort 1451 um Schutz bei den d'Este aus Modena, bei denen sie, mit Ausnahme der Zeit der Napoleonischen Besetzung, bis zur Einheit Italiens im 19. Jahrhundert verblieben.

Sehenswürdigkeiten

 Rocca di Ceserana (auch Rocca Estense), Burgruine oberhalb des Ortes, die um das 12. Jahrhundert entstand und 1309 als Ciciorana Castello dokumentiert wurde. Mit der Übernahme der d'Este im 15. Jahrhundert wurde die Burg bis 1474 und 1616 umgebaut und verstärkt.
 Sant'Andrea, Kirche in der Pfarrei Sant'Andrea e Santa Maria. Die Kirche gehört zum Erzbistum Lucca und befindet sich innerhalb der Wehrmauern der ehemaligen Burg. Erstmals erwähnt wurde sie im 10. Jahrhundert als den Gütern des Heiligen Stuhl zugehörig. In der heutigen Form entstand die Kirche im 12. Jahrhundert als Teil der Burg. Erneute Erwähnung fand sie 1168 in einem Dokument von Papst Alexander III., die Taufrechte bekam sie von Papst Urban VI. 1388. Die Kirche wurde im 17. Jahrhundert vergrößert und 1844/1845 sowie 1969 restauriert. Der heutige Campanile entstand aus einem der  Wehrtürme der alten Festung, die Fassade und die Apsis sind noch heute größtenteils in der Originalform. Im Inneren enthält die Kirche zwei Holzstatuen (San Giacomo und Sant'Andrea), die am Ende des 14. Jahrhunderts entstanden.

Verkehr

 Der Haltepunkt Fosciandora-Ceserana liegt an der Bahnstrecke Aulla Lunigiana–Lucca, etwa 1 km westlich von Ceserana.
 Die Staatsstraße Strada statale 445 della Garfagnana verläuft etwa 1 km westlich von Ceserana.

Literatur
 Emanuele Repetti: CESARANA o CESERANA (Cicerana) nella valle superiore del Serchio. In: Dizionario Geografico Fisico Storico della Toscana (1833–1846). Onlineausgabe der Universität Siena (PDF, italienisch)

Weblinks

 italia.indettaglio.it zu Ceserana
 Contado Lucchese zu den Ortsteilen von Fosciandora (Ceserana)
 Bacino Culturale Valle del Serchio – Sistema delle Rocche e delle fortificazioni: Rocca di Ceserana

Einzelnachweise

Fraktion in der Toskana
Fosciandora


text:
Game-day security to 'meet needs of people'
By Bill Schwingel
The presence of game-day security might not blend in well with blue skies, trees blowing in the wind and students enjoying football games, but to one NIU administrator, the mix makes for a perfect day of gridiron fun.
NIU Athletic Director Gerald O'Dell said the most important goal of security at the football games is "how to best meet the needs of people at the event."
No security problems came up at last week's game against Cal State-Fullerton, said James Webster, who is in charge of football security.
Security last weekend consisted of 28 patrol officers and seven security guards from the university police force, and four officers from other area police departments, Webster said.
The number of security personnel varies, depending on holidays and availability of officers, he said.
Most of the behavior problems at the football games stem from people getting "slightly drunk," Webster said, adding that many people felt the football game was just an "open-air tavern."
When this atmosphere was taken away, people interested only in drinking no longer came to the football games, he said.
After a "riot" a couple years ago when students started throwing corn cobs and beer bottles, the university made changes in the alcohol policies at the football games, Webster said.
The changes prohibited any mass consumption of alcohol and outlawed kegs and bottles at the tailgate gatherings.
The picnic area, where people gather before the games, can no longer be used as a playing field for Frisbee or football because of the possibility of injuries to on-lookers, he said.
The athletic department said it was going to supply additional security to aid the university police and new private security services, said acting Vice President for Student Affairs Barbara Henley.
The athletic department decided to hire a new security service because the department did not feel it was getting adequate service, O'Dell said. "It was time for a change."
In previous years, a "peer group" from University Planning and Activities assisted the security guards at the games, but this group was used mainly in the designated tailgating areas, said O'Dell.
The peer group will not be continued this year because there is no designated tailgating area, he said.
"I would be extremely disappointed if individuals or groups of individuals were disruptive in any way of making a very positive environment," O'Dell said.


brochure pamphlet template booklet vectors photos and files free download tri fold photoshop elements.
fundraising sheet template sponsor examples free church pledge balance tem.
store visit checklist template schedule retail evaluation survey images of download cleaning templa.
square business card size template cards moo inspirational guidelines artwork templates for resu.
working hours policy template tools for staff work uniform training flexible arrangements.
one page brochure template free spa fold templates examples 2 download.
vehicle use policy template related post acceptable usage and procedure tracking uk.
joomla social network template networking tachyon premium free download.
24 7 schedule template rotating shift manual download hour free.
service review report template management nh.


Mulled wine, twinkling lights, street artists and merry shopkeepers: join us for late-night Christmas shopping in Broadway.
Friday 29th November and Friday 6th December: 5.30-8.30pm.
Cotswold Trading for gifts and more.
Landmark for quality kit for the great outdoors.
Rikki Tikki toy shop for indoor, outdoor, traditional and modern children's toys.
Slate Clothing for a beautiful collection selected from top designers.
Sue Parkinson for designer homeware plus.
Visit Broadway for a full list of shops and Broadway Cotswolds for further information on late night shopping, and more.
Enjoy the festive period in any way you choose. A chilled bottle of champagne and a warm welcome await your arrival.
Check availability and book using the small calendar in this right hand margin or using the larger calendar on this page.
You can also email Jacki or call her on +44 (0) 1386 853769.
Sheepscombe Byre, Snowshill, Broadway, Worcestershire, WR12 7JU.
We regret that for the health and wellbeing of our future guests, we cannot allow pets or smoking.
Learn more about our Privacy and Cookies Policy.
© 2016 Jacki Harrison and Grounded Solutions Ltd.


text:
NOTES AND QUOTES – LIVE FROM THE MASTERS (MONDAY EVENING)
"Jordan Spieth may not have the firepower of Tiger Woods, but in the last 25 years, he is the player that appears, to me, to have the brainpower of Tiger Woods." – Brandel Chamblee
"Unquestionably the best major championship player of the last four years." – Rich Lerner on Brooks Koepka
"When people doubt him, as I think we kind of are right now because of his health, that usually spells success for Brooks Koepka." – Justin Leonard
Live From the Masters Continues Tomorrow Morning at 9 a.m. ET and in Primetime at 7 p.m. ET on GOLF Channel
STAMFORD, Conn. – April 5, 2021 – NBC Sports began its Live From the Masters coverage from Augusta National Golf Club in Augusta, Ga., today and in primetime tonight on GOLF Channel.
Throughout the week, GOLF Channel's daily live coverage features more than 55 hours and includes pre- and post-tournament coverage through the final round on Sunday, April 11.
In advance of the 2021 Masters at Augusta National Golf Club, NBC Sports Golf's lead studio team of Rich Lerner, Brandel Chamblee and Justin Leonard held a media conference call last week to preview the tournament. Click here for the transcript.
On Jordan Spieth
Brandel Chamblee: "He was so often, over the last three years, getting stuck and missing all over the place — I think (it was) that little technical problem that he was trying to get over. But also when he said, 'I took ownership' that is, to me, to a golfer what the La Brea Tar Pits are to a paleontologist looking for a woolly mammoth."
Chamblee: "Jordan Spieth may not have the firepower of Tiger Woods, but in the last 25 years, he is the player that appears, to me, to have the brainpower of Tiger Woods, and digging himself out of it, taking ownership of his game, is the larger piece of the puzzle here."
Rich Lerner: "Jordan has a make-it-happen gear. A run-it-in, roll-it-in, chip-it-in gear, where he can just pile up the birdies and eagles in a way that kind of blows your mind."
Chamblee: "This guy has the ability to make magic. And that's what makes him so fun to watch."
Lerner: "It's a mistake to doubt the will of a great athlete, and he is a great athlete."
On Brooks Koepka
Justin Leonard: "When people doubt him, as I think we kind of are right now because of his health, that usually spells success for Brooks Koepka. He loves proving people wrong; he's not afraid to talk about it, and more times than not, he's backed it up."
Lerner: "Unquestionably the best major championship player of the last four years."
Chamblee: "He has everything necessary that you need at this place, just capable of ridiculously big shots. In the last three, four years we've seen him at major championships, (he hits) jaw-dropping, laser bullets at the heat of the moment in major championships."
On the 2021 Masters Tournament
Leonard: "Normally there's a nine-month buildup to the Masters Tournament. Just four-and-a-half months ago (this year)…being that short between competitions of this event, you might think that would lend to a sense of compliancy by the players and patrons. But, after successful competitions and completions of the Augusta National Women's Amateur and Drive, Chip and Putt, to me its very evident that there's an excitement around this Masters Tournament that only the Masters can provide."
Chamblee: "I can't think of another place in sport that we go to year after year with elevated expectations, and then are surprised, year after year, after year. There's just something about this place."


text:
DFAT COMICS
Towelite Talk
Star Warriors
Those Geeks You Know
Gourmet Scum Radio
Gutsy Media
GEEKLY NEWS
Towelite Zone
FRIENDS OF DFAT
It Crept from the 80s
Ruben Romero
Critical Mass Podcast
Nerds From the Crypt
Pete's Basement
Popanimecomics
Marvel Studios Phase 2 Updates: Captain America: The Winter Solider, Ant-Man, Agents of S.H.I.E.L.D.
by Chris Walk | Jan 7, 2014 | Movies | 0 comments
Towelites! It's time once again to round up all the latest news and rumors surrounding Marvel Studios' Phase 2!
The beautiful thing about all of the Marvel Studios films is that they all have a different tone and feel about them, and that's one of the best aspects of Captain America is that since it takes place in different time periods you can totally treat it differently than the first film. The Winter Soldier looks to be an action packed spy thriller that will pit Cap and SHIELD against a shadowy figure from his past. Co-writer Stephen McFeely had the following to say about the film in an interview with Empire Magazine:
[box_light]"We knew we were going to make the first film a period piece; that really appealed to us. It was the only way to make a Captain America movie that would not come off as ridiculous. A man does not decide to put on an American flag outfit in 2011, but he might in World War II. How do we make sense of him in the modern world? I mean, he's really Gary Cooper. The solution is kind of that the world changes in response to him. He is usually correct. In this one, we're dealing with the Cap after Marvel thawed him out in the '60s, the one whose values don't necessarily match ours. We went down a bunch of roads and kept coming back to this idea of a conspiracy movie," he adds. "It's what's going to get the most out of the moral dilemma for the character. How does he fit in? How does he see where we've ended up? He hasn't experienced everything we've gone through to get to this point, where agendas are now really murky." [/box_light]
McFeely then added the following in regards to the Winter Soldier being the perfect villain for the film:
[box_light]"a product of dubious, corrupt practices that don't make you feel good when you drag them into the light, everything kept pointing to something like Three Days Of The Condor." [/box_light]
Marvel president Kevin Feige chimed in:
[box_light]"In our attempt to make all of our films feel unique and feel different we found ourselves going back to things like Condor. Also the other political thrillers of the '70s: The Parallax View, All The President's Men. This was a time that Cap existed in the comics. He found himself in the swinging '60s followed by the Watergate era followed by the Reagan era followed by where we are today. In the comics it was a hell of a journey for Steve, And we couldn't take him through those years because in our cinematic universe he was asleep, but we wanted to force him to confront that kind of moral conundrum, something with that '70s flavour. And in our film that takes the form of S.H.I.E.L.D.." [/box_light]
Edgar Wright has kept the identity of his Ant-Man a mystery even after Paul Rudd was cast in the role. On his official site Wright shared the following picture on his official website from the awesome animated series Earth's Mightiest Heroes' episode "To Steal an Ant-Man". In this episode Scott Lang steals the Ant-Man suit from Hank Pym's lab and uses it for his own needs.
So does this "homework" mean that Scott Lang will be the leading character but Hank Pym will be recognized? Maybe Pym will be in Avengers and then it will lead to Ant-Man. Who knows, I think that the Scott Lang story would make a good film but without the use of Pym there will be a lot of story and cannon changes. Not that Marvel Studios hasn't done this already, whatever the case may be I will be interesting to see a B-Lister headline his own film and see how the general public accepts this. Ant-Man hits theaters in July of 2015!
After the mid-season break of Agents of SHIELD we are hoping that the show will return with a bang, at least for the writers and producers sake. I do really enjoy this show I really am hoping that they will be building up to something that is Marvel Universe cohesive, but what has been shown so far has been lukewarm at best. Well Agent Coulson himself , Clark Gregg had he following to say in an interview with Zap2it about what is coming next for our favorite agents:
[box_light]"The fact that they've already managed to kind of make the reveal of what Coulson's doing alive [be] the one thing that the Clairvoyant wants is just part of the way the back half of this season is going to be — more exciting and more than the first half of this season, where so much work had to be done to kind of set up this world of these people and this team, and all these separate pieces get kind of woven together. "People have been patient in really wanting to know the true nature of the secrets. … We're going to get a big fat window at the real truth, and it's going to be very, very surprising, when I read the sequence, I just stopped and put the script down and went 'Wow. That was worth waiting for.'"[/box_light]
Agents of SHIELD returns tonight at 8pmEST on ABC!
~CynicNerd
SHARE IT!!
© 2020 Don't Forget a Towel™ Established 2012. Towelite Talk™, DFAT Comics™, Pursuit of Plastic™, Tuesday Blusday™ Weekly One-Up™, Crowd$ourcery™ & Geekly ™ All rights reserved.


It's a big day for Moore's Law. I'm not sure anyone else has noticed this, but by my calculations we have in the past few months reached the penny-per-MIPS* milestone. Intel's Core Duo running at 2.13 GHz now costs around $200 at retail (it's around $180 at volume), but can do about 20,000 MIPS. I remember my first 6 MHz 286 PC in 1982 that did 0.9 MIPS. I have no idea what the CPU cost then, but the PC it came in cost nearly $3,000 so it couldn't have been cheap. Say it was around $1,000/MIPS back then. Now it's $0.01/MIPS. I know I shouldn't be astounded by Moore's Law anymore, but that really is something.
I begin my economics of abundance speech with Carver Mead's mind-bending question: "What happens when things get (nearly) free?" His answer is that you waste them, be they transistors or megabytes of bandwidth capacity. You use them profligately, extravagantly, irresponsibly. You shift out of conservation mode and get into exploitation mode. You do crazy things like offering people the ability to put their whole music collection in their pocket, or promising the average email user that they'll never have to delete another message to conserve space. Just as Alan Kay "wasted" transistors to create the graphic user interface, we will all learn how to waste newly abundant resources, retraining our minds to ignore our instincts about costs and scarcity.
Today we have an unprecedented number of resources that are closing in on free when measured in units that were once meaningful to regular folks. Through the 1950s and 1960s Mead watched transistors drop from $100 each to $10, then $1, then $0.10, then a penny. Then, in the 1970s as transistors were integrated into semiconductor chips, they fell to a millicent and then a microcent. They're now nearly down to a nanocent–virtually free. Hard drives now go for about 30 cents per gigabyte, or .03 cents per megabyte (I remember my first 10-megabyte drive, which cost me a few weeks salary at the time). Bandwidth now costs less than ten cents per gigabyte at retail, and it wouldn't surprise me to hear that it's fallen below the penny-per-gigabyte level for big commercial outfits. How long would it have taken you to download a gigabyte of data in the old dial-up days, if you could even keep a connection open that long?
With apologies to Levitt and Dubner, I'll cheekily call the emerging realization that abundance is driving our world "freeconomics". Understanding when to shift out of scarcity mode and start giving away what you once held dear is a core competency for our age. Heck, there might even be a book in it!


2400 SF of move in ready space with central air, radiant heat, private office space, retail or receptionist space, and more. This charming building is located in the heart of downtown Delta, CO. It is perfect for retail or professional use. Have you thought about starting your own business or want to expand? This is the location to do so, this property is for sale or lease, call for more information.


Summary: Market volatility continues to see muted contagion into FX as yesterday's chunky rally on Wall Street failed to elicit much more than a head nod. Too many conflicting narratives and a frozen CNY rate are the likely culprits here, but the pressure on the euro is beginning to stand out a bit more in today's session.
Headline exhaustion and further headline risk on Italy's budget and Brexit may be making traders hesitant to commit to a position in the euro or sterling, respectively until we get more clarity in either direction, a prospect that feels weeks away at best. Also weighing on global markets is whether China will choose to abandon the CNY floor.
Italian BTPs are stuck in the middle of the local range after the EU yesterday asked Italy to resubmit its 2019 budget, an unprecedented stop. Italy's Finance Minister Tria was out this morning making conciliatory noises, but the Lega's Salvini and FSM's Di Maio call all of the shots here and are not likely to back down one centimeter. Their confrontational stance on the budget has seen their aggregate popularity in the polls rise from just below the 50% level at the election to well north of 60% – mostly down to a doubling of the Lega's popularity, while FSM has only faded slightly. A recent poll suggests that a majority of Italians now are in favour of leaving the EU, even if a stronger majority are in favour of keeping the euro. Could the populists begin to dangle an election to solidify their majority with the embedded promise of an "Italexit" referendum threat if the EU refuses to back down?
EU existential risks remain front-burner through the weekend as the Hesse state election in Germany could prove a precursor to a Merkel exit at the CDU party powwow in December, as well as the risk that the collapsing SPD pulls out of the grand coalition to avoid further carnage and leaking of voters to the Greens and generally leaving Germany's political scene adrift at least until the next election in 2021.
This morning's Riksbank meeting disappointed with its cautious tone and the lack of a commitment to a move in December, as the "December of February" language was retained and the CPI forecasts for 2019 and 2020 were kept unchanged despite some recent hotter inflation data. For some reason, 2019 growth expectations were revised down to 1.9% from 2.0%. In short, this meeting has so far not pressed the hawkish buttons the market was looking for and EURSEK could get lost in limbo again above 9.30 unless Governor Ingves pulls something out of his hat at the press conference.
In EM, watch USDBRL ahead of and after this weekend, which should see the right populist Bolsonaro, seen as a Trump-like figure (or Duterte-like), winning the second round vote. USDBRL is approaching the 200-day moving average and enthusiasm for BRL is rather remarkable given the wobbly global backdrop and concerns on China, etc. Brazil is a major commodity exporter to China, with some 19% of its exports China-bound in 2016. Brazil's credit spreads have improved in line with the BRL rally, but a further strengthening will require a President Bolsonaro to discipline the fiscal outlays linked to Brazil's overgenerous pension system.
Also on the EM agenda is South Africa and the 14.00 level in USDZAR as the market looks for signs that the South African government is showing the required fiscal prudence to continue to shore up the ZAR's prospects when Finance Minister Mboweni unveils a mid-term budget policy statement this afternoon.
EU existential risks and a new global risk-off move would likely favour EURJPY downside, a prospect we have visited many times before, but the two-way headline risk and recent rise in bond yields have kept the action very choppy. Still, as long as risk appetite remains nervous and the Italy budget stand-off continues – which we feel it will – the downside risk appears greater. The next area of interest lower ist he big 125.00 zone where the action was twice supported earlier this year.
USD – holding the greenback back are positioning and weak risk appetite finally starting to erode Fed policy expectations, but on the other hand, the bid dollar's endless liquidity is still attractive if markets remain in a funk.
EUR – EURUSD testing the 1.1432 lows this morning after a couple of weak flash October manufacturing PMI's from France and Germany (not yet seen the Euro Zone release yet) and on the uncertainty linked to further existential pain.
JPY – JPY looks firm against European currencies, but the only manages to pick up strength versus the greenback when "risk off" heats up sufficiently to see safe haven seeking in government bonds – keeping the BoJ's YCC policy in the background.
GBP – EURGBP hovering right at the 200-day moving average as dual headline risk a constant threat from both sides of the exchange rate. Discord from within May's own cabinet and mutiny from other Tory party members the chief risks du jour.
CHF – EU existential pain could extend the upside for the franc here, particularly if BTP yields start to threaten the highs again. A failure through 1.1400 in EURCHF looks technically significant.
AUD – the Aussie in nowhere land – look at the Australian banks and big mining stocks – the biggest driver of a massive 10%+ correction in Australian equities from multi-year highs just a few months ago. But AUD may remain stuck in the mud until we see a pulse in the USDCNY.
CAD – an ugly sell-off in oil not yet seeing spillover into CAD, where the market awaits a likely hike from the Bank of Canada today. The loonie will likely prove vulnerable if Poloz and company fail to put a fairly hawkish spin on forward expectations. Technically, CAD bears will want USDCAD to remain above the 1.3000-1.3050 pivot zone.
NZD – ditto for comments on AUD, though with a bit of extra energy if the AUDNZD level rises back above 1.0850 – suggesting the recent downside break was a red herring. New Zealand reports its trade balance late today, and the country's large and recently widening current account deficit should be a strong concern if we dip back into another cycle of risk off.
SEK – a real disappointment to the market's lean from the Riksbank today after a very hot September inflation report from Sweden, nevertheless this may just mean more limbo within recent ranges rather than a decided SEK sell-off of any note.
NOK – Norges Bank likely to recycle their previous stance tomorrow and the heavy sell-off in oil prices will offer little support for NOK. Dare we ask the question now: if EU existential risks get bad enough, wouldn't that at some point prove NOK positive? Think back to 2011-12 and how well SEK and NOK performed versus the Euro. Shortly put, in the big picture the EURNOK direction risk looks pointed firmly lower.


text:
Q: How do I build the Hub & Spoke setup when VMs cannot be fronted by Application Gateways outside their virtual network? Ported from Stackoverflow
I am building a Hub & Spoke network topology - described here - as recommended in the Azure docs

However when it comes to create the VM in the separate VNet, I get the following when trying to link it to the Application Gateway in the Hub Virtual Network ("...Application Gateway must be in the same virtual network...") ...

I have peered the Hub and Spoke virtual networks together using a peering at either end. Am I missing something here?

A: I would approach it differently, created a listerner in the app gateway for the ip of the vm which you want to protect with a app gateway. And add a public ip to the VM.
https://docs.microsoft.com/en-us/azure/application-gateway/configuration-listeners https://docs.microsoft.com/en-us/azure/application-gateway/create-ssl-portal
Also allow the following in the NSG of the VM:

You must allow incoming Internet traffic on TCP ports 65503-65534 for
the Application Gateway v1 SKU, and TCP ports 65200-65535 for the v2
SKU with the destination subnet as Any and source as GatewayManager
service tag.

*

*Back end server pool: The list of IP addresses of the back end
servers. The IP addresses listed should either belong to the VNet
subnet, or should be a public IP/VIP.

*Back end server pool settings: Every pool has settings like port,
protocol, and cookie based affinity. These settings are tied to a
pool and are applied to all servers within the pool. Front end Port:
This port is the public port opened on the application gateway.
Traffic hits this port, and then gets redirected to one of the back
end servers.

*Listener: The listener has a front end port, a protocol (Http or
Https, these are case-sensitive), and the SSL certificate name (if
configuring SSL offload).

*Rule: The rule binds the listener and the back end server pool and
defines which back end server pool the traffic should be directed to
when it hits a particular listener. Currently, only the basic rule
is supported. The basic rule is round-robin load distribution.

A: Here is an ARM template also (gives you a overview)
{
    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "sslCertificateData": {
            "type": "string",
            "metadata": {
                "description": "The base-64 encoded SSL certificate PFX data. Must be supplied via a parameters file references to a Key Vault / Secret Name."
            }
        },
        "sslCertificatePassword": {
            "type": "securestring",
            "metadata": {
                "description": "The SSL certificate password. Must be supplied via a parameters file references to a Key Vault / Secret Name."
            }
        },
        "vNetId": {
            "type": "string",
            "metadata": {
                "description": "The ID of the VNet."
            }
        },
        "subnetName": {
            "type": "string",
            "metadata": {
                "description": "The name of the DMZ Subnet."
            }
        }

    },
    "variables": {
        "networkApiVersion": "2017-04-01",

        "subnetId": "[concat(parameters('vNetId'), '/subnets/', parameters('subnetName'))]",

        "appGatewayPublicIpAddressId": "[resourceId('Microsoft.Network/publicIPAddresses', 'appGatewayPublicIpAddress')]",

        "appGwId": "[resourceId('Microsoft.Network/applicationGateways', 'appGateway')]",

        "appGwSize": "Standard_Small",
        "appGwTier": "Standard",
        "appGwCapacity": 5,
        "appGwFePort": 443,
        "appGwFeProtocol": "Https",
        "appGwBePort": 80,
        "appGwBEProtocol": "Http"
    },
    "resources": [
        {
            "type": "Microsoft.Network/publicIPAddresses",
            "name": "appGatewayPublicIpAddress",
            "location": "[resourceGroup().location]",
            "apiVersion": "[variables('networkApiVersion')]",
            "comments": "This creates a single, dynamically allocated public IP address for use by the Application Gateway.",
            "properties": {
                "publicIPAllocationMethod": "Dynamic"
            }
        },
        {
            "type": "Microsoft.Network/applicationGateways",
            "name": "appGateway",
            "location": "[resourceGroup().location]",
            "apiVersion": "[variables('networkApiVersion')]",
            "comments": "This creates the Application Gateway.",
            "dependsOn": [
                "[concat('Microsoft.Network/publicIPAddresses/', 'appGatewayPublicIpAddress')]"
            ],
            "properties": {
                "sku": {
                    "name": "[variables('appGwSize')]",
                    "tier": "[variables('appGwTier')]",
                    "capacity": "[variables('appGwCapacity')]"
                },
                "gatewayIPConfigurations": [
                    {
                        "name": "gatewayIpCofig",
                        "properties": {
                            "subnet": {
                                "id": "[variables('subnetId')]"
                            }
                        }
                    }
                ],
                "frontendIPConfigurations": [
                    {
                        "name": "frontendIpConfig",
                        "properties": {
                            "PublicIPAddress": {
                                "id": "[variables('appGatewayPublicIpAddressId')]"
                            }
                        }
                    }
                ],
                "frontendPorts": [
                    {
                        "name": "frontendPort",
                        "properties": {
                            "Port": "[variables('appGwFePort')]"
                        }
                    }
                ],
                "sslCertificates": [
                    {
                        "name": "appGwSslCertificate",
                        "properties": {
                            "data": "[parameters('sslCertificateData')]",
                            "password": "[parameters('sslCertificatePassword')]"
                        }
                    }
                ],
                "backendAddressPools": [
                    {
                        "name": "BackendAddressPool"
                    }
                ],
                "backendHttpSettingsCollection": [
                    {
                        "name": "HttpSettings",
                        "properties": {
                            "Port": "[variables('appGwBePort')]",
                            "Protocol": "[variables('appGwBeProtocol')]"
                        }
                    }
                ],
                "httpListeners": [
                    {
                        "name": "HttpListener",
                        "properties": {
                            "FrontendIPConfiguration": {
                                "Id": "[concat(variables('appGwId'), '/frontendIPConfigurations/frontendIpConfig')]"
                            },
                            "FrontendPort": {
                                "Id": "[concat(variables('appGwId'), '/frontendPorts/frontendPort')]"
                            },
                            "Protocol": "[variables('appGwFeProtocol')]",
                            "SslCertificate": {
                                "id": "[concat(variables('appGwId'), '/sslCertificates/appGwSslCertificate')]"
                            }
                        }
                    }
                ],
                "requestRoutingRules": [
                    {
                        "Name": "RoutingRule",
                        "properties": {
                            "RuleType": "Basic",
                            "httpListener": {
                                "id": "[concat(variables('appGwId'), '/httpListeners/HttpListener')]"
                            },
                            "backendAddressPool": {
                                "id": "[concat(variables('appGwId'), '/backendAddressPools/BackendAddressPool')]"
                            },
                            "backendHttpSettings": {
                                "id": "[concat(variables('appGwId'), '/backendHttpSettingsCollection/HttpSettings')]"
                            }
                        }
                    }
                ]
            }
        }
    ]
}
```

### ARM Template Parameter File

*app-gateway-parameters.json*
```
{
    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "sslCertificateData": {
            "reference": {
                "keyVault": {
                    "id": "/subscriptions/SUBSCRIPTION_ID/resourcegroups/RESOURCE_GROUP/providers/Microsoft.KeyVault/vaults/KEY_VAULT_NAME"
                },
                "secretName": "CERT_SECRET_NAME"
            }
        },
        "sslCertificatePassword": {
            "reference": {
                "keyVault": {
                    "id": "/subscriptions/SUBSCRIPTION_ID/resourcegroups/RESOURCE_GROUP/providers/Microsoft.KeyVault/vaults/KEY_VAULT_NAME"
                },
                "secretName": "CERT_PASSWORD_SECRET_NAME"
            }
        },
        "vNetId": {
            "value": "/subscriptions/SUBSCRIPTION_ID/resourceGroups/RESOURCE_GROUP/providers/Microsoft.Network/virtualNetworks/VNET_NAME"
        },
        "subnetName": {
            "value": "SUBNET_NAME"
        }
    }
}

redpajama_set_name: RedPajamaStackExchange