text:
\subsubsection*{\bibname}}

\bibliographystyle{apalike}

\usepackage[T1]{fontenc}
\usepackage{microtype}
\usepackage{graphicx}
\usepackage{subfigure}
\usepackage{booktabs}

\usepackage{xcolor}
\usepackage[nice]{nicefrac}

\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{algorithm}
\usepackage{algorithmic}
\usepackage{caption}

\DeclareMathOperator*{\argmin}{argmin}
\DeclareMathOperator*{\argmax}{argmax}
\DeclareMathOperator*{\cov}{cov}

\newcommand{\mathbf{X}_t}{\mathbf{X}_t}
\newcommand{\mathbf{y}_t}{\mathbf{y}_t}
\newcommand{\mathbf{\Phi}_t}{\mathbf{\Phi}_t}
\newcommand{\mathbf{w}_t}{\mathbf{w}_t}
\newcommand{\mathrm{Normal}}{\mathrm{Normal}}
\newcommand{\mathbf{A}_t}{\mathbf{A}_t}
\newcommand{\mathbf{\Phi}_{t, \mathbf{b}_t\downarrow}}{\mathbf{\Phi}_{t, \mathbf{b}_t\downarrow}}
\newcommand{\boldsymbol{\phi}_{t, \mathbf{b}_t\downarrow}}{\boldsymbol{\phi}_{t, \mathbf{b}_t\downarrow}}
\newcommand{\mathbf{z}}{\mathbf{z}}
\newcommand{\mathbf{I}}{\mathbf{I}}
\newcommand{\mathbf{0}}{\mathbf{0}}
\newcommand{\mathbb{E}}{\mathbb{E}}
\newcommand{\mathbf{x}}{\mathbf{x}}
\newcommand{\mathbf{\alpha}}{\mathbf{\alpha}}

\newcommand{\Diag(\mathbf{\alpha}_{\textit{new}})}{\Diag(\mathbf{\alpha}_{\textit{new}})}
\newcommand{\mathbf{\Phi}_{\textit{new}, b\downarrow}}{\mathbf{\Phi}_{\textit{new}, b\downarrow}}
\newcommand{\mathbf{X}_{\textit{new}}}{\mathbf{X}_{\textit{new}}}
\newcommand{\mathbf{y}_{\textit{new}}}{\mathbf{y}_{\textit{new}}}
\newcommand{\mathbf{\Phi}_{\textit{new}}}{\mathbf{\Phi}_{\textit{new}}}
\newcommand{\mathbf{w}_{\textit{new}}}{\mathbf{w}_{\textit{new}}}
\newcommand{\alpha_{\textit{new},i}}{\alpha_{\textit{new},i}}
\newcommand{\beta_{\textit{new}}}{\beta_{\textit{new}}}
\newcommand{\mathbf{\Phi}_{\textit{new}, b\downarrow}}{\mathbf{\Phi}_{\textit{new}, b\downarrow}}
\newcommand{\textit{new}}{\textit{new}}

\DeclareMathOperator{\Diag}{Diag}

\newtheorem{remark}{Remark}

\newcommand{\arabic{algorithm}}{\arabic{algorithm}}
\usepackage{hyperref}

\newcommand{\mathbb{R}}{\mathbb{R}}
\newcommand{\note}[1]{\textcolor{red}{#1}}
\newcommand{\ip}[2]{\left\langle #1, #2\right\rangle}
\newcommand{\sum\nolimits}{\sum\nolimits}
\newcommand{\jk}[1]{#1^{(j)}_k}
\newcommand{\jkp}[1]{#1^{(j)}_{k+1}}
\newcommand{\jkm}[1]{#1^{(j)}_{k-1}}
\newcommand{\jkn}[1]{#1^{(j)}_{0}}
\newcommand{\tilde{x}}{\tilde{x}}
\newcommand{\left(}{\left(}
\newcommand{\right)}{\right)}
\newcommand{\mathbbm{1}}{\mathbbm{1}}

\newcommand{\tilde}{\tilde}
\newcommand{\epsilon}{\epsilon}
\renewcommand{\mathbb{E}}{\mathbb{E}}
\newcommand{\mathbb{R}}{\mathbb{R}}
\newcommand{\mathcal{R}}{\mathcal{R}}
\renewcommand{\P}{\mathbb{P}}
\renewcommand{\mathbf{I}}{\mathbf{I}}
\newcommand{\mathcal{L}}{\mathcal{L}}
\newcommand{\left\langle}{\left\langle}
\newcommand{\right\rangle}{\right\rangle}

\newcommand{\mathrm{es}}{\mathrm{es}}
\newcommand{\mathrm{Comp}}{\mathrm{Comp}}
\renewcommand{\O}{\mathcal{O}}
\newcommand{x^{(j)}}{x^{(j)}}
\newcommand{y^{(j)}}{y^{(j)}}
\newcommand{\nu^{(j)}}{\nu^{(j)}}
\newcommand{\zeta^{(j)}}{\zeta^{(j)}}
\newcommand{Q^{(j)}}{Q^{(j)}}
\newcommand{M^{(j)}}{M^{(j)}}
\newcommand{\td{M}^{(j)}}{\tilde{M}^{(j)}}
\newcommand{W_{j}}{W_{j}}
\newcommand{g^{(j)}}{g^{(j)}}
\newcommand{G^{(j)}}{G^{(j)}}
\newcommand{\hat{x}^{(j)}}{\hat{x}^{(j)}}
\newcommand{\hat{\nu}^{(j)}}{\hat{\nu}^{(j)}}
\newcommand{\mathcal{I}^{(j)}}{\mathcal{I}^{(j)}}
\newcommand{\eta_{j}}{\eta_{j}}
\newcommand{\eta_{j-1}}{\eta_{j-1}}
\newcommand{\chi_{j}}{\chi_{j}}
\newcommand{\lambda_{t}}{\lambda_{t}}
\newcommand{\lambda_{j}}{\lambda_{j}}
\newcommand{\lambda_{j-1}}{\lambda_{j-1}}
\newcommand{\Delta_{L}}{\Delta_{L}}
\newcommand{\td{\Delta}_{L}}{\tilde{\Delta}_{L}}
\newcommand{J}{J}
\newcommand{T}{T}
\newcommand{B_{j}}{B_{j}}
\newcommand{B_{j-1}}{B_{j-1}}
\newcommand{B_{t}}{B_{t}}
\newcommand{\td{B}_{j}}{\tilde{B}_{j}}
\newcommand{\td{B}_{j-1}}{\tilde{B}_{j-1}}
\newcommand{\td{B}_{t}}{\tilde{B}_{t}}
\newcommand{b_{j}}{b_{j}}
\newcommand{b_{j-1}}{b_{j-1}}
\newcommand{m_{j}}{m_{j}}
\newcommand{m_{j-1}}{m_{j-1}}
\newcommand{e_{j}}{e_{j}}
\newcommand{\E_{j}}{\mathbb{E}_{j}}
\newcommand{\I_{j}}{\mathbf{I}_{j}}
\newcommand{N_{j}}{N_{j}}
\newcommand{\mathcal{L}_{j}}{\mathcal{L}_{j}}
\newcommand{\mathcal{L}_{j-1}}{\mathcal{L}_{j-1}}
\newcommand{\theta_{j}}{\theta_{j}}
\newcommand{\Lihua}[1]{{\color{red}#1}}

\makeatletter
\newcommand{\printfnsymbol}[1]{%
  \textsuperscript{\@fnsymbol{#1}}%
}

\DeclareMathOperator{\Ocal}{\mathcal{O}}

\begin{document}

\twocolumn[

\aistatstitle{Hyperparameter Transfer Learning \\with Adaptive Complexity}

\aistatsauthor{ Samuel Horv\'{a}th \And Aaron Klein \And  Peter Richt\'{a}rik \And C\'{e}dric Archambeau}

\aistatsaddress{ KAUST \And  Amazon Web Services  \And KAUST \And Amazon Web Services } ]

\begin{abstract}
Bayesian optimization (BO) is a sample efficient approach to automatically tune the hyperparameters of machine learning models.
In practice, one frequently has to solve similar hyperparameter tuning problems sequentially. For example, one might have to tune a type of neural network learned across a series of different classification problems. Recent work on multi-task BO exploits knowledge gained from previous tuning tasks to speed up a new tuning task.
However, previous approaches do not account for the fact that BO is a sequential decision making procedure. Hence, there is in general a mismatch between the number of evaluations collected in the current tuning task compared to the number of evaluations accumulated in all previously completed tasks.
In this work, we enable multi-task BO to compensate for this mismatch, such that the transfer learning procedure is able to handle different data regimes in a principled way. We propose a new multi-task BO method that learns a set of ordered, non-linear basis functions of increasing complexity via nested drop-out and automatic relevance determination. Experiments on a variety of hyperparameter tuning problems show that our method improves the sample efficiency of recently published multi-task BO methods.
\end{abstract}

\section{Introduction}

Bayesian Optimization (BO)~\citep{ review_shahriari2015taking, review_frazier2018tutorial} is a well-established framework to optimize an expensive black-box function $f: \mathcal{X} \rightarrow \mathbb{R} $ with a minimal number of function evaluations:
\begin{equation}
    \label{eq:bo}
    \mathbf{x}^{\star} = \argmin_{\mathbf{x} \in \mathcal{X}} f(\mathbf{x}),
\end{equation}
where $\mathcal{X} \subset \mathbb{R}^D$ denotes the configuration space. We assume $f(\mathbf{x})$ is only observed through noisy evaluations $y \sim \mathrm{Normal}(f(\mathbf{x}), \sigma)$. Intuitively, BO searches for the global optimum $\mathbf{x}^{\star}$ by evaluating a sequence of promising candidates $\mathbf{x}_1, \ldots, \mathbf{x}_N$. At each step, candidates are scored according to an exploration-exploitation criterion based on the evaluations collected so far. In the case of hyperparameter optimization (HPO), $f$ denotes the validation error and $\mathbf{x}$ is a vector of hyperparameters over which to optimize.

Let $\mathcal{D}= \{(\mathbf{x}_{n}, y_{n})\}_{n=1}^{N}$ be a set of evaluated hyperparameter configurations and their associated validation error. BO uses a probabilistic model $p(f \mid \mathcal{D})$ of the unknown objective function $f$ to select the next configuration by optimizing an acquisition function $a: \mathcal{X} \rightarrow \mathbb{R}$ that performs the explore-exploit trade-off.
The acquisition function only relies on the model $p(f \mid \mathcal{D})$ and, hence, is cheap-to-evaluate compared to the objective function.
Typical choices for the probabilistic model are Gaussian Processes (GPs)~\citep{gp_jones2001taxonomy, gp_snoek}, random forests~\citep{random_forrest}, tree parzen estimators~\citep{tpe_bergstra2011algorithms}, and Bayesian neural networks~\citep{dngo_snoek2015scalable, springenberg2016bayesian,ablr_perrone2018scalable}. Arguably, the most common acquisition function is the expected improvement (EI) \citep{BO_mockus1978application}. EI selects the next candidate to evaluate as the one with the highest expected improvement $a(\mathbf{x}) = \mathbb{E}[ \max\{0, f^* - f(\mathbf{x})\} ]$, where $f^*$ is the best evaluation observed so far. Numerous other acquisition functions have been proposed \citep{aq_kushner1964new, aq_frazier2009knowledge, aq_srinivas2009gaussian, aq_hoffman2011portfolio, aq_hennig2012entropy}.

\begin{figure*}[t]
  \subfigure[\texttt{ABLR}]{
  \includegraphics[width=0.24\textwidth]{plots/1d__ablr.png}
  \includegraphics[width=0.24\textwidth]{plots/basis_ablr.pdf}}
\hfill
  \centering
  \subfigure[\texttt{ABRAC}]{
  \includegraphics[width=0.24\textwidth]{plots/1d_nest_drop.png}
  \includegraphics[width=0.24\textwidth]{plots/basis_nest_drop.pdf}}
\hfill
  \caption{ Comparison of the surrogate objectives along with their basis functions obtained by (a) conventional multi-task BO, implimented by ABLR \citet{ablr_perrone2018scalable}, and (b) multi-task BO with adaptive complexity, ABRAC (see Section~\ref{sec:abrac}), which restricts the number of basis functions that can be activated at transfer. In the first and the third panel the target function is shown in blue, the five evaluations are the red points (which includes the same $3$ initial random points for both methods); the surrogate predictive mean is shown in orange with two standard deviations in transparent light blue. The second and fourth panel show the basis functions with their color representing their relative weight used at iteration 6 of the BO algorithm. It can be observed that overfitting occurs in (a) due to the large number of basis functions used, while no overfitting occurs in (b) by restricting that number (note that there are more than 2 basis functions (5) for (b), but they are almost invisible as their relative weights are very close to zero).}
  \label{fig:1d_unc_intro}
\end{figure*}

In this work, we focus on multi-task BO. We assume that we have already completed $T$ related HPO tasks $\{f_1, \dots f_T\}$ that share the same configuration space $\mathcal{X}$. Our goal is to leverage knowledge acquired in previous HPO tasks to accelerate the optimization of a new HPO task $f_{T+1}$.
In multi-task learning \citep[see, e.g.,][]{argyriou2006nips} the number of training data available for each task is typically of the same order. Multi-task BO, similar to BO, differs from conventional multi-task learning as it is an instantiation of Bayesian sequential decision making. This means that the training data are collected \emph{sequentially} during the training procedure, which in turns means that the number of data associated to the new task is initially much smaller than the number data that were collected in each previous task. Not accounting for this in practice is problematic as the transfer learning procedure is not adapted to the amount of information available in the target task as illustrated in Figure~\ref{fig:1d_unc_intro}.

Our proposed method achieves transfer learning by learning a set of shared basis functions across multiple tasks. Following \citet{ablr_perrone2018scalable}, we use an underlying shared dense neural network to learn the basis functions on top of which we place task-specific Bayesian linear regression head models.

We introduce a novel regularization strategy based on nested dropout~\citep{nested_rippel2014learning}, which we apply to the final layer of the neural network. This enforces the network to learn an ordered set of basis functions of increasing complexity, which in turn allows us to model different data regimes. We further leverage automatic relevance determination (ARD) to automatically determine the number of basis functions to be activated for transfer learning~\citep{mackay}.

Next, we summarize our contributions and relate our work to the recent literature on multi-task BO. In Section~\ref{sec:ablr} and \ref{sec:nested} we review, respectively, multi-head ABLR for multi-task BO and nested dropout. In Section~\ref{sec:abrac}, we describe our new multi-task BO method, which is able to learn ordered features for a multi-head ABLR model. We present an empirical evaluation in Section~\ref{sec:experiments}, showing that the sample efficiency of our proposed approach is superior to conventional multi-task BO. Finally, we provide a conclusion and future directions in Section~\ref{sec:disc}.

\begin{figure*}[!t]
\centering
\includegraphics[width=0.39\textwidth]{plots/for_all_regret_ablation.pdf}
\includegraphics[width=0.39\textwidth]{plots/for_rank_regret_ablation.pdf}
\caption{Comparison of the average regret and rank on Forrester functions for 5 different methods. Solid lines display average over all the runs of given method, transparent areas depict $2$ standard deviations.}
\label{fig:ablation}
\end{figure*}

\textbf{Contributions.}
Our contributions can be summarized as follows:
\begin{itemize}
\item We use nested dropout to learn an ordered set of features for transfer learning in the context of BO. To the best of our knowledge, no other multi-task learning approach proposed in the literature is able to learn features that take the adaptive complexity of the target task into account.
\item We use ARD to automatically determine which basis functions to activate at transfer in a data-driven fashion. Hence, the resulting transfer learning model is able to adapt its capacity to the amount of data available in the target task.\\
\item We show that we can improve the sample efficiency of multi-task BO and avoid overfitting in low data regimes without hurting the transfer learning performance in high data regimes.
\end{itemize}

Our multi-task BO method scales linearly with respect to the number of data points collected in the target BO task. This is achieved by adopting a two-step procedure: (1) a more expensive offline step consists in extracting knowledge in an ordered fashion from the completed BO tasks; (2) a fast online step exploits this knowledge to solve the new BO task data efficiently. The complexity of (1) is linear in the evaluations collected across all BO tasks, while the complexity (2) is linear in the data collected in the target BO task only.

\textbf{Related Work.}
\citet{gp_feurer2015initializing} defined a set of hand-designed meta-features to describe similarity between tasks.
Initial configurations to warm-start Bayesian optimization are then selected based on the euclidean distance of the meta-features of the current task to previous tasks.
Instead of manually defined features to describe tasks, \citet{law-nips19} proposed a feature extractor based on neural networks that operates directly on the underlying dataset.

Instead of learning features explicitly, \citet{swersky2013multi} proposed a discrete kernel to enable GP-based BO to model the correlation across tasks. While this model does not make any strong assumptions, it does not scale well with the number of tasks.
In a similar vein, \citet{springenberg2016bayesian} proposed Bayesian neural networks for BO and showed that, by using a linear embedding that is learned together with the other parameters of the neural network, one can model the objective function across tasks.
Instead of an additional input to the model, \citet{ablr_perrone2018scalable} considered a multi-head neural network architecture with one head per tasks. To obtain uncertainty estimates, they followed \citet{dngo_snoek2015scalable} by using Bayesian linear regression head models. \citet{salinas-icml20} used a semi-parametric Gaussian Copula distribution to map hyperparameter configurations to quantiles of the objective function across different tasks.
Most recently, \citet{moss-arxiv20} proposed a new information-theoretic acquisition function for Multi-Task Bayesian optimization which is much faster to compute than previous existing approaches.
Compared to our method, none of these transfer learning approaches learn an ordered set of features of variable complexity.

Also related to our approach is the idea to learn low-dimensional embeddings for high dimensional optimization tasks~\citep{moriconi-arxiv19,nayebi-icml19}. However, the key difference is that these embeddings are only learned on a single task without any transfer across tasks.

There also has been a large body of work on multi-task feature learning since the seminal work of \citet{argyriou2006nips}. Most of the published work focusses on learning a sparse set of features and methods that avoid negative transfer in multi-task learning~\citep{guo2014nips}. More recently, there has been a growing interest in transfer learning in the context of deep learning~\citep{ruder2017arxiv}. To the best of our knowledge, none of these techniques attempt to learn an ordered set of multi-task features.

\section{Multi-head ABLR for Multi-task Bayesian Optimization}\label{sec:ablr}

\begin{figure*}[t]
\centering
\includegraphics[width=0.39\textwidth]{plots/for_all_regret_with_BOH.pdf}
\includegraphics[width=0.39\textwidth]{plots/for_rank_regret_with_BOH.pdf}
\caption{Comparison of the average regret and rank on the class of Forrester functions for 6 different methods. Solid lines display average over all the runs of given method, transparent areas depict $2$ standard deviations.}
\label{fig:1d_comp}
\end{figure*}

Adaptive Bayesian linear regression (ABLR) is Bayesian linear regression with nonlinear basis functions $\boldsymbol{\phi}_{\mathbf{z}} (\mathbf{x}): \mathbb{R}^D \rightarrow \mathbb{R}^d$, here parameterized by $\mathbf{z}$ ~\citep{bishop2006pattern}. \citet{ablr_perrone2018scalable} extend this approach to the multi-task setting by maintaining a separate Bayesian linear regression head for each task on top of a \emph{shared} neural network to learn the nonlinear basis functions:
\begin{align}
\label{eq:ablr}
\begin{split}
p(\mathbf{y}_t|\mathbf{w}_t,\mathbf{z},\beta_t) &= \mathrm{Normal} (\mathbf{\Phi}_t(\mathbf{z}) \mathbf{w}_t, \beta_t^{-1}\mathbf{I}_{N_t}), \\
p(\mathbf{w}_t|\alpha_t) &= \mathrm{Normal} (\mathbf{0}, \alpha_t^{-1}\mathbf{I}_{d}),
\end{split}
\end{align}
where $\mathbf{y}_t \in \mathbb{R}^{N_t}$ is the vector of observed evaluations of $f_t$, $\mathbf{w}_t \in \mathbb{R}^d$ is the weight vector of the Bayesian linear regression head $t$, and $\mathbf{\Phi}_t(\mathbf{z}) = \boldsymbol{\phi}_\mathbf{z} (\mathbf{X}_t) \in \mathbb{R}^{N_t \times d}$ the corresponding design matrix with $\mathbf{X}_t \in \mathbb{R}^{N_t \times D}$ containing the configurations associated to $\mathbf{y}_t$. The identity matrices $\mathbf{I}_{N_t}$ and $\mathbf{I}_d$ are respectively of size $N_t \times N_t$ and $d \times d$, and the scalars  $\alpha_t >0 $ and $\beta_t >0 $ are precisions parameters (or inverse variances). The weight vectors $\{\mathbf{w}_t\}_t$ can be integrated out analytically. The resulting log-marginal likelihood can then be maximized with respect to the network parameters $\mathbf{z}$, as well as the task precisions $\{\alpha_t\}_t$ and $\{\beta_t\}_t$.

The shared underlying neural network used in multi-head ABLR makes it a suitable for multi-task BO~\citep{swersky2013multi,springenberg2016bayesian,ablr_perrone2018scalable}. Let $\mathcal{D}= \big\{ D_t : D_t =\{(\mathbf{x}_{tn}, y_{tn})\}_{n=1}^{N_t} \big\}_{t=1}^{T}$ be the data collected while optimizing the set of black-box functions $\{ f_t\}_{t=1}^T$. The basis functions $\boldsymbol{\phi}_\mathbf{z}(\cdot)$ encode a shared representation based on $\mathcal{D}$, while the task-specific head model parameters are learned from the corresponding $\mathcal{D}_t$. Hence, the optimization of a new task $f_{T+1}$ can be warm-started by adding a new head model on top of the neural network (that was trained  on $\mathcal{D}$) and use it as a surrogate model to optimize:
\begin{equation}
    \label{eq:bo_new}
    \mathbf{x}^{\star}_{T+1} = \argmin_{\mathbf{x} \in \mathcal{X}} f_{T+1}(\mathbf{x}),
\end{equation}
A practical advantage of multi-head ABLR is that the complexity of learning the neural network is linear in the total number of evaluations, unlike multi-ouput GPs~\citep{nguyen2014collaborative}, while the complexity of learning the head parameters is linear in the number of task evaluations. These attractive computational properties are retained by our proposed method, ABRAC, as discussed in more detail in Section~\ref{sec:abrac}.

\begin{algorithm}[tb]
   \caption{ABRAC}\label{alg:abrac}
\begin{algorithmic}
   \STATE {\bfseries Input:} number of initial points $n_0$, budget $N$, feature net $\phi_z(\cdot): \mathbb{R}^P \rightarrow \mathbb{R}^d$, filter $F^k$, previous evaluations $\{\{(x_{ti}, y_{ti})\}_{i=1}^{N_t}\}_{t=1}^T$.
   \STATE Fit $\phi_\mathbf{z}(\cdot)$ using $\{\{(x_{ti}, y_{ti})\}_{i=1}^{N_t}\}_{t=1}^T$ --Section~\ref{sec:offline_know_extr}.
   \STATE Observe $f_{\textit{new}}$ at $n_0$ randomly selected points $x_1, x_2, \dots, x_{n_0}\in \mathcal{X}$.
   \STATE $\mathcal{C} = \{(x_i, y_i)\}_{i=1}^{n_0}$, where $y_i = f(x_i)$.
    \STATE Set $n = n_0$.
   \WHILE{$n < N $}
   \STATE Fit probabilistic model $g$ based on  $\mathcal{C}$--Section~\ref{sec:online_tr}.
   \STATE $x_n = \argmax_{x \in \mathcal{X}} A_g(x)$, where $A$ is given acquisition function.
   \STATE Observe $y_n = f_{\textit{new}}(x_n)$.
   \STATE Update $\mathcal{C} = \mathcal{C} \cup \{(x_n, y_n)\}$.
    \STATE $n = n + 1$
   \ENDWHILE
   \STATE {\bfseries Output:} $\hat{x} = \argmin_{i = 1,2,\dots, N} f_{\textit{new}}(x_i)$
\end{algorithmic}
\end{algorithm}

\begin{figure*}[t]
  \centering
  \subfigure[\texttt{ABRAC}]{\includegraphics[width=0.22\textwidth]{plots/1d_nest_drop.png}}
\hfill
  \subfigure[\texttt{ABLR L-BFGS}]{\includegraphics[width=0.22\textwidth]{plots/1d__ablr.png}}
\hfill
  \subfigure[\texttt{ABLR L-BFGS fixed}]{\includegraphics[width=0.22\textwidth]{plots/1d_fixed_ablr.png}}
\hfill
  \subfigure[\texttt{ABLR SGD fixed}]{\includegraphics[width=0.22\textwidth]{plots/1d_fixed_sgd.png}}
\hfill
  \caption{ Comparison of the surrogate objective along with its basis functions after 5 total evaluations obtained by 4 different methods. The target function is shown in blue, the evaluations are the red points (which includes the same $3$ initial random points for both methods), and the surrogate predictive mean is shown in orange with two standard deviations in transparent light blue.}
  \label{fig:1d_unc}
\end{figure*}

\section{Nested Dropout for Ordered Representations}\label{sec:nested}

Nested dropout is a structured type of dropout originally proposed for autoencoders to learn ordered representations in their bottleneck layer~\citep{nested_rippel2014learning}. It was also used in later works to learn spectral representations in convolutional neural networks~\citep{rippel2015spectral} and ordered word representations~\citep{liu2016learning}.

Assume there are $d$ neurons in the bottleneck layer. Nested dropout is applied during training by deactivating these neurons according to a discrete distribution $Q$ over the indices $1, \dots, d$. In each training step, one samples $b \sim Q$ and drops the neurons $b+1, \dots, d$. This results in the activation of a different, but nested subset of indices at each training step for which, if $b$ belongs to the subset, then $1, \dots, b-1$ also belongs to it. Hence, every mini-batch gradient update during training can only touch the neurons in the bottleneck layer that were not dropped, which, as a result, leads to an importance ranking of the representation dimensions. Nested dropout comes with strong theoretical properties for linear autoencoders: the solution obtained when applying nested dropout to the linear autoencoder recovers principal component analysis under an additional orthonormality constraint when $Q(b = j) > 0,  \forall j \in 1, \dots, d$~\citep[see][for details]{nested_rippel2014learning}.

\section{Multi-head ABLR with Adaptive Complexity (ABRAC)}\label{sec:abrac}

A deficiency of multi-head ABLR when applied to multi-task BO is that the number of nonlinear basis functions is not adapted to the target task, where the number of observations is typically smaller than in the previous tasks, making it prone to overfitting. This issue is due to the direct use of conventional multi-task models in the context of sequential decision making problems such as BO. To solve this issue we propose to learn an ordered set of learned basis functions, which we selectively activate during the transfer learning procedure. Pseudocode is given in Algorithm~\ref{alg:abrac}.

To endow multi-head ABLR with ordered representations for transfer learning, we apply nested dropout during training to the last layer of the shared neural network $\boldsymbol{\phi}_\mathbf{z} (\mathbf{x})$, which operates as the feature extractor. At each training step, we activate a nested subset of the output neurons according to a uniform distribution over the indices: $Q(b = j) = \nicefrac{1}{d}$, $\forall j \in 1, \dots, d$. Departing from \citet{nested_rippel2014learning}, who proposed to sample the truncation index according to a geometric distribution, we observed empirically that the uniform distribution performed well without any further modifications, and with the additional benefit that we sample neurons with a high index more often and do not have to set additional hyperparameters.

Next, we introduce multi-head ABLR with adaptive complexity (ABRAC) for multi-task BO and detail the inference procedure. We propose a two-step approach to do transfer learning in the context of Bayesian sequential decision making consisting in (1) an off-line learning of ordered shared representations and (2) an online adaptation of the transfer learning procedure to improve the sample efficiency of the target BO task.

\textbf{Probabilistic model.} We make a minor modification of (\ref{eq:ablr}), which consists in allowing for a different precision parameter per basis function:
\begin{align}
\label{eq:model}
p(\mathbf{w}_t | \boldsymbol{\alpha}_t) &= \mathrm{Normal} (\mathbf{0}, \Diag(\boldsymbol{\alpha}_t)^{-1}),
\end{align}
where $\alpha_{ti}>0$ is the precision associated to basis function $i$. Training consists in optimizing the log-marginal likelihood wrt the shared parameters $\mathbf{z}$ and the head model parameters $\{\boldsymbol{\alpha}_t,\beta_t\}_t$:
\begin{align}
\label{eq:logmarginal}
&\sum_t\ln p(\mathbf{y}_t | \mathbf{z}, \boldsymbol{\alpha}_t,\beta_t) =\\
&\sum_t\ln \mathrm{Normal}(\mathbf{0} , \beta_t^{-1}\mathbf{I}_{N_t} + \mathbf{\Phi}_t(\mathbf{z})\Diag(\boldsymbol{\alpha}_t)^{-1}\mathbf{\Phi}_t(\mathbf{z})^\top) . \nonumber\
\end{align}
Allowing for different $\alpha_{ti}$ promotes sparsity by enabling us to automatically determine the relevance of every single component of $\mathbf{w}_t$~\citep{mackay}. The application of automatic relevance determination (ARD) to Bayesian linear regression was studied in \citep{rvm2} and \citep{wipf}.

\begin{figure*}[t]
\centering
\includegraphics[width=0.31\textwidth]{plots/quad_all_regret.pdf}
\includegraphics[width=0.31\textwidth]{plots/quad_rank_regret.pdf}
\includegraphics[width=0.31\textwidth]{plots/comp_time.pdf}
\caption{Comparison of the average regret and rank on the class of parameterized quadratics for 7 different methods. Solid lines display average over all the runs of given method, transparent areas depict $2$ standard deviations. The most right plot displays comparison of the speed of ABRAC, different ABLR based methods and GP enriched by subsampling.}
\label{fig:quad_comp}
\end{figure*}

\subsection{Offline Ordered Representation Learning}
\label{sec:offline_know_extr}

To learn ordered representations suitable for transfer learning, we first train a neural network $\phi_\mathbf{z}(\cdot)$ off-line. We minimize the mean square error between neural network predictions and the observations collected during all previous tasks using SGD with momentum~\citep{sutskever2013importance}. This is different from \citet{ablr_perrone2018scalable}, where L-BFGS is used to learn $\mathbf{z}$ jointly with $\{\boldsymbol{\alpha}_t,\beta_t\}_t$, and akin DNGO~\citep{dngo_snoek2015scalable}. However, we apply nested dropout to the last layer during training as discussed next.

Before applying a mini-batch update, we first sample a random truncation $b$ for each data in the mini-batch. Let $\boldsymbol{\phi}(\mathbf{x})_{b\downarrow}$ be a random truncation of the vector $\boldsymbol{\phi}(\mathbf{x})$ where the last $d-b$ elements are masked with 0. For example, $\boldsymbol{\phi}(\mathbf{x})_{2\downarrow}^\top = [\phi_1(\mathbf{x}),\phi_2(\mathbf{x}),\phi_3(\mathbf{x}),\phi_4(\mathbf{x})]_{2\downarrow}^\top = [\phi_1(\mathbf{x}),\phi_2(\mathbf{x}), 0, 0]^\top$. If $b = d$, then $\boldsymbol{\phi}(\mathbf{x})_{b\downarrow}=\boldsymbol{\phi}(\mathbf{x})$ and $(\cdot)_{b\downarrow}$ is the identity operator. Hence, we can view the off-line training procedure as maximizing a likelihood of which the terms correspond to a product of normal distributions:
\begin{align}
\label{eq:modelinterpret
p(y_{tn}|\mathbf{w}_t,\mathbf{z},\beta_t) &= \prod_{i=1}^d \mathrm{Normal} (\boldsymbol{\phi}_{\mathbf{z}}(\mathbf{x}_{tn})_{i\downarrow}^\top\mathbf{w}_t, \beta_t^{-1})^{\delta_{ib_{tn}}}
\end{align}
where $\delta_{ij}$ is the Kronecker delta and the truncation $b_{tn}\sim \mathrm{Uniform}(1,\ldots,d)$ is resampled at each iteration of the optimizer.

\subsection{Online Adaptive Transfer Learning}
\label{sec:online_tr}

When tackling a new BO task, we fix the feature extractor $\boldsymbol{\phi}_\mathbf{z}(\mathbf{x})$ and learn the additional head model parameters by maximizing the corresponding term in (\ref{eq:logmarginal}) after each addition of a new observation. Doing so dynamically adjusts the level of sparsity of Bayesian linear regression (BLR) via ARD. Maximizing the log-marginal likelihood of BLR has overall complexity $\mathcal{O}(d^2\max\{N_\textit{new}, d\})$, which is linear in number of evaluations of the new BO task. See \citet{bishop2006pattern}.

\section{Experiments}\label{sec:experiments}

We now present several experiments that show the benefits of our new proposed method, dubbed ABRAC.
Firstly, we use the 1-D synthetic Forrester function, which allows us to visually quantify the model fit and uncertainty estimations.
Afterwards, we present results on the parametrized quadratic functions by~\citet{ablr_perrone2018scalable}.
Lastly, we benchmark ABRAC on real-world HPO problems containing feed-forward neural networks and support vector machines.

We implemented ABRAC and ABLR in the python framework Emukit~\citep{emukit2018} where the neural network code is based on MXNet~\citep{mxnet_chen2015mxnet}. For L-BFGS, we used the SciPy~\citep{scipy} implementation.
For Gaussian Process (GP) based BO,  we used a Matern52 to model the correlation between hyperparameters with automatic relevance determination, and optimize all hyperparameters by empirical Bayes~\citep{rasmussen2003gaussian}.
All BO methods used expected improvement as acquisition function~\citep{BO_mockus1978application}.

\begin{figure*}[!t]
\centering
\includegraphics[width=0.39\textwidth]{plots/nas_all_rank.pdf}
\caption{Comparison of the average rank on the NAS Benchmarks for 5 different methods. Solid lines display average over all the runs of given method, transparent areas depict $2$ standard deviations. }
\label{fig:nas_comp}
\end{figure*}

\begin{figure*}[!t]
  \centering
 \includegraphics[width=0.24\textwidth]{plots/nas_ProteinStructure_regret.pdf}
  \includegraphics[width=0.24\textwidth]{plots/nas_NavalPropulsion_regret.pdf}
   \includegraphics[width=0.24\textwidth]{plots/nas_SliceLocalization_regret.pdf}
  \includegraphics[width=0.24\textwidth]{plots/nas_ParkinsonsTelemonitoring_regret.pdf}
\hfill

   \includegraphics[width=0.24\textwidth]{plots/nas_ProteinStructure_rank.pdf}
  \includegraphics[width=0.24\textwidth]{plots/nas_NavalPropulsion_rank.pdf}
   \includegraphics[width=0.24\textwidth]{plots/nas_SliceLocalization_rank.pdf}
  \includegraphics[width=0.24\textwidth]{plots/nas_ParkinsonsTelemonitoring_rank.pdf}
\hfill
\caption{Comparison of the average regret and rank on the NAS Benchmarks for 5 different methods. Each column corresponds to one dataset (from left:  \texttt{Protein Structure, Naval Propulsion, Slice Localization}, and \texttt{Parkinsons Telemonitoring}). Solid lines display average over all the runs of given method, transparent areas depict $2$ standard deviations.
}
  \label{fig:nas_ind}
\end{figure*}

We include BOHAMIANN~\citep{springenberg2016bayesian} as baseline, but, because of its significant computational overhead (approximately $100\times$ slower than ABLR~\citep{ablr_perrone2018scalable}) we show its performance only on the first experiment. For all other experiments, we ran BOHAMIANN for $10$ iterations and it did not outperform ABRAC in any of those.

To highlight the benefits of using an offline-trained feature extractor with hierarchical basis functions, we include the following two modifications of ABRAC without nested dropout: \textit{ABLR SGD fixed} trains the features extractor by minimizing the squared error with SGD, which can be seen as an efficient multi-task version of DNGO~\citep{dngo_snoek2015scalable} and \textit{ABLR L-BFGS fixed} which minimizes the negative log-likelihood with L-BFGS. Unlike  \textit{DNGO}, both versions do not require any contextual information. The main difference to standard \textit{ABLR} is that the neural network $\boldsymbol{\phi}_{\mathbf{z}}$ is fixed for the new task.
Furthermore, to analyse the benefit of using a neural network based feature extractor, we also include another baseline that uses random kitchen sinks as basis functions (ABLR RKS).

For each experiment, we average the performance across tasks in a leave-one-task-out fashion.
Each optimization run was initialized with $3$ random data points and we reported average performance of each method across $5$ repetitions with a different seed.
All experiments were conducted on AWS \texttt{ml.c4.2xlarge} instances.

\begin{figure*}[t]
\centering
\includegraphics[width=0.37\textwidth]{plots/svm_all_regret.pdf}
\includegraphics[width=0.37\textwidth]{plots/svm_rank_regret.pdf}
\caption{Comparison of the average AUC and rank on the OpenML-SVM Benchmarks for 6 different methods. Solid lines display average over all the runs of given method, transparent areas depict $2$ standard deviations.}
\label{fig:openmlsvm_comp}
\end{figure*}

\textbf{Forrester Function.}
For the first experiment, we consider the parameterized class of Forrester functions $f_t(x|a_t,b_t,c_t) = (a_t x-2)^2\sin(b_t x-4)+c_t$ with the configuration space $\mathcal{X} = [0,1]$. We create new tasks by sampling different coefficients $a_t \sim \mathrm{Normal}(6,1)$, $b_t \sim \mathrm{Normal}(12,4)$, $c_t \sim \mathrm{Uniform}(0,10)$.
We generated 10 tasks with $20$ uniformly distributed data points each.
The architecture of the feature extractor $\boldsymbol{\phi}_\mathbf{z}(\mathbf{x})$ consists of a $2$-layers fully connected neural network with $50$ hidden units and $\tanh$ activation and $d=20$ output units with linear activation functions.

Firstly, we investigate the effects of the two major components of ABRAC: the offline training with nested dropout and the online transfer learning with adaptive basis function selection.
To address the former we compare against a modified version of ABLR trained with SGD, dubbed \textit{ABLR SGD adapt}, where we incorporate our adaptive procedure during the optimization.
For  the latter we include a version of ABRAC which considers all basis functions at every step (ABRAC fixed).
Looking at Figure~\ref{fig:ablation}, one can see that not only in isolation, each of the two components improves over vanilla ABLR (with SGD training) but in combination, i.e  ABRAC, we achieve the best performance.

In Figure~\ref{fig:1d_comp} we compare against a range of baselines: Multi Task Gaussian process based Bayesian optimization (MT-GP-BO),  which is a standard GP defined over the search space and the task-specific contextual information $(a_t, b_t, c_t)$ , random search (RS), ABLR, BOHAMIANN as well as ABLR SGD fixed and ABLR L-BFGS fixed ascribed above.
Similarly to MT-GP-BO,  BOHAMIANN is  provided with contextual information $(a_t, b_t, c_t)$ for each task to enable multi-task learning.  No other algorithm is given this information.
One can see that ABRAC quickly adapts to the new task and achieves already after 5 function evaluations an acceptable regret. In contrast,  GP-BO and ABLR require more samples to approach the global optimum.
Interestingly, the baselines ABLR SGD fixed and ABLR L-BFGS fixed perform substantially worse, highlighting the importance to nested dropout to learn adaptive basis functions.
To obtain a better intuition of our method, we show in Figure~\ref{fig:1d_unc} the posterior mean and standard deviation of ABRAC and the ABLR variants after two optimization steps. All methods were initialized with the same three data points.
Vanilla ABLR is not able to account for the reduced complexity of this low data regime and heavily overfits.
On the other hand, ABLR with fixed basis functions explains all data points with observation noise.
ABRAC is able to capture the right complexity and provides already in this early stage of optimization a meaningful model. Similar plots can be found in Figure~\ref{fig:1d_unc_app} in the Appendix, where we show the same plots but with 5 fixed data points among all the tasks. In addition, we also show the extracted basis functions with their relative weights in Figure~\ref{fig:1d_unc_intro}.

\textbf{Parameterized Quadratic functions.}

In the second experiment, we compare different methods on the parameterized quadratic functions of the form $f_t(x|a_t,b_t,c_t) = a_tx^\top x + b_t \mathbf{1}^\top x + c_t$ from \citet{ablr_perrone2018scalable}, where $\mathbf{1}$ is an all-ones vector, and the coefficients are sampled uniformly $a,b,c \sim \mathrm{Uniform}(0.1, 1)$.
The configuration space is defined by $\mathcal{X} = [-10, 10]^5$.
To train the feature extractor we generated $30$ tasks with $100$ random data points each.

Due to scalability issues we warm started MT-GP-BO with only $10$ randomly selected datasets from each task, i.e in total $290$ data points.
In order to distinguish between tasks, we provided MT-GP-BO with contextual information $(a_t, b_t, c_t)$ for each task.
Note that, none of the other methods is provided with contextual information.
Figure~\ref{fig:quad_comp} shows that ABRAC is able to outperform all other methods. The only method that can reach ABRAC's performance after $45$ iterations is ABLR SGD fixed.
In Figure~\ref{fig:quad_comp} right, we report the optimization overhead of all neural network based methods.
One can see that in all scenarios, ABRAC is not just the most efficient method in terms of sampling efficiency, but also in terms of computational overhead where it is more than  $100 \times$ faster than ABLR or GPs.

\textbf{Tabular Benchmarks.}
For the next experiment, we used the tabular benchmark by \citet{klein-arxiv19a} which simulates the discrete HPO of a 2-layer fully-connected neural network on 4 different regression datasets.
\citet{klein-arxiv19a} performed an exhaustive search of all $62208$ possible configurations in the search space per dataset, such that for a single function evaluation one can simply look up the performance in the table instead of training the neural network from scratch.
For further details we refer to the original paper.
For each task, we generate  $1024$ random configurations which are used for the offline feature learning. Since we have only access to $4$ datasets, we increase the number of repetition with a different seed to $10$.  Since we have no access to contextual information we don't include MT-GP-BO as a baseline but only GP-BO.

Figure~\ref{fig:nas_comp} shows that in terms of rank, ABRAC outperfoms all other method with significant margin, which do not improve much upon random search in the given budget. We include 4 additional graphs in Figure~\ref{fig:nas_ind}, where each of the figures displays average regret and rank for one of four datasets. One can see that for all the datataset, ABRAC is the best in terms of average rank and the same holds for the average regret except for \texttt{Naval Propulsion}. However, for this dataset random search already achieves a good performance and none of the methods significantly improves over it.

\textbf{SVM on OpenML datasets.}

Lastly, we consider the HPO of support vector machines (SVM) with 2 continuous hyperparameters: the regularization parameter \textit{C}(min: 0.000986, max: 998.492437) and the kernel parameter \textit{gamma}(min: 0.000988, max: 913.373845) on OpenML~\citep{vanschoren2014openml} classification datasets.
We chose the most evaluated flow (\texttt{flow id} 5891) with its 30 most evaluated datasets/tasks with almost one million evaluations in total.

We provide MT-GP-BO with contextual information: the number of features, the number of data points and the percentage of the majority class, and warm start it using 10 random points from each of the 29 related tasks.
We compare these methods based on the average rank and average Area Under Curve (AUC). To model AUC on unseen configurations, we use the surrogate modeling approach from \citet{eggensperger2015efficient} based on a random forest model.
Figure~\ref{fig:openmlsvm_comp} confirms the superior behaviour of ABRAC to all other methods while the only method that can reach comparable performance in terms of AUC is ABLR, which considers all data in each step.

\section{Conclusion}
\label{sec:disc}

We introduced ABRAC, a probabilistic model for Bayesian optimization with adaptive complexity.
It first learns a set of basis function based on neural networks on offline generated data with increasing complexity.
During Bayesian optimization, we use Bayesian linear regression to obtain uncertainty estimates and thereby regularizes our basis functions.
Compared to Gaussian processes our model scales linearly rather than cubically with the number of datapoints.
In a battery of experiments, we show that ABRAC obtains much higher speed ups compared to other baseline methods, such as ABLR or Gaussian processes based Bayesian optimization.

redpajama_set_name: RedPajamaArXiv


My promotions tour continues as I revise book 2, slated for November of this year!
Today I was over at the site of the illustrious Mary Robinette Kowal, talking about my favorite bit from my novel.
Tuesday I was at Black Gate, sitting down with John O'Neill to talk about writing and reading and editing and nearly everything I love.
Wednesday Paul Weimer reviewed my new book for TOR.com.
Monday I talked with Paul Semel about the book and my life in general.
There've been some lovely additional reviews from more lovely places, like The Quill to Live, and the UCReview, and Color me Read, and Infinite Free Time.
And, of course, a few weeks ago I sat down with the Little Red Reviewer.
I'll be appearing in more places around the web in the coming days. For now, I must return to my writer's garret!
Tuesday, February 19th, I'll be over at Reddit Fantasy holding an AMA. Which means that you can swing by at any time of the day or night and ask me about writing or editing or reading or nearly anything I like to do and I'll be on hand to talk about it. I hope to see you there! Once the interview thread goes live I'll post a link, but for now all I have is the link to the site.
In other news, I recently learned that For the Killing of Kings is the top pick for BookPages' Fantasy and Science Fiction category this March, and I got a nice review of the book in the Philadelphia Free Press.
Tomorrow is book launch day! I will probably be too busy promoting the book to be nervous. Keep your fingers crossed for me. Better yet, buy the book and help spread word about it!
The update is nearly complete, and the new face on the web site is working smoothly, thanks to MeanThemes, who supplied this great new look and helped me get it up and running.
I'll write more about them soon, but for now I wanted to share the brand new copy of For the Killing of Kings. I've been showing the paperback advanced reader copy around, but this is the first actual hardback I've had in my own possession, and it is a nice thing to have. Here's hoping many other people think so as well!
Here's a new review that came in last week, as well.
Behind-the-scenes, much prep work is underway for the book's debut on February 19th. Revisions are afoot on its sequel, and I'm making copious notes on a brand new idea I had the other day that might turn into a follow-up series of books. Once I finish revising book 2 and writing book 3, I mean.
In the coming days there are apt to be some curious changes underway on the web site. Colors will be changing, as will typeface, organization, and so on — a lot of it's up for grabs. Hopefully I'll have it all straightened out in just a few days, so bear with me… The intention is to make the entire web site a lot more user friendly.


text:
OGCA speaks out against third party verification companies
Contractors petition government as association urges members not to accept contracts requiring verification services
The Ontario General Contractors Association (OGCA)has issued a letter urging its members not to accept contract pre-qualification conditions requiring the use of third-party verification companies.
The association also started a petition to the Ontario Minister of Labour and the legislature asking for government intervention in the matter.
The third-party services verify that safety and other regulatory documentation is in place, and construction project owners (including public agencies) which contract with these businesses will generally not accept or invite contractors and sub-trades to bid on work unless they use the services and the correct documentation is in place, according to these services.
"We do not support the requirement being imposed by owners that bidders be pre-qualified on the basis of membership in, or agreement with, these third party companies," OGCA president Clive Thurston wrote in an Oct. 31 memo to members and stakeholders. "This new requirement on the industry does not, in our opinion, provide added value proportionate to its costs. Moreover, we believe that Ontario's culture of safety", of "internal responsibility" and focus on due diligence best promotes the conduct we expect from everyone involved in the construction industry."
The verification services have expanded from their US home bases, especially within the petrochemical and heavy industry sectors, to Canadian markets. One of the largest, ISNetworld, plans to open a Toronto office in January. The company has won contracts with the Regional Municipality of York and Ontario Power Generation, among others, and is seeking to expand its business In Ontario, Quebec and Atlantic Canada.
The verification services consolidate and validate health, safety and regulatory compliance documentation, removing the administrative and verification challenges from owners, In return, contractors and suppliers purportedly gain access to business opportunities, ensure their compliance with a myriad of rules and regulations, and have a one-stop system to manage their compliance requirements.
However, several members have complained to the OGCA and others about the third party services cost and their impact on administrative costs and client relationships.
In the letter to members, Thurston says that the association supports "a strong auditing system for companies and their safety programs."
"That is why, we, the IHSA and others invest in and promote training and educational programs such as CoR, that are not just classroom-based but actually reach out to sites," Thurston wrote. "our on-site audit and verification system s are actually implemented in the field. CoR accreditation, which is a nationally recognized standard, confirms programs and policies are in place and that they are fully implemented. The IHSA accredits that firms have achieved a high level of performance and continue to perform. Infrastructure Ontario, TTC, and Metrolinx, among many others, use COR to pre-qualify their construction procurement."
Thurston said the third-party services, on the other hand, do not provide "added value to procurement, construction or health and safety as the services they provide are very limited."
"These companies generally confirm that the contractor has submitted documentation of policies and processes that have been put into place, but not that the contractor has achieved a high standard of health and safety performance or that the written processes and procedures have been implemented."
The OGCA petition says:
"To the Honorable Minister of Labour and the Ontario Legislature:
"We, as members for the Ontario Construction Industry, request that the Hon. Minister of Labour and the Ontario Legislature address the problem of Third Party Safety Verification. We collectively request that immediate action be taken to accredited employers who meet the standards for health and safety excellence via programs such as COR.
"There are numerous private unregulated companies that verify safety compliance, many do not verify that the programs are delivered. As a result, owners and contractors will be left with the mistaken belief that they meet or exceed their legal Health and Safety obligations; further, programs such as COR exist which verify that that the contractor is in full compliance.
"The Minister of Labour must act immediately to implement the Expert Advisory Panel recommendation #23 to: 'Develop an accreditation system that recognizes employers who successfully implement a health and safety management system'."
Previous articleDecember 2014 – Current Issue
Next articleTony Dean introduces himself with new website for OCOT review
Gateway to invest around $31 million to build casino in City of North Bay
Osgoode Professional Development: Osgoode Certificate in Construction Law 2019
Toronto stands alone in remaining a "construction employer" as other Ontario municipalities embrace Bill 66 changes

redpajama_set_name: RedPajamaCommonCrawl


text:
Lionsgate Execs Talk 'Mockingjay' Filming, Marketing and More!
Carla Pinilla Archive January 22, 2014	Marketing Campaigns 9 Comments
To accompany the amazing reveal of the official The Hunger Games: Mockingjay Part 1 logo, several Lionsgate execs involved with the franchise spoke to Variety about the success of Catching Fire, and what's coming up for the next two movies. Some very interesting details were revealed in this article!
Tim Palen, aka the man behind the excellent marketing campaigns for the first two movies, explained that they've worked the franchise as one long project, and reiterates their focus on international marketing:
"When we started, we decided to look at this as one big movie that's eight hours long," notes Tim Palen, the architect of Lionsgate's marketing effort. "Otherwise, it's going to be kind of overwhelming to do a new campaign for each movie."
"A big part of the second film was growing the franchise, and the greatest potential was international," he says. "We did a major presentation to distributors in Cannes, and did six premieres outside the U.S. And we still have room to grow. We've matched 'Iron Man 3' on domestic, but we can do better on international."
Erik Feig, production president at Lionsgate, had some comments about Katniss as a cinematic icon:
"Not since Luke Skywalker shot into the Death Star has one person had that kind of impact in a franchise film." […] "What I really love about how (the 23-year-old Lawrence) plays that character is that Katniss is larger than life and yet she wears her emotions in a very relatable way. She's not a superhero by any stretch — she's all too human in the face of extreme conditions — and she faces them with the presence of mind that we all wish we had."
The article also reveals that the film will have a presence at Cannes in May, and at San Diego Comic Con in July (rest assured, Mockingjay.net will be present at the latter to bring you all the details!). On top of that, Feig announced two locations they will be filming at in Europe: Tempelhof Airport in Berlin (excuse me while I do a little "they're filming here!" dance 😉 ), and a series of apartment complexes in Paris.
"We (looked for) big buildings that have been around forever. We kept coming back to classic war movies, and we started thinking we should look into Europe."
There's plenty more details like this scattered throughout the article, so be sure to give it a read! It's fantastic that we're already getting so much info about Mockingjay. Looks like it's going to be another crazy fun year filled with Hunger Games surprises! We can't wait. 🙂
Tags Berlin Cannes Film Festival Erik Feig Filming Locations Francis Lawrence Nina Jacobson Paris San Diego Comic Con The Hunger Games: Catching Fire The Hunger Games: Mockingjay The Hunger Games: Mockingjay Part 1 Tim Palen Variety
About Carla Pinilla Archive
All posts under the "Carla Pinilla Archive" were originally written by Carla Pinilla and posted on Mockingjay.net. Original comments have been deleted and discussion closed.
Previous Welcome to Jabberjays.net
Next Jabberjays Episode #5 – A Very Friendly Victor
'The Hunger Games: Catching Fire' Named one of The New York Times' Most Influential Films of the Decade
Mahershala Ali Nominated For Golden Globe, Wins Critics' Choice Award
VIDEOS: Chrysler and Dodge Ram Join the Revolution with TV Spots
Chrysler and Dodge Ram are the latest commercial partners and sponsors Lionsgate has teamed up with, and to celebrate several supercharged TV spots have been debuted.


text:
TransGenRx receives LED R&D tax credit
Linda Benedict, Cooper, Richard K. | 11/15/2012 3:19:57 AM
News Louisiana
BATON ROUGE, La. – TransGenRx Inc. (tgrxinc.com), a biopharmaceutical manufacturing company based here, recently received a substantial Louisiana R&D Tax Credit award.
The credit, administered by Louisiana Economic Development and available through the 2019 tax year, allows for small businesses to receive refunds equal to 40 percent of qualified research expenses. The goal of the program is to encourage companies to make investments in Louisiana to innovate and grow their businesses. Working with Baton Rouge-based tax credits and incentives advisory firm TRCG Advisors (trcgadvisors.com), TranGenRx Inc. was awarded one of the largest R&D Tax Credits in the history of the program.
"The R&D tax credit has allowed us to take the steps we need for our business to grow here in Baton Rouge," said Richard Cooper, president and chief scientific officer of TranGenRx Inc. "Our goal is to hire more highly skilled employees, and we thank LED Secretary Stephen Moret, Louisiana R&D tax credit program administrator Shamelda Pete and the entire LED staff for working with us toward achieving these goals."
"It was gratifying to see LED award the R&D tax credits to such a deserving company. It is an honor to work with TransGenRx. We look forward to watching TranGenRx use the R&D credits to fuel even greater growth here in Louisiana," said Adam Weems, managing director of TRCG Advisors.
TransGenRx Inc. was started with technology licensed from the LSU AgCenter. Cooper also is a professor in the AgCenter's School of Animal Sciences.
Linda Foster Benedict
Email Linda Benedict


text:
import User from '../model';
import R from 'ramda';

export default function handler(request, reply) {
  User.Collection
    .forge()
    .fetch()
    .then(users => JSON.parse(JSON.stringify(users)))
    .then(R.map(User.cleanSensitiveData))
    .nodeify(reply);
}

redpajama_set_name: RedPajamaGithub


text:
The American Presidential Nomination Process
Category: American History
Type of paper: Essay
Hey! We can write a custom essay for you.
All possible types of assignments. Written by academics
In recent years, the presidential primary season has served as a backdrop to instances of human drama that have illuminated man's ever-hopeful nature, yet also revealed his capability for cruelty. In 2000, for example, a malicious scene unfolded in South Carolina as John McCain fell victim to a vicious smear campaign that essentially ended his bid for the Republican nomination in what the New York Times later described as "a painful symbol of the brutality of American politics." Eight years later, conversely, amid growing controversy over comments made by his pastor, Barack Obama delivered an eloquent speech on the state of race relations in contemporary America. He described ugly vestiges of Jim Crow discrimination but also articulated his hope for a bright future in which society is not "irrevocably bound to a tragic past." The speech endeared him to countless Americans of all races, and helped put his candidacy back on track and eventually became the first African-American president of the United States. With such displays of triumph and controversy, it is unfortunate that many citizens are utterly confused by the procedure of nominations, and therefore, become disengaged from the political process. One explanation is that many people believe the nomination process is excessively complicated, anachronistic, and arbitrary, rendering some voters more powerful than others. As a result of this dissatisfaction, many legitimate reform proposals have been promulgated, but none yet have been instituted. Consequently, many of the nagging flaws persist.
One significant problem with the contemporary system is the monopoly that is given to Iowa and New Hampshire in conducting the first caucus and primary, respectively. Every presidential election cycle, these two diminutive states exert an inordinate amount of influence over the nomination process. Candidates who perform well in these states can gain a great advantage and can segue a victory, or even just a better-than-expected showing, into great momentum with respect to fund-raising, media attention, and victories in later primaries. Politicians understand this and begin campaigning in these states as much as a year in advance. They listen attentively to the problems and needs of these two state's citizenry, which can sometimes morph into outright pandering to the special interests of the Granite and Hawkeye states. There is "no constitutional basis" (Lansford 39) for this advantageous position but is rather a mere consequence of historical precedence. Moreover, Iowa and New Hampshire are not representative of the country as a whole, further exacerbating the problem of them being the bellwethers of presidential hopefuls. For instance, African-Americans constitute 12.3 percent of the United States but only 2.1 percent of Iowa and 0.7 of New Hampshire (U.S. Census Bureau, 2002). They have similar under representation of every major minority group and also posses a more rural population and experience less poverty than the rest of the nation. Therefore, a candidate who has great appeal to urban areas with diverse populations does not compete on a level playing field in New Hampshire and Iowa. Larry Sabato, director of University of Virginia's Center for Politics, succinctly writes, "Put simply, why should two tiny, heavily white, disproportionately rural states have a staggeringly powerful and electorally imbalanced say in the making of the next president? Answering that question is easy enough: They shouldn't" (Sabato 143).
Warning! This essay is not original. Get 100% unique essay within 45 seconds!
GET UNIQUE ESSAY
We can write your paper just for 11.99$
i want to copy...
This essay has been submitted by a student and contain not unique content
Straight Forward, Familiar, and Easily Defined
The Scientific Experiment Lab 1 - the Scientific Method
The Scope of Practice
The Life of an Infant
The Legal Form of Enterprises
World War 2 1939-1945
The Business and Financial Performance of an Oranganization over Three Years
thebillhudson.com © 2022 All rights reserved


There may be scary monsters under the bed but there's nothing to fear when it comes to our Furry Little Monster Dome Tent! Our 100% polyester dome tent features a playful monster design to stimulate your child's imagination and channel her inner "friendly monster!" Measuring in at 48" x 48" x 42", this furry tent includes color coated fiber glass poles and a tunnel port with hook and loop fasteners. Instead of doors, our Friendly Monster Dome Tent has funky ears, feet, teeth, and tail appliques. Let the adventures begin!
There may be scary monsters under the bed but there's nothing to fear when it comes to Izzy the Friendly Monster Dome Tent! Our 100% polyester dome tent features a playful monster design to stimulate your child's imagination and channel her inner "friendly monster!" Measuring in at 48" x 48" x 42", this tent includes color coated fiber glass poles and a tunnel port with hook and loop fasteners. Instead of doors, our Friendly Monster Dome Tent has funky ears, feet, teeth, and tail appliques. Let the adventures begin!
There may be scary monsters under the bed but there's nothing to fear when it comes to Sparky the Friendly Monster Dome Tent! Our 100% polyester dome tent features a playful monster design to stimulate your child's imagination and channel his inner "friendly monster!" Measuring in at 48" x 48" x 42", this tent includes color coated fiber glass poles and a tunnel port with hook and loop fasteners. Instead of doors, our Friendly Monster Dome Tent has funky ears, feet, teeth, and tail appliques. Let the adventures begin!
Planes are ready to take your little wanderer on a learning adventure. Bring juice and snacks inside because this tent comes fully equipped with a waterproof floor, protecting your precious floors. With mesh side panels and a T-style door, your child will have plenty of air ventilation. There is also a tunnel port so many of our tunnels can be connected for more imaginative playtime fun! What are you waiting for? Bring your friends inside and explore what's high in the sky!


text:
Q: Do we need more than one Pager(viewpager) class, if we want to add tabs in more than one activity? I have to make project which consist of 3 Activities. 1st and 2nd activity consist of 2 Tabs while 3rd activity needs 3 tabs. Do i need to make 3 Pager clases for each activity? Or if One Pager.class can handle all tabs then how we can sequence it??
Thanks to all In Advance!

A: You can use the same class. Just make sure your overridden methods can differentiate which activity it is in. So getCount() in FragmentPagerAdapter as an example would return two in your 1st and 2nd activity, but three in your 3rd one. Same goes for layouts.

A: yes, you can use this pageradapter class for each activity. it will create size that you want in a activity
public class ExamplePagerAdapter extends FragmentPagerAdapter {
        private final List<Fragment> mFragmentList = new ArrayList<>();
        private final List<String> mFragmentTitleList = new ArrayList<>();

    public ExamplePagerAdapter(FragmentManager manager) {
        super(manager);
    }

    @Override
    public Fragment getItem(int position) {
        return mFragmentList.get(position);
    }

    @Override
    public int getCount() {
        return mFragmentList.size();
    }

    @Override
   public CharSequence getPageTitle(int position) {
       return mFragmentTitleList.get(position);
    }

   public void addFragment(Fragment fragment) {
        mFragmentList.add(fragment);
        mFragmentTitleList.add(title);
    }
}

A: You can use same PagerAdapter class. Below is an sample PagerAdapter.
TabsPagerAdapter.java
import android.support.v4.app.Fragment;
import android.support.v4.app.FragmentManager;
import android.support.v4.app.FragmentPagerAdapter;

import java.util.ArrayList;
import java.util.List;

public class TabsPagerAdapter extends FragmentPagerAdapter {

    private final List<Fragment> mFragmentList = new ArrayList<>();
    private final List<String> mFragmentTitleList = new ArrayList<>();

    public TabsPagerAdapter(FragmentManager manager) {
        super(manager);
    }

    @Override
    public Fragment getItem(int position) {
        return mFragmentList.get(position);
    }

    @Override
    public int getCount() {
        return mFragmentList.size();
    }

    public void addFrag(Fragment fragment, String title) {
        mFragmentList.add(fragment);
        mFragmentTitleList.add(title);
    }

    @Override
    public CharSequence getPageTitle(int position) {
        return mFragmentTitleList.get(position);
    }
}

From your FirstActivity.java:
TabLayout mTabLayout;
ViewPager mViewPager;
TabsPagerAdapte mAdapter;

@Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_first);

    // Tab layout
    mTabLayout = (TabLayout) findViewById(R.id.tabs);

    // View pager
    mViewPager  = (ViewPager) findViewById(R.id.view_pager);

    // Adapter
    mAdapter = new TabsPagerAdapter(getSupportFragmentManager());

    // 2 Tabs
    mAdapter.addFrag(new FragmentOne(), "ONE");
    mAdapter.addFrag(new FragmentTwo(), "TWO");

    mViewPager.setAdapter(mAdapter);

    mTabLayout.setupWithViewPager(mViewPager);
    mTabLayout.setOnTabSelectedListener(new TabLayout.OnTabSelectedListener() {
        @Override
        public void onTabSelected(TabLayout.Tab tab) {

            mViewPager.setCurrentItem(tab.getPosition());
        }

        @Override
        public void onTabUnselected(TabLayout.Tab tab) {

        }

        @Override
        public void onTabReselected(TabLayout.Tab tab) {

        }
    });
}

From your SecondActivity.java:
TabLayout mTabLayout;
ViewPager mViewPager;
TabsPagerAdapte mAdapter;

@Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_second);

    // Tab layout
    mTabLayout = (TabLayout) findViewById(R.id.tabs);

    // View pager
    mViewPager  = (ViewPager) findViewById(R.id.view_pager);

    // Adapter
    mAdapter = new TabsPagerAdapter(getSupportFragmentManager());

    // 2 Tabs
    mAdapter.addFrag(new FragmentOne(), "ONE");
    mAdapter.addFrag(new FragmentTwo(), "TWO");

    mViewPager.setAdapter(mAdapter);

    mTabLayout.setupWithViewPager(mViewPager);
    mTabLayout.setOnTabSelectedListener(new TabLayout.OnTabSelectedListener() {
        @Override
        public void onTabSelected(TabLayout.Tab tab) {

            mViewPager.setCurrentItem(tab.getPosition());
        }

        @Override
        public void onTabUnselected(TabLayout.Tab tab) {

        }

        @Override
        public void onTabReselected(TabLayout.Tab tab) {

        }
    });
}

From your ThirdActivity.java:
TabLayout mTabLayout;
ViewPager mViewPager;
TabsPagerAdapte mAdapter;

@Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_third);

    // Tab layout
    mTabLayout = (TabLayout) findViewById(R.id.tabs);

    // View pager
    mViewPager  = (ViewPager) findViewById(R.id.view_pager);

    // Adapter
    mAdapter = new TabsPagerAdapter(getSupportFragmentManager());

    // 3 Tabs
    mAdapter.addFrag(new FragmentOne(), "ONE");
    mAdapter.addFrag(new FragmentTwo(), "TWO");
    mAdapter.addFrag(new FragmentThree(), "THREE");

    mViewPager.setAdapter(mAdapter);

    mTabLayout.setupWithViewPager(mViewPager);
    mTabLayout.setOnTabSelectedListener(new TabLayout.OnTabSelectedListener() {
        @Override
        public void onTabSelected(TabLayout.Tab tab) {

            mViewPager.setCurrentItem(tab.getPosition());
        }

        @Override
        public void onTabUnselected(TabLayout.Tab tab) {

        }

        @Override
        public void onTabReselected(TabLayout.Tab tab) {

        }
    });
}

Hope this will help~

redpajama_set_name: RedPajamaStackExchange


The Moldovian broadcaster offered Romania's 2016 disqualified representative Ovidiu Anton to compete together with their hopeful Lidia Isac at the 2016 Eurovision Song Contest in Stockholm with just one week until rehearsal starts.
Friday's breaking news of Romania's 2016 disqualification has been a big shock for Eurovision fans all over Europe. Due to high debts, the Romanian broadcaster TVR was excluded from all EBU member services, which included being able to take part in the Eurovision Song Contest. Now, the Moldovan broadcaster M1 has tried to save the Eurovision experience for Romania's Ovidiu Anton by proposing him that they make the Moldovian entry into a duet between Lidia Isac and him.
According to an official document, the broadcaster believed in the plans of making Falling Stars a duet between the two singers.
I would like to thank our brothers from Moldova very much. Your gesture is incredibly generous and unique.
I know how much energy and work is behind a Eurovision participation. Therefore I can't risk to put Lidia Isac's work in danger or to possibly harm her performance. Her song is very beautiful and it fits to her voice perfectly. I really wished to be on stage but I can't assume the risk to put all of Lidia's work in danger.
At the same time I would like to sing my song Moment Of Silence in Stockholm for all of you in Stockholm, Romanians, Moldovans, and all Eurovision fans in general.
Best of luck and success Lidia, it's your time to shine. Thank you very much for your generous offer.
It further remains doubtful if this would even have been possible with the rehearsals starting in Stockholm in less than a week, on Monday the 2nd of May. First the Swedish producers would have to decide if they think they could cope with this late change, and should they approve it would then be up to the European Broadcasting Union if they would allow it.
Below you can check out Lidia Isac's live performance during Amsterdam's Eurovision In Concert. Would this entry work for two vocalists as well?


text:
AMINO ACID AND VITAMIN CONTENT OF PROPOL...
AMINO ACID AND VITAMIN CONTENT OF PROPOLIS COLLECTED BY NATIVE CAUCASICAN HONEYBEES
Eroğlu N., Akkuş S., Yaman M., Asci B., SİLİCİ S.
JOURNAL OF APICULTURAL SCIENCE, cilt.60, ss.101-109, 2016 (SCI İndekslerine Giren Dergi)
Doi Numarası: 10.1515/jas-2016-0021
Dergi Adı: JOURNAL OF APICULTURAL SCIENCE
The polyphenol content of propolis has received a lot of attention due to the benign biological properties noted in the chemical composition studies. However, there are very limited studies about other chemical components found in trace amounts in nature which contribute to the therapeutic properties of propolis. The present study, therefore, investigated the amino acid and vitamin composition of propolis. Propolis harvested by 60 colonies of Apis mellifera caucasica belonged to local non-migratory beekeepers. The A. m. caucasica is known for its distinctive propolis collecting capability which native to the secluded Ardahan Province of Turkey. Vitamin (Thiamine, Riboflavin) combinations of propolis were determined using the HPLC (High-Performance Liquid Chromatography) fluorescent detector. An amino acid analysis was also performed with the UFLC (Ultra-Fast Liquid Chromatography) system consisting of binary pump and UV/VIS. Our findings record that the vitamin and amino acid content of propolis samples collected from three areas of different altitudes in the same region differed from each other. Vitamin 61 content and Vitamin B2 content ranged between 0.025-0.16 mg/100g, and 0.304-0.777mg/100g, respectively. A maximum amount of amino acid was reported as leucine, while a minimum amount of amino acid was seen as tryptophan in Ardahan propolis. Consequently, the vitamin and amino acid content of propolis, which derived from secondary plant metabolites of resin, varied depending on their geographical altitudes. Those vitamin and amino acids found in the propolis composition are believed to have beneficial therapeutic properties.


Watch out for revenue when purchasing Platinum Sleeper Sofa By A&J Homes Studio. Most of the time furnishings are placed on purchase. Should you need a specific part, you can save lots of money by with patience expecting the cost to get minimized. You might even ask your favorite Platinum Sleeper Sofa By A&J Homes Studio retail store when the object will be designated on the cheap. If you want a new couch, try out locating a single with some completely included soft cushions. These cushions keep going longer and might be switched routinely. Preferably, you should buy a settee with easily-removed addresses in order to very easily wash them. Removable handles may also be replaced quickly in the event you cannot clean a spot Buy Platinum Sleeper Sofa By A&J Homes Studio in the future around. At the conclusion of this current year, it is actually typical for stores to have these parts off of their display room ground. This can result in some great product sales and savings.
If there is any one factor that you could change to make investing eight hours each day within an workplace easier, its your seat. There isn't any shortage of evidence proving that becoming caught inside a chair for too long can increase the risk of cardiovascular disease and worsen lower back pain. In fact, seated an excessive amount of is even known as even worse than cigarette smoking. Many those who have office work develop issues like pins and needles, spine misalignment, joint pain, neck discomfort, and slipped discs usually from seated too long on a low quality chair with out support. While you most likely don't want to spend lots of money on the seat, a higher-high quality ergonomic chair is an purchase of your wellbeing, comfort and ease and efficiency. Two separate research, released in 1990 and the year 2003, discovered efficiency goes up greater than 17% when folks operate in an ergonomic setting with an flexible chair. The right chair combined with ergonomic training may also decrease workplace accidents. Even if you do not curently have arm or back again problems, an ergonomic chair will help you maintain the right position to prevent stress, carpel canal affliction, low back pain, and spinal disk injury. The Nation's Start of Health suggests choosing a seat with all the required alterations in assistance correct posture. Including a chair with casters along with a 5-stage foundation a chair skillet with thick, small-cell froth padding or circles a back rest that is either bent or small enough to fit the little of the back again soft arm rests by having an adjustable height and width a gas chair heat realignment feature and a tilt realignment to transfer some body weight towards the seats back rest.
This is definitely the most adorable and ingeniously designed loveseat. Contingent within the tone, it gives a disturbing look to your office or rooms. The sofa unimaginably imperceptible for a early morning java or perhaps an animated journal reading carefully. It is accessible both in leather and in consistency depending on what you support a bright appear, or perhaps a lovely, lovely view. The good thing, it is very reasonable! The super chic tufting is shaped stone, which holds an exquisite Chesterfield style. Probably the most lovable point about this sofa is its form. Along with the appeal, the trunk design provides the back understanding of rest. It is cleverly designed to ensure that smaller sized rooms combine luxury inside a smaller sized room. It is a clever decision for the childrens room, because it provides a number of sweet shades. They would like to hit them for reading the storyline or a fantastic dream. No hassle for grown ups as well for any comfy night with Manga or Marvels.
This fits you if all you need is a beautiful trainer. It's made with a flared frame and pillow top amrests in a awesome cobblestone gray. Now, it's resilient froth soft cushions which are obsessed with inside a rayon furniture to create the desired comfort. It has a durable part obstructed body that increases the toughness. Also, the feet are in a faux wooden finish. It has an remarkable gray colour that matches with any decoration. Its dimensions are 89 W x 39 Deb x 40 They would therefore large enough to support you together with your familyOrfriends. More importantly, it occurs completely put together. This saves the pain of having to put together the established.
When it comes to purchasing room furniture, leather-based is definitely a good option. Not only does it look good with many designs, nevertheless its very long lasting (it is the ideal materials for any home with children or pets) and its ultra-simple to clean, too. The down-side of leather furniture? It can have a much higher price tag than material, micro-fiber or faux leather furniture. This reclining loveseat remains a financial budget-pleasant option because of one guru trick: its sitting area is padded with leather, while the sides are padded with more affordable fake leather. That means you get the appear and feel of a full leather loveseat without the significant cost. Make use of the handle around the loveseats equip to relax and relax on its higher-denseness foam filling up, and think about the money you saved. This particular loveseat posseses an additional bonus: expert assembly is available in many areas of the country for an additional fee.
This luxurious group of urban-stylish sofas is made to provide an embellished domain. In fact, even your modest rooms might have you required to have an immaculate film evening. Lay him lower to get a totally altered mattress. This avant-garde person futon is accessible in various shades. This includes some bright tones out of control for example Lemon, Mild Blue, and Light Purple. Place them in your inventive research or loft to assess the exuberance. It's also accessible in decreasing shades for example dark, dark and dark. These subtleties, once again, printing an honorable part of the higher class. With the breaking component of your back, you can rest to your favorite position. This couch looks rich with tufted fabric consistency upholstery. Fortunately that it is personalized with hypoallergenic filling up. A conclusive outcome supplies a comfortable rest.
This established is upholstered inside a push and comfy textured cushioned velvet. This provides it the amazing comfort and ease that it is cherished for. Again it facilitates chaise style seating for sufficient comfort and ease. Moreover, the sinus spring base increases the durability and comfort. Additionally, the couch has hard wood frames that increase its sturdiness and durability. The sofa has cushions on both the seat and also the back. But still, the couch can move from a seated placement to some lying down position easily. That said, simply to replicate, the sofa is very gentle and gives exceptional comfort and ease.
Fluffy, overstuffed loveseat soft cushions are not your lifestyle? This reclining loveseat from GDF has sharper, cleaner lines than your conventional reclining loveseat, which makes it a stylish addition to a mid-century contemporary, contemporary, eclectic or modern type of home. And, at only 46.46 in . in width, it's very easy to squeeze into a smaller space, like flats, dog dens or office spaces. Obtainable in several color and material choices, such as charcoal fabric, navy blue fabric and slate micro-fiber, its easy to find the right match for your home decorations, and also the durability your family requirements for daily use. Gone are the days of good-looking, but unpleasant furniture you will no longer have to sacrifice comfort and ease for style. This reclining loveseat functions a big, soft, luxurious-filled seat cushioning with enough room for 2 or enough space for one to disseminate and unwind.
The best part that compensates for that lost time the consideration of the purchasers is its style without problems. Just for the feel of the upholstery, you receive a nice tasteful atmosphere. It's available in rich strong shading, imitating as incredible official clothing. While keeping an honorable prominence, the material furniture could be washed effortlessly. You should not rue after turning over a glass of consume in your nice sofa. All you need is to wash the spot without end exactly when it hits the furniture. Even though great repute comes with delicacy, it's not the situation here. This amazing sofa is fed having a contour growth and development of wood. You are much more reluctant to listen to squeaks and walking seems.
Even though this reclining loveseat has a heftier cost, its classic style, choices for customization and automated lying system make it worth the money. Its hand made wood body is cushioned with lower blend soft cushions for any much softer feel than you would get from high-density foam soft cushions, and also the furniture will come in an astonishing 72 colour and material combinations. The availability of so many choices causes it to be additional simple to pair this lying loveseat with different patterns, materials, accents and decorations designs. Plus, if you have children or animals (or each!), you can select a more durable, spot-proof fabric. One customer shared that the Sunbrella canvas materials has organized well against her children and pets. This lying loveseat could have a classic, old-school appear, nevertheless its built-in tech is anything but. Not only does it have an automatic reclining system (that will give you a full lie down, together with your footrest completely similar to the seat), but it features a USB interface for using or simply charging your iPad, iPhone, Kindle or laptop.
The good thing that compensates for the lost time the glory of the buyers is its style effortlessly. Only for the feel of the upholstery, you get a nice stylish atmosphere. It's available in wealthy solid shading, emulating as amazing official attire. While maintaining an honorable stature, the fabric upholstery can be washed very easily. You shouldn't rue after generating a glass of consume in your good couch. All you need is to wash the spot for ever and ever exactly if this hits the furniture. Despite the fact that great repute comes with delicacy, it is not the problem right here. This amazing couch is given with a shape development of solid wood. You're more unwilling to listen to squeaks and walking seems.
Whilst the rear of this sofa shows the Chesterfield, with its series of tufting, a tuxedo sofa has cleaner, more angular lines. It is said to have been the bellwether more modern designs within the 1920s. While some say this sofa style takes a title from Tuxedo Recreation area in Ny, it's also considered named after the traditional fancy men's match. You can identify a tux sofa by its arms that are the same height because the back again. The tufting on the rear of the sofa and its rectangle-shaped outline are also traditional characteristics. Cushions add comfort and ease to this high-armed design. Also, while the couch above from Upcountry is upholstered in leather-based, this popular style is usually completed in textiles of all kinds, such as the very stylish velvet.
Copyright © Platinum Sleeper Sofa By A&J Homes Studio in Nesting Tables All right reserved.


text:
Facebook LinkedIn GMBs Instagram
Leave Review on Google Leave Review on Facebook Check out what our clients say
Call Now! 24/7 Emergency Service
Disaster restoration in High Point, North Carolina
Emergency Restoration and Cleanup Services in High Point, North Carolina
High Point is a city located in the Piedmont Triad region of the U.S. state of North Carolina. Most of the city is located in Guilford County, with portions extending into neighboring Randolph, Davidson, and Forsyth counties. High Point is North Carolina's only city that extends into four counties. As of the 2010 census the city had a total population of 104,371, with an estimated population of 112,791 in 2019. High Point is currently the ninth-largest municipality in North Carolina, and the 259th-largest city in America.
High Point is the only city in North Carolina that exists within four counties: Davidson, Forsyth, Guilford and Randolph. It also stands within two major watersheds: the Yadkin–Pee Dee to the west and the Cape Fear to the east. Parts of the city rise above 1,000 feet (300 m), making it among the highest cities in North Carolina's Piedmont. According to the United States Census Bureau, the city has a total area of 55.4 square miles (143.6 km²), of which 53.8 square miles (139.3 km²) is land and 1.7 square miles (4.3 km²), or 2.96%, is water.
As of the census of 2000, there were 3,557 people, 1,452 households, and 985 families residing in the city. The population density was 997.4 people per square mile (384.7/km²). There were 1,542 housing units at an average density of 432.4 per square mile (166.8/km²).
Serving High Point, North Carolina
out of 5 ratings
Default Silver Night mode Dark Retro Aubergine Hide features
State Select state Alabama Alaska Arizona Arkansas California Colorado Connecticut Delaware District Of Columbia Florida Georgia Hawaii Idaho Illinois Indiana Iowa Kansas Kentucky Louisiana Maine Maryland Massachusetts Michigan Minnesota Mississippi Missouri Montana Nebraska Nevada New Hampshire New Jersey New Mexico New York North Carolina North Dakota Ohio Oklahoma Oregon Pennsylvania Rhode Island South Carolina South Dakota Tennessee Texas Utah Vermont Virginia Washington West Virginia Wisconsin Wyoming
Best Option Restoration of High Point is committed to serving you, no matter the disaster. Our team is chalked with experience, knowledge, and the equipment necessary to make sure your home is properly restored
We have fielded thousands of calls and recovered just about any situation out there in a professional and compassionate matter
We maintain strict adherence to OSHA rules and regulations to ensure the safety of your family, our workers and the public
We are Insured
We are proud to have a great working relationship with the Police and Fire departments, as well as major insurance companies.
PROUDLY SERVING THESE FINE TOWNS AND THE SURROUNDING COMMUNITIES
Randleman
Best Option Restoration of High Point is part of Best Option Restoration Network, a trusted national leader in the disaster restoration industry. If you are interested in becoming a Best Option Restoration franchise owner, request a Franchise Discovery Kit.
265 Eastchester Drive Suite 133 #174
High Point, North Carolina 27262
© 2023 Best Option Restoration of High Point Powered by SeoSamba Franchise Brand Marketing Services
High Point phone
Do you want to get notifications from this site?


text:
Posts tagged "Iran"
Case of the Day: Maalouf v. Iran
The case of the day is Maalouf v. Iran (D.C. Cir. 2019). The cases arose out of the 1983 and 1984 bombings of US diplomatic missions in Beirut by Hizbullah and the 1998 bombings of the US embassies in Kenya and Tanzania by Al Qaeda. The Iranian government has been linked to all of the bombings, and the Sudanese government to the bombings in Africa. I wrote about one of the cases in the lower court, Kinyua v. Sudan, about a year ago, and about Maalouf itself in April 2018.
Tagged: FSIA, Iran, Sudan
Case of the Day: Azadeh v. Iran
The case of the day is Azadeh v. Government of the Islamic Republic of Iran (D.D.C. 2018). Asfaneh Azadeh sued the Iranian government, asserting she was wrongfully imprisoned for months and was tortured in prison. She served process via diplomatic channels under 28 U.S.C. § 1608(a)(4), without first attempting service on the Iranian foreign ministry under 28 U.S.C. § 1608(a)(3). The court's guidebook for service abroad seemed to permit this, saying: The countries of Iran and Iraq have not objected to service by mail. However, many attempts at service by mail or courier are unsuccessful. Therefore, it is okay for an attorney to request service directly through diplomatic channels (28 U.S.C. § 1608(a)(4)) without attempting service under any other provision first.
Tagged: FSIA, Iran
Case of the Day: Kinyua v. Sudan
The case of the day is Kinyua v. Republic of Sudan (D.D.C. 2018). The case arose out of the bombing of the US embassies in Kenya and Tanzania in 1998. Since 2001, victims and their families had been suing Iran and Sudan, state sponsors of the terrorist attacks, under the FSIA's state-sponsored terrorism exception to the rule of foreign sovereign immunity. The first round of cases resulted in default judgment against Sudan and Iran. Kinyua and others then sued, bringing similar claims. By that time, Sudan, which had defaulted earlier, was again participating in the FSIA cases against it. It moved to dismiss the action on the grounds that it was barred by the statute of limitations. The terrorism exception requires actions to be…