At Loftel Station Hostel, the excellent service and superior facilities make for an unforgettable stay. While staying at the property, you can enjoy free Wi-Fi, linen, cleaning service, coffee & bar, key card system, 24 hour front desk and security with CCTV, ensuring you have the most comfort and safety during your stay with us.
Female Dormitory 8 beds with A/C and shared bathrooms (Bunk bed) which is perfect for female travelers to meet friends in a relaxed and secure female only environment. Personal reading light, socket and locker ladies-only bathroom.
Mixed Dormitory 10 beds with A/C and shared bathrooms hot shower (Bunk bed) which offer you the opportunity to mix with other travelers to meet friends. Personal reading light, socket and locker. This room is no window.
Mixed Dormitory 6 beds with A/C and shared bathrooms hot shower (Bunk bed) which offer you to mix with other travelers to meet friends. Personal reading light,socket, locker. There are rooms with and without window, It's depend on room and bed available.
Private Family room for 4 person with A/C and shared bathrooms (Bunk bed). Family room is perfect for the family to share the backpackers experience with the comfort of their private room. No extra bed. There are rooms with and without window depend on room available.
Private Family room for 6 person with A/C and shared bathrooms (Bunk bed). Family room is perfect for the family to share the backpackers experience with the comfort of their private room. No extra bed. There are rooms with and without window depend on room available.
Private double room with A/C, television and shared bathrooms (queen size bed). This is perfect for couples looking to share the backpackers experience with the their own private room. No extra bed. There are rooms with and without window depend on room available.
Private Twin room (Bunk bed) with A/C and shared bathrooms with hot shower. Twin room is perfect for those friends looking to share the backpackers experience with the comfort of their own private room. No extra bed.
Female Dormitory 8 beds with A/C and shared bathrooms (Bunk bed) which is perfect for female travelers to meet friends in a relaxed and secure female only environment. Personal reading light, socket and locker ladies-only bathroom. Rate include with breakfast.
Mixed Dormitory 10 beds with A/C and shared bathrooms hot shower (Bunk bed) which offer you the opportunity to mix with other travelers to meet friends. Personal reading light, socket and locker. This room is no window. Rate include with breakfast.
Mixed Dormitory 6 beds with A/C and shared bathrooms hot shower (Bunk bed) which offer you to mix with other travelers to meet friends. Personal reading light,socket, locker. There are rooms with and without window, It's depend on room and bed available. Rate include with breakfast.
Private Family room for 4 person with A/C and shared bathrooms (Bunk bed). Family room is perfect for the family to share the backpackers experience with the comfort of their private room. No extra bed. There are rooms with and without window depend on room available. Rate include with breakfast.
Private Family room for 6 person with A/C and shared bathrooms (Bunk bed). Family room is perfect for the family to share the backpackers experience with the comfort of their private room. No extra bed. There are rooms with and without window depend on room available. Rate include with breakfast.
Private double room with A/C, television and shared bathrooms (queen size bed). This is perfect for couples looking to share the backpackers experience with the their own private room. No extra bed. Rate include with breakfast.
Private Twin room (Bunk bed) with A/C and shared bathrooms with hot shower. Twin room is perfect for those friends looking to share the backpackers experience with the comfort of their own private room. No extra bed. Rate include with breakfast.
Private double room with A/C, television and shared bathrooms (queen size bed). This is perfect for couples looking to share the backpackers experience with the their own private room. No extra bed. This room is no window.
- Check In : 14.00 PM / Check Out : 12.00 PM (NOON) / Reception opening hours: 7:00 AM - 20:00 PM / Staff waiting for check-in 24 H.
- A passport or government photo ID and FULL payment and is required upon check in.
- A key card deposit 200 baht is required upon check in. and will be return when you check out.
- All room are air-condition and shared bathrooms with hot shower.
- Due to the characteristic of our building. We do not have lift or elevator.
- Non-Smoking policy throughout the hostel. Smoking is permitted in designated area only. Any violation is subjected to a fine of THB 2,000.
- There are rooms with and without window depend on a room and bed available.
- We provide lockers. Please, bring your own padlock or purchase with us.
- 2 days advance notice for free cancellation, After that we have to charge 100% for 1 night of your booking.
* Loftel Station Hostel accepts no liability and shall not be responsible for any loss or damage of your belongings. Guest will be liable for any loss, damage and remain solely responsible for safekeeping of any such items.
Full payment is required upon check in. We accept cash or credit card accepted with 3% charge. Credit card information is only required to guarantee your booking and in case of Cancellation and No show.
2 days advance notice for free cancellation, After that we have to charge 100% for 1 night of your booking. No-show case also we have to charge 100% for 1 night of your booking.


Fairmont Dubai is close to popular attractions including the Burj Khalifa, the tallest freestanding structure in the world and the Dubai fountain, whose choreographed, illuminated performances will take your breath away. Nearby you'll also find Dubai shopping mall, golf clubs and Jumeirah public beach. It is just steps away from the World Trade centre metro and directly linked via an air-conditioned bridge to the building.
The hotel has a fully equipped health club with two rooftop swimming pools. It is also linked to the metro system, with direct service to the Dubai International Airport (Terminal 1 and 3).
The 40,000 sq ft Fairmont Spa has separate male and female sections. They both feature an opulent Jacuzzi area with Turkish Hammam inspired steam rooms, invigorating ice shower, two relaxing footbaths and a total of eight private treatment and relaxation rooms.
Classic elegance and contemporary décor are reflected throughout the guest rooms. Each combines timeless style with generous comfort, and is complemented by stunning panoramic views of the city.
Fairmont Gold guest rooms have bespoke amenities, four or five piece bathrooms and full Fairmont Gold services and amenities. This private dedicated area offers you the experience of an exclusive oasis, within the bustle of the hotel.
Gold Angus beef and Stockyard Wagyu beef.


text:
Call of Duty: Infinite Warfare launches 'Willard Wyler's Halloween Scream' event
By Joe Donnelly 2017-10-10T10:30:10.282Z
With new modes, maps and giveaways.
After a somewhat unsuccessful foray into the future, the next and imminent Call of Duty instalment will return to World War 2. Before it does, though, the latest series entry seems determined to squeeze some fun out of a game that otherwise punishes improvising players—something underscored by its newly-launched 'Willard Wyler's Halloween Scream' event.
Set to run now through November 1, the Halloween special includes giveaways, double XP periods, a new zombies mode and a new 'Carnage map', among other things. As you might expect, it comes with a trailer. Look, see:
And here's publisher Activision with some of that in more detail:
Free Halloween-themed Supply Drop at the beginning of each week.
Free Halloween-themed cosmetic item every Friday, along with one final gift on 30th October.
Halloween-themed gear and loot, including brand-new zombie-themed Rig skins, available to earn.
Boss Battle mode, which starts on 13th October, allows players to take on their favourite final bosses from each of the Infinite Warfare zombies modes. It begins with the first chapter, Zombies in Spaceland, and continues with the following chapters becoming available, two per week, ending with the ultimate battle in The Beast from Beyond (players must own DLC packs to access DLC chapters).
Return of Gesture Warfare multiplayer mode to celebrate Halloween, which runs now through 12th October and allows fans to use gestures to humiliate and annihilate their opponents.
Carnage, a fan-favourite multiplayer map from the Retribution DLC pack, free to play and with double XP for all players from 12th October to 1st November.
More information on Call of Duty: Infinite Warfare's Willard Wyler's Halloween Scream event can be found here. Tyler found the base game to be below average last year—here's his review in full.

redpajama_set_name: RedPajamaCommonCrawl


Air Canada has adjusted its schedule through to April 30 to cover 98 per cent of its planned flying following Transport Canada's closure of Canadian airspace to Boeing 737 MAX aircraft operations. In compliance with the safety notice, Air Canada has grounded its 24 737 MAX aircraft and Boeing has advised that deliveries of its 737 MAX are currently suspended. Air Canada was expecting six new aircraft in March and April.
"The Boeing 737 MAX accounted for six per cent of Air Canada's total flying, but there is a domino effect from removing the 737s from our fleet that impacts the schedule and ultimately will impact some customers. We have been working very hard to minimize that impact," said Lucie Guillemette, Executive Vice President and Chief Commercial Officer at Air Canada.
"Customers who have travel plans between now and July can be reassured that we will keep them informed every step of the way as we revise our schedule. We have a deep global network and many partner airlines to provide solutions so serving our customers and minimizing any disruption is our first priority. We remain committed to delivering the same safe, reliable transportation customers expect from Air Canada. Customers can continue to book and travel on Air Canada with full confidence," said Ms. Guillemette.
National and Enterprise Rent-A-Car are giving travellers the chance to earn 5x the points when booking Emerald Club Rentals.
Valid from March 1-April 30, 2019, the offer includes 5x Cash-In Club points for every qualifying National or Enterprise rental that includes a valid Emerald Club member number. Five points per rental day are available for up to a maximum of 14 days. If you are not an Emerald Club member, contact your UNIGLOBE Travel agency to enroll in the program and start earning reward points.
Travellers can log in or enroll at cashinclub.com and use promotion code GREEN19 to register for 5x points. Promotion code must be registered at cashinclub.com in the Promotion Opportunities area on the member's home page prior to rental in order to qualify.
With Emerald Club, clients have the flexibility of selecting their own car and bypassing the rental counter for a seamless rental experience.
In January 2018 Canadians were added to Brazil's current e-visa system. The e-visa program eliminated the need for visitors to physically visit a visa centre in their own country to apply for and receive a tourist visa to travel to Brazil and was aimed at increasing visitation to the country.
The new developments come as part of a series of measures that Brazil has taken to facilitate visitor access to the country.
What's behind the new plan? Canada, the U.S., Australia and Japan are considered strategic for the development of tourism in Brazil and the country wants to do whatever it can to ease access.
Effective March 11, 2019 WestJet has updated their change and cancel penalty fee structure. The penalty amounts will not change. WestJet change penalty fees can be found here.
The new penalty structure will take effect for tickets purchased on or after March 11, 2019. Tickets issued prior to this date will follow the penalty structure in place at the time of ticket issuance.
The BusinessFlex and PremiumFlex fares still offer no change or refund penalty fees, as well as other benefits such as no fee on checked baggage and seats in the Business or Premium cabin.


Do you ever look at that old fridge or drier in your basement and wonder how you can get rid of it? Well scrapping these obsolete devices alongside other pieces of metal is the answer to your question. Not only will it help you de-clutter but also give you some extra cash. In addition, by selling scrap metal to metal recyclers you are greatly contributing to the going green movement. In this post, however, we will focus only on getting some money into your pocket.
As you consider turning your metal trash into a fortune, there are a few things to note. Some metals are worth the effort while others are not. The scrapping industry pays by the pound. So you need to consider scrap metals that can fetch lots of weight. However, there are those like copper that are not heavy but are still valuable. So here is a classification of the things you can scrap for money to make your work easy.
In simple terms, these are metals that are attracted by a magnet. Steel is an example of such metals. They usually do not fetch much in the scrap yard. Even though they are easy to find you'll need tons and tons to justify the effort and time that you've used.
These types of metal are not attracted by a magnet. They include copper, aluminium, lead, bronze, brass, and stainless steel. Copper in good condition is the most valuable metal in recycling. Aluminium comes second and can be found from cans, old doors, window frames, and gutters. Brass ranks third. However, it is quite heavy and will bring more cash if sold by the weight. You can also consider stainless steel even though it is priced low at the metal recyclers. Just make sure you have lots of it. Lead may be heavy but since it is claimed to be carcinogenic, it might be snubbed by some recyclers.
Most recyclers offer cash for scrap cars. However, don't be in a rush to give away your car wholesomely. There's a way to make the most money for scrap cars, which is to sell them in parts. Have a professional tear it down and then with his/her help identify some parts that are still in pristine order. Sell these separately as second-hand parts. The remaining junk can then pass as scrap metal.
Appliances such as refrigerators, ovens, cookers, microwaves, toasters suffer mechanical failure quite often. These can be worth a fortune provided you find a reputable recycler.
These contain more metallic parts than all the kitchen appliances. If you can find those old models with heavier material the better. They deliver a fortune when sold at the scrap yards.
Other things you can sell as scrap metal include old plumbing pipes particularly those made of zinc. Bicycles, paint cans, and broken tools are also worth considering.
You may have heavy and high-quality metals that need recycling. However, if you don't know where to scrap metal for money then you'll end up getting a raw deal. Once you have your heap of scrap metal, head over to the local scrap yard. The rule of the thumb is to always do your research. Compare the price per pound offered by various scrap yards for different metals. Make use of online price comparison tools to make your work easier. You might also consider a junkyard although they usually focus more on giving cash for damaged car, wrecked cars and car parts.
It's high time you started collecting the scrap metal strewn around your house and selling them. Whether they are in the form of an electrical appliance or damaged cars, scrap metal might just help you out of a financial fix. To get the most out of your scrap, ensure that you clean them to get rid of the surface rust.


text:
Posts published in "Health and Fitness"
JONES: Starting stamina slowly is right way to go
By The Citizen on November 9, 2022
Aloha Doug, In a few of your previous columns, you've mentioned the importance of including "stamina" exercises at the beginning of each workout session. Could…
JONES: Why your exercise should include cardio
By The Citizen on October 26, 2022
Aloha Doug: In your column from a few weeks ago, you mentioned the different types of activities, which actually qualify as "true and on-purpose" exercise.…
ERSOY: How to improve bone health with exercise
Most people know what they need to do to lose weight, although applying the knowledge can still be tricky. However, it's very hard to know…
JONES: Where should I exercise in Kaua'i?
Aloha Doug: I have been working out fairly consistently for my entire life. Sometimes I had access to the poshest of health clubs. Other times,…
ERSOY: Are we expecting too much of others?
By The Citizen on October 5, 2022
Coming back to visit my family and friends, after a long time living outside my own country where I was born, has made me realize…
Seats available for free Certified Nurse Aide training on Kaua'i
By The Citizen on September 16, 2022
LIHU'E — Kaua'i residents are invited to apply for a new, free hybrid (mix of online and in-person) program this fall for nurse aide training…
ROSEN: One-upping your ailments
I'm around many senior citizens since I live in a senior housing community, and almost all my friends are over 65. We all like to…
ERSOY: The simplest solution for your health
By The Citizen on September 7, 2022
Unfortunately we do not always know what the cause is for some of the health struggles that we may be facing. Most of the time…
ERSOY: Salt' we can't survive without it
By The Citizen on August 10, 2022
Sodium is an essential nutrient, which means that it's something that our body can not produce on its own, so we need to consume it.…
Lung cancer is the leading cause of cancer deaths in Hawai'i
By The Citizen on August 4, 2022
HONOLULU — Here in Hawai'i and across the nation, lung cancer is the leading cause of cancer deaths. New survey data now show that only…


Doxa Deo Bloemfontein hosts the Street store every year. Street store is a rent-free, premise-free, free pop-up clothing store for the homeless.The poor people of our city is given a dignified shopping experience while they are prayed for and given some refreshments. We would like to continue expanding our sites each year impact more people in our city.
As a church we are working into the Erlichpark and Heidedal areas with different social impact programs, spearheaded by some of our partners. We have teams every week that reaches out to the people in these areas.
We are committed to the singular idea inspired, encouraged and equipped leaders transform their communities. That is why Doxa Deo Bloemfontein hosts the GLS every year. The GLS consists of fresh, actionable and inspiring leadership content from a world-class faculty. The leaders we serve are not just full-time ministry staff, but leaders in all sectors of society committed to pursuing their grander visions—whatever they might be.
TREE stands for "Transforming the Educational Environment". TREE is Doxa Deo's answer to making a Christ-centred difference in schools. Doxa Deo believes that the answer to city transformation lies with the younger generation as they are the future leaders of South Africa.
We have the privilege of placing youth workers in Fichardpark primary, Fichardpark High and Bloemfontein High school. Some of our TREE leaders are also running a weekly program in Grey College to development future leaders.


text:
Bourg-Sainte-Marie est une commune française située dans le département de la Haute-Marne, en région Grand Est.

Géographie

Localisation

Bourg-Sainte-Marie est bordée par la Meuse. Elle est située à  de Bourmont, à  de Clefmont, à  de l'A31, à  de Neufchâteau et à  de Chaumont.

Urbanisme

Typologie
Bourg-Sainte-Marie est une commune rurale, car elle fait partie des communes peu ou très peu denses, au sens de la grille communale de densité de l'Insee.
La commune est en outre hors attraction des villes.

Occupation des sols

L'occupation des sols de la commune, telle qu'elle ressort de la base de données européenne d'occupation biophysique des sols Corine Land Cover (CLC), est marquée par l'importance des territoires agricoles (68,7 % en 2018), une proportion identique à celle de 1990 (68,7 %). La répartition détaillée en 2018 est la suivante :
prairies (43,7 %), forêts (31,3 %), terres arables (21 %), zones agricoles hétérogènes (4 %).

L'IGN met par ailleurs à disposition un outil en ligne permettant de comparer l'évolution dans le temps de l'occupation des sols de la commune (ou de territoires à des échelles différentes). Plusieurs époques sont accessibles sous forme de cartes ou photos aériennes : la carte de Cassini (), la carte d'état-major (1820-1866) et la période actuelle (1950 à aujourd'hui).

Histoire

Politique et administration

Population et société

Démographie

Évolution démographique

Pyramide des âges

Lieux et monuments
 L'église Notre-Dame de la Nativité, inscrite au titre des Monuments historiques le 12 mars 1987 .
 Abbaye de Corrupt.

Personnalités liées à la commune

Voir aussi

Bibliographie

Articles connexes
 Liste des communes de la Haute-Marne
 Liste des monuments historiques de la Haute-Marne

Liens externes

 Bourg-Sainte-Marie sur le site de l'Institut géographique national

Notes et références

Notes

Références

Bourg-Sainte-Marie
Localité sur la Meuse en France
Marie

redpajama_set_name: RedPajamaWikipedia


text:
\section{Introduction}
Teaching machines to learn programs is a challenging task. Numerous models have been proposed for learning programs, e.g. Neural Turing Machine \citep{graves2014}, Differentiable Neural Computer \citep{graves2016}, Neural GPU \citep{kaiser2015}, Neural Programmer \citep{neelakantan2015},
Neural Random Access Machine \citep{kurach2015} and Neural Programmer-Interpreter \citep{reed2016}. These models are usually equipped with some form of memory components with differentiable access. Most of these models are trained on program input-output pairs and the neural network effectively learns to become the particular target program, mimicking a particular Turing machine.

Of these models one notable exception is Neural Programmer-Interpreters (NPI) \citep{reed2016} and its extension that supports recursion \citep{cai2017} (referred to in this paper as RNPI). NPI has three components: a core controller that is typically implemented by a recurrent neural network, a program memory that stores embeddings of learned programs, and domain-specific encoders that enable a single NPI to operate in diverse environments. Instead of learning any particular program, the core module learns to \emph{interpret} arbitrary programs represented as program embeddings, mimicking a universal Turing machine. This integration of the core (interpreter) and a learned program memory (programmer) offers NPIs with better flexibility and composability by allowing the model to learn new programs by combining subprograms. Despite these merits, the NPI model bears some theoretical and practical limitations that hinder its application in real world problems.

One hypothetical theoretical property of the NPI model that makes it appealing for multi-task, transfer, and life-long learning settings is its \emph{universality}, i.e. the capability to represent and interpret \emph{any} program. As the NPI relies solely on the core to interpret programs, universality requires a fixed core to interpret potentially many programs. A universal fixed core is critical for learning and re-using learned programs in a continual manner, because a core with changing weights may fail to interpret old learned programs after learning new ones. Although the original NPI paper shows empirically that a single shared core can interpret 21 programs to solve five tasks, and that a trained NPI with fixed core can learn a new simple program MAX, it is unclear whether a universal NPI exists or how universal it could be. Specifically, given the infinite set of all possible programs, the subset of programs that can be interpreted by a fixed core is not explicitly defined. Even though a universal NPI exists, it may still be intractable to provable guarantee of universality by the verification method proposed in \cite{cai2017}, because there may be infinite programs to verify.

Practically, as proposed in \cite{reed2016}, the training of an NPI model relies on a very strong form of supervision, i.e. the example execution traces of programs. This form of training data is typically more costly to obtain than input-output examples. Training with weaker form of supervision is desirable to unlock NPI's full potential.

In this paper, we propose to overcome these limitations of NPI by incorporating combinator abstraction into the NPI model and augmenting the original NPI architecture with necessary components and mechanisms to support this abstraction. We refer to this new architecture as the Combinatory Neural Programmer-Interpreter (CNPI). As an important abstraction technique in functional programming, combinators, a.k.a. higher-order functions, are used to express some common programming patterns shared across different programs. We find that combinator abstraction can dramatically reduce the number and complexity of programs (i.e. combinators) that need to be interpreted by the core, while still allowing the CNPI to represent and interpret arbitrary complex programs by the collaboration of the core with the other components.
We propose a small set of four combinators to capture four most pervasive programming patterns. Due to the finiteness and simplicity of this combinator set and the offloading of some burden of interpretation from the core, we are able construct a CNPI with a fixed core that can represent and interpret an infinite number of programs which is adequate for solving most algorithmic tasks. This CNPI is universal with respect to the set of all combinatorizable programs. Moreover, we show empirically that besides supervised training on execution traces, it is possible to train the CNPI by policy gradient reinforcement learning with appropriately designed curricula.

\section{Overview of combinator abstraction}
\subsection{Review of NPI with its limitations}
In this section, we give a brief review of the NPI architecture from \cite{reed2016} and \cite{cai2017}. Then we analyze its limitations to motivate our combinator abstraction. The NPI model has three learnable components: a task-agnostic core controller, a program memory, and domain-specific encoders that allow the NPI to operate in diverse environments. The core controller is a long short-term memory (LSTM) network \citep{hochreiter1997} that acts as a router between programs. At each time step, the core can decide either to select another programs to call with certain arguments, or to end the current program. When the program returns, control is returned to the caller by popping the caller's LSTM hidden units and program embedding off of a program call stack and resuming execution in this context.

NPI's inference procedure is as follows (see Section 3.1 and Algorithm 1 in \cite{reed2016} for more detail). At time step $t$, an encoder $f_{enc}$ takes in the environment observation $e_t$ and arguments $a_t$ and generates a state $s_t$. The core LSTM $f_{lstm}$ takes in the state $s_t$, a program embedding $p_t \in \mathbb{R}^P$ and the previous hidden state $h_{t-1}$ to update its hidden state $h_t$. From the top LSTM hidden state several decoders generate the following outputs: the return probability $r_t$, the next program's key embedding $k_{t+1}$, and the arguments to the next program $a_{t+1}$.
The next program's ID is obtained by comparing the key embedding $k_{t+1}$ to each row of key memory $M_{key}$. Then the program embedding is retrieved from program memory $M_{prog}$ holding $N$ programs as:
$i^* = \argmax _{j=1..N}(M^{key}_j)^T k_{t+1}\ ,\ p_{t+1} = M^{prog}_{i^*}.$

The above-described NPI architecture bears two limitations. First, as shown in the above equation, at each time step, a decision must be made by the core to select the next program to call out of \emph{all} $N$ currently learned programs in the program memory. As $N$ grows large, e.g. to hundreds or thousands, interpreting programs correctly becomes a more and more difficult task for a single core. What makes things worse is that the core has to learn to interpret new programs without forgetting old ones.
Second, it is common that programs with different names and functionalities share some common underlying programming patterns. We take two programs used in \cite{reed2016} and \cite{cai2017} as example (see Figure \ref{fig:CNPI-vs-RNPI}): the ADD1 program in grade school addition, and the BSTEP program to perform one pass of bubble sort. We use their recursive forms described in \cite{cai2017}. The two programs share a very common looping pattern.
However, the core needs to learn each of these programs separately without taking any advantage of their shared patterns. The total number of programs that need to be learned by the core thus become infinite.
We argue that these two limitations make it very challenging, if not impossible, to construct a universal NPI.

\subsection{Our approach using combinator abstraction}
To overcome the limitations of the NPI, we propose to incorporate combinator abstraction into the NPI architecture. In functional programming, combinators are a special kind of higher-order functions that serve as power abstraction mechanisms, increasing the expressive power of programming languages. We adapt the concept to neural programming and make it play the central role in improving the universality of NPI.

Conceptually, a \emph{combinator} is a ``program template'' with blanks as \emph{formal} arguments that are callable as subprograms. An actual program can be formed by wrapping a combinator with another program called an \emph{applier}, which invokes the combinator and passes the \emph{actual} arguments to be called when executing the combinator. Alternatively, an applier applies a combinator to a set of actual programs as callable arguments. Note that the callable arguments themselves can also be wrapped programs (i.e. appliers), and programs with increasing complexity can thus be built up. As in the original NPI, the interpretation of a combinator is conditioned on the output of a lightweight domain-specific encoder which we call an \emph{detector}. It is also provided on the fly by the applier. Figure \ref{fig:CNPI-vs-RNPI} illustrates the usage of combinator abstraction in the NPI architecture.

\begin{figure}
  \begin{center}
  \includegraphics[width=4.5in]{CNPI-vs-RNPI}\\
  \caption{NPI with combinator abstraction.}
  \label{fig:CNPI-vs-RNPI}
  \end{center}
\end{figure}

In the CNPI architecture, combinators are the only type of programs that need to be interpreted by the core. By prohibiting a combinator to call programs other than those passed to it as arguments, the selection range for next program to call at each time step is reduced from a growing $N$ to a constant $K$, which is the maximum number of arguments for combinators ($\le$ 9 in our proposed model). Meanwhile, compared to the infinity of all possible programs, the number of useful combinators is finite and typically small. In practice, we construct a small set of four combinators to express four most pervasive programming patterns. Therefore, the core only needs to interpret a small number of simple programs. We will show that a quite small core suffices for this job, and that by the collaboration of this core and the other components a universal CNPI can be constructed.

\section{Combinatory NPI model}
\subsection{Combinators and combinatory programs}
We propose a set of four combinators to express four most pervasive programming patterns for algorithmic tasks: sequential, conditional, linear recursion and tree recursion (i.e. multi-recursion). The pseudo-code for these combinator are shown in Figure \ref{fig:comb}. Each combinator has four callable arguments $self$, $a1$, $a2$ and $a3$ and one detector argument $c?$. $self$ is a \emph{default} argument referring to the combinator itself and is used for recursive call.
For {\small \sffamily linrec} and {\small \sffamily treerec}, we give more readable aliases to $a1$, $a2$ and $a3$ to hint their typical roles.
The detector argument detects some condition (e.g. a pointer P2 reaching the end of array) in the environment and provides signals for the combinator to condition its execution. It outputs $0$ if the condition satisfies, otherwise $1$. For {\small \sffamily seq} a default blind detector is passed, which always outputs $0$. Although not directly callable, detectors can also be viewed as programs ``running in background'' as perception modules. In this paper, the conditions to detect is often used to name detectors and we append a `?' to their names to differentiate them from callable programs. Like primitive actions (ACTs), we could also define primitive detectors (DETs) for specific tasks.
Note that this combinator set is by no means unique or minimal. They take their current forms mainly for ease of use and learning.

\begin{figure}[!htb]
  \begin{center}
  \includegraphics[width=5.5in]{comb}\\
  \caption{Pseudo-code for the set of combinators.}
  \label{fig:comb}
  \end{center}
\end{figure}

The four combinators are classified into two categories. {\small \sffamily seq}, {\small \sffamily cond} and {\small \sffamily linrec} are basic combinators, which only call their callable arguments during execution. {\small \sffamily treerec} is an advanced combinator. Besides callable arguments, an advanced combinator can also call built-in programs, such as {\small \sffamily \_push\_sentinel} and {\small \sffamily \_mapself} in {\small \sffamily treerec}. These built-in programs are used to facilitate multiple recursive calls to $self$ in {\small \sffamily treerec} combinator. Basically, $divide$ prepares states necessary for each recursive call and push these states to a stack. The built-in combinator {\small \sffamily \_mapself} shares a similar structure with {\small \sffamily linrec}. It loads the states one by one from the stack and makes the recursive call with each state until a sentinel is met (The sentinel is pushed to the stack before $divide$ by {\small \sffamily \_push\_sentinel}, which is a built-in ACT). More details on built-in programs and {\small \sffamily treerec} are given in Appendix \ref{subsec:builtin-prog}, and examples of using them can be found in Appendix \ref{subsec:comb-prog}.

We now describe how to compose combinator programs using combinators by taking the BSTEP program (i.e. one pass of bubble sort) as example. The normal and combinatory version of the program are shown in Figure \ref{fig:comb-prog} (a) and (b) respectively.
Recall that an applier applies a combinator to a set of actual programs (ACTs or other predefined appliers) to form a new actual program. Composing a combinatory program amounts to defining appliers iteratively.
As shown in Figure \ref{fig:comb-prog} (c), during the execution of a combinatory programs, combinators and appliers call each other to form an alternating call sequence until reaching a ACT. Combinators, appliers and detectors are all highly constrained programs, and thus are all easily interpretable and learnable. Nevertheless, they can collaborate to build arbitrarily complex programs.

\begin{figure}[!htb]
  \begin{center}
  \includegraphics[width=5.5in]{comb-prog}\\
  \caption{Example combinatory program of BSTEP. NOP is special ACT which does nothing.}
  \label{fig:comb-prog}
  \end{center}
\end{figure}

\subsection{CNPI architecture and algorithm} \label{subsec:cnpi-arch-algo}
Having introduced combinators and how to use them to compose combinatory programs, we now describe how these programs are interpreted by the CNPI and the necessary augmentations to the original NPI architecture to enable the interpretation. The complete inference procedure for CNPI is given in Algorithm \ref{cnpi-alg}. An example execution of the BSTEP program is illustrated in Figure \ref{fig:example-exec}.

Appliers are effectively one-line programs that apply a program $prog$, which could be either a combinator or an ACT, to a set of arguments. To interpret an applier $appl$ we just need to identify the program to be called and its arguments, prepare environment for the invocation, and make the invocation.
For easy of interpretation, we propose to store the key embeddings of $prog$ and its detector and callable arguments $c?$, $a1$, $a2$ and $a3$ directly in the applier's program embedding $p_{appl}$:
\begin{equation}
\label{eq:p_appl}
p_{appl} = k_{prog} | k_{c?} | k_{a1} | k_{a2} | k_{a3}
\end{equation}
where $|$ denotes concatenation \footnote{For the {\small \sffamily seq} combinator and ACTs which do not need detector arguments, the blind detector's key embedding is stored. For ACTs with arguments the arguments' values are stored in place of the key embeddings.}. We use a \emph{fixed} parser to extract the key embeddings from the applier's embedding.
Then the combinator or ACT ID $i$ is computed by comparing the key $k_{prog}$ with each row of memory $M_{key}$ and finding the best match. The callable arguments' IDs are computed similarly.
In CNPI architecture, the models for detectors are stored in a detector memory ($W^{key}$, $W^{weight}$) which has the same key-value structure as the program memory. The detector argument ID $i'$ is computed by comparing $k_{c?}$ to each row of $W_{key}$.
Note that the core LSTM does not participate in the interpretation of appliers. As the format for storing these key embeddings is predefined, the fixed parser can parse \emph{any} applier's embedding.

We use a dynamically constructed data structure called a \emph{frame} to pass arguments to a combinator. Each frame is a table of $K$ bindings which associate formal callable arguments with their corresponding actual IDs, with $K$ the number of callable arguments for combinators. When calling a combinator, a new frame is created. The IDs of the combinator's callable arguments (including the combinator's ID $i$ as it corresponds to the $self$ argument) are filled into the frame \footnote{If the combinator is treerec, the IDs of built-in programs also need to be appended to the end of the frame in a predefined order. In this case the frame's size is $K + B$, with $B$ the total number of built-in programs.}. In practice we do not use a key-value structure for frames. Instead the frame only stores values, i.e. the arguments' IDs in a fixed order of $self$, $a1$, $a2$ and $a3$.

The interpretation of combinators is in general similar to the inference procedure in Algorithm 1 in \cite{reed2016}. Here we highlight several key differences. In the initialization stage, besides retrieving combinator embedding from the program memory, the detector model is also loaded from the detector memory. Then instead of using the combinator embedding as input to the LSTM at every time step, we use it to initialize the LSTM's state, i.e. each layer's hiddens and cells. We find empirically that this parameterization has better efficiency and accuracy for our combinators; see Section \ref{subsec:sl-results}.
The second difference is that we binarize the output of detector $f_{det}$ to get a binary condition $c$ before feeding it to the LSTM:
\begin{equation}
c \gets \mathbbm{1}(f_{det}(e) \ge \beta) \ , \ h \gets f_{lstm}(c, h)
\end{equation}
where $\mathbbm{1}()$ is an indicator function. This operation effectively decouples the detector from the core LSTM. This enables us to verify the core's behaviors separately without considering any specific detectors, given that the correct condition is provided. This is difficult to achieve in the original NPI architecture where the core is trained jointly with the encoders.

The third and most important difference is on how the next subprogram to call is computed. We use a decoder $f_{prog}$ to compute a score vector $S \in \mathbb{R^K}$ to assign a score for each formal callable argument. The argument with the maximum score is selected and its actual program ID is retrieved from the frame $F$. This ID is used in turn to retrieve the program embedding from the program memory $M^{prog}$ when the next program is executed:
\begin{equation}
z^* = \argmax _{j=1..K}\, S_j \ , \ i^* \gets F[z^*] \ , \ p_{t+1} = M^{prog}_{i^*}
\end{equation}
where $K$ is the maximum number of callable arguments for combinators.
We consider this indirection of subprogram embedding retrieval, together with the dynamic binding of formal arguments to actual programs in the frames, to be the key to the superior universality and learnability of CNPI.

When calling the subprogram the same detector ID and frame are re-used, which is equivalent to passing the combinator's arguments to all of its subprograms. This facilitates recursion as these arguments are needed by {\small \sffamily linrec} and {\small \sffamily treerec} when calling $self$ (see Figure \ref{fig:comb}). For other subprogram calls to appliers or ACTs, these arguments are safely ignored.

\begin{algorithm}[!htb]
  \caption{Combinatory neural programming inference}
  \label{cnpi-alg}
  \footnotesize
  \begin{algorithmic}[1]
  \State \textbf{Inputs:} Environment observation $e$, program ID $i$, detector ID $i'$, frame $F$,
  stop threshold $\alpha$, condition threshold $\beta$, number of arguments (including $self$) $K$ for combinators
  \Function {Run}{$i, i', F$}
    \State $r \gets 0,\ p \gets M^{prog}_i$
    \If {$p$ is an applier}
      \State $i_2,\ i'_2,\ a_2 \gets$ \Call{\_Parse}{$p$}  \Comment{Get the next program to run with its detector and args.}
      \State $F_2 \gets $ \Call{Frame}{$K$},\ $F_2[1] \gets i_2$   \Comment{New an empty frame and fill in $self$ arg.}
      \For{$j=1$ to $K-1$} $F_2[j+1] \gets a_2[j]$  \Comment{Fill in the other args.} \EndFor
      \State \Call{Run}{$i_2, i'_2, F_2$}  \Comment{Run subprogram $i_2$ with detector $i'_2$ and frame $F_2$.}
      \State \Call{Free}{$F_2$}  \Comment{Free frame $F_2$'s space.}
    \ElsIf{$p$ is a combinator}
      \State $f_{det} \gets W^{weight}_{i'},\ h \gets p$  \Comment{Load detector from detector memory and init LSTM.}
      \While{$r<\alpha$}
        \State $c \gets \mathbbm{1}(f_{det}(e) \ge \beta),\ h \gets f_{lstm}(c, h)$  \Comment{$c$ is a binary condition.}
        \State $r \gets f_{end}(h),\ S \gets f_{prog}(h)$  \Comment{$S$ is a $K$-dim score vector.}
        \State $z_2 \gets \argmax _{j=1..K}\, S_j$    \Comment{Decide the argument ID of the next program to run.}
        \State $i_2 \gets F[z_2]$    \Comment{Retrieve the ID of the next program to run from frame.}
        \State \Call{Run}{$i_2, i', F$}   \Comment{Run subprogram $i_2$ with the same detector and frame.}
      \EndWhile
    \Else  \Comment{$p$ is a primitive action.}
      \State $a \gets F[2:K],\ e \gets f_{env}(e, p, a)$ \Comment{Unpack args from $F$ and do the action.}

    \EndIf
  \EndFunction
  \State
  \Function{\_Parse}{p}  \Comment{Helper function for interpreting appliers.}
    \State $k, k', a \gets \Call{Split}{p}$  \Comment{Get program, detector and arg keys from applier embedding.}
    \State $i \gets \argmax_{j=1..N}(M^{key}_j)^T k,\ i' \gets \argmax_{j=1..M}(W^{key}_j)^T k'$ \Comment{Program key to id.}
    \If{$i$ is not a primitive action}
      \For{$j=1$ to $K-1$} $a[j] \gets \argmax_{j'=1..N}(M^{key}_{j'})^T a[j]$  \Comment{Arg keys to IDs.} \EndFor
    \EndIf
    \State \Return $i, i', a$
  \EndFunction
  \end{algorithmic}
\end{algorithm}

\begin{figure}[!htb]
  \begin{center}
  \includegraphics[width=5.5in]{example-exec}\\
  \caption{Example execution of the combinator BSTEP program. SWAP, P1\_RIGHT and P2\_RIGHT are in fact appliers which need to be parsed as COMPSWAP and RSHIFT. We omit this step and treat them as ACTs in the figure for brevity.
  \label{fig:example-exec}
  \end{center}
\end{figure}

\subsection{Training}
CNPI has four components: the core, the program (combinator and applier) memory, the detector memory, and the parser, of which the first three are learnable. The combinators are trained jointly with the core. Detectors and appliers are trained separately.

Supervised learning (SL) of CNPI uses execution traces of combinatory programs. A single element of an execution trace consists of a step input-step out pair, which takes one of the two forms: $\xi^{comb}_t: \{e_t, i_t\} \to \{c_t, z_{t+1}\}$ for combinator execution and $\xi^{appl}_t: \{i_t\} \to \{i_{t+1}, i'_{t+1}, a_{t+1}\}$ for applier execution.
$z_{t+1}$ is the formal callable argument ID to be called by the applier at time step $t+1$. $c_t$ is the correct condition at time step $t$ and is used as the output target for detectors. $i_{t+1}$ and $r_t$ provide targets for the core.
Detectors and the core are trained on the $\xi^{comb}_t$ elements of the trace, using stochastic gradient ascent to maximize the likelihood of their corresponding targets.
\begin{equation}
\Delta w \propto \sum_{t=1}^T(\nabla_w \log p_w(c_t \mid e_t)) \ , \
\Delta \theta \propto \sum_{t=1}^T(\nabla_\theta \log p_\theta(i_{t+1} \mid c_t) +
\nabla_\theta \log p_\theta(r_t \mid c_t))
\end{equation}
where $w$ are parameters of the detector model, $\theta$ are the collective parameters of the core and the combinator embedding, $T$ is the length of the sequence of $\xi^{comb}_t$ elements.
The probability $p(i_{t+1} \mid c_t)$ of calling subprogram $i$ is computed by applying a softmax to the scores produced by $f_{prog}$: $p(i_{t+1} \mid c_t) = \exp{s(i_{t+1} \mid c_t)} / \sum_j{\exp{s(j_{t+1} \mid c_t)}}$.
In SL the applier embeddings do not need to be trained; they are just generated from $\xi^{appl}_t$ elements of the trace according to equation (\ref{eq:p_appl}).

CNPI can also be trained by policy gradient reinforcement learning (RL) \footnote{In this paper we only train CNPI by RL on tasks that can be solved by the three basic combinators.}. No execution trace is given and the core tries to complete the task by making program calls following the probabilities $p(i_{t+1} \mid c_t)$ and feeding-forward the LSTM. An episode ends if the task is completed or the number of steps reaches MAX\_NSTEP. In our experiments, MAX\_NSTEP = $K \cdot n$, where $K$ is the number of callable arguments for combinators, $n$ is the complexity of the problem to be solved. A reward $R_T$ is given when an episode ends at step $T$:
\begin{equation}
R_T =
\begin{cases}
+1-0.1 \times T & \text{if task is completed} \\
-1-0.1 \times T & \text{otherwise}
\end{cases}
\end{equation}

At each time step $t$, a condition $\tilde{c}_t$ is sampled from a Bernoulli distribution defined by the output of the detector. The next program to be called is identified as $F[\tilde{i}_{t+1}]$, where $\tilde{i}_{t+1}$ is sampled from $\{p(i_{t+1} \mid \tilde{c}_t)\}$. The core is trained using stochastic gradient ascent on a mixed objective with two parts: an RL objective of maximizing expected reward, plus an SL objective of maximizing the likelihood of correct flag of program return:
\begin{equation}
\Delta \theta \propto \sum_{t=1}^T(\nabla_\theta \log p_\theta(\tilde{i}_{t+1} \mid \tilde{c}_t)R_T +
\nabla_\theta \log p_\theta(r_t \mid \tilde{c}_t) \mathbbm{1}(R_T>0))
\end{equation}
The RL objective is derived from the REINFORCE algorithm \citep{williams}. Note that the SL objective only takes effect on episodes where a positive reward is received on task completion. This combination of RL and SL objectives to optimize a policy is also used in \cite{Oh2017} to learn parameterized skills.
The detector is also trained to maximize expected reward using REINFORCE:
\begin{equation}
\Delta w \propto \sum_{t=1}^T(\nabla_w \log p_w(\tilde{c}_t \mid e_t)R_T)
\end{equation}

Once the detector and the core have been learned, applier embeddings can also be learned using RL. After the program $\tilde{i}$ is called with detector and callable arguments $\tilde{i'}$ and $\tilde{a}_j, j=1..K-1$, a reward $R \in \{-1, +1\}$ is given according to whether the task has been completed. The applier embedding parameterized by $\phi$ is updated as:
\begin{equation}
\label{eq:learn-applier}
\Delta \phi \propto \nabla_\phi \left(\log p_\phi (\tilde{i}) + \log p_\phi (\tilde{i'}) + \sum_{j=1}^K{\log p_\phi (\tilde{a}_j)}\right) R
\end{equation}
where the identifiers $\tilde{i}$, $\tilde{i'}$ and $\tilde{a}_j$ are sampled respectively from the distributions derived from the corresponding keys stored in the applier's embedding: $\tilde{i} \sim {\rm softmax}(M^{key} k_{prog})$, $\tilde{i'} \sim {\rm softmax}(W^{weight} k_{c?})$, $\tilde{a}_j \sim {\rm softmax}(M^{key} k_{a_j}), \ j=1..K-1$.

Note that in both SL and RL, detectors are trained separately from the core. This decoupling facilitates the sharing of detectors across programs and the verification of the behavior of the core.

\section{Analysis} \label{sec:analysis}
Training CNPI with SL to solve algorithmic tasks consists of three steps. First, train and verify the core jointly with the combinators with synthetic \emph{abstract traces}, i.e. sequences of $\xi^{comb}_t$ elements corresponding to the correct invocation of formal callable arguments given conditions (a total of 11 traces for the four combinators).
After having been verified for correct behavior, the core and the combinator embeddings are fixed. This step is done only once before solving any specific task. Second, for a new task, identify the conditions needed to solve the task, train and verify detectors to detect these conditions, and then add them to the detector memory. Finally, iteratively define appliers from the bottom up by adding them to the program memory with program embeddings set according to equation (\ref{eq:p_appl}) given $\xi^{appl}_t$ elements of the traces, and call the topmost applier to solve the task. We state the universality of CNPI with the following theorem and proposition:

\begin{theorem} \label{thm-university}
If 1) the core along with the program embeddings of the set of four combinators and the built-in combinator \_mapself are trained and verified before being fixed, and 2) the detectors for a new task are trained and verified, then CNPI can 1) interpret the combinatory programs of the new task correctly with perfect generalization (i.e. with any input complexity) by adding appliers to the program memory, and 2) maintain correct interpretation of already learned programs.
\end{theorem}
\begin{proposition} \label{prop-combinatorizable}
Any recursive program is combinatorizable, i.e., can be converted to a combinatory equivalent.
\end{proposition}

Theorem 1 states that CNPI is universal with respect to the set of all combinatorizable programs and that appliers can be continually added to the program memory to solve new tasks. Proposition 1 shows that this set of programs is adequate for solving most algorithmic tasks, considering that most, if not all, algorithmic tasks have a recursive solution.
We prove Theorem \ref{thm-university} in Appendix \ref{subsec:proof}. For Proposition \ref{prop-combinatorizable}, instead of giving a formal proof, we propose a concrete algorithm for combinatorizing any program set expressing an recursive algorithm in Appendix \ref{subsec:comb-prog}.

We argue that universality is a property harder to achieve than the generalization property discussed in \cite{cai2017}, which provides provable guarantees of perfect generalization for several programs. However, the authors did not consider the problem of universality with a fixed core. In fact, although RNPI can be trained on a particular task and verified for perfect generalization, after training on a new task causing changes to the parameters of the core, the property of perfect generalization on old tasks may not hold any more. In contrast, CNPI provides both generalization and universality. Table \ref{tab:compare-models} qualitatively compares CNPI with NPI and RNPI.

\begin{table}[!htb]
\caption{Qualitative comparison of CNPI with NPI and RNPI.} \label{tab:compare-models}
\footnotesize
\begin{center}
\begin{tabular}{c|c|c|cc|cc}
\hline
\multirow{2}*{Model} &
\multirow{2}*{\makecell{provable \\ perfect \\ generalization}} &
\multirow{2}*{\makecell{provable \\ universality}} &
\multicolumn{2}{c|}{\# verifications of} & \multicolumn{2}{c}{\# trainings of} \\
& & &
\makecell{programs / \\ combinators} & \makecell{encoders / \\ detectors} &
\makecell{programs / \\ combinators} & \makecell{encoders / \\ detectors}\\
\hline
NPI & $\times$ & $\times$ & $-$ & $-$ & per task & per task \\
RNPI & \checkmark & $\times$ & per task & per task & per task & per task \\
{\bf CNPI} & \checkmark & \checkmark & {\bf once} & {\bf per condition} & {\bf once} & {\bf per condition} \\
\hline
\end{tabular}
\end{center}
\end{table}

Due to the decomposition of programs into combinators and appliers, and the decoupling of detectors from the core, we can verify the perfect generalization of a particular program using much fewer test inputs than RNPI. For example, to verify the perfect generalization of bubble sort with RNPI we need 2078 test inputs for 6 subprograms while with CNPI we need only 123 for 4 detectors.

\section{Experiments}
While the previous section analyzes the universality of CNPI, this section shows results on the empirical evaluation of its learnability via both SL and RL experiments. We mainly report results on learning the core and the combinators, assuming that detectors for the tasks have been trained. Learning a detector in our CNPI architecture is a standard binary classification problem, which can be trivially solved by training a classifier.

To evaluate how CNPI improves learnability over the original NPI architecture, in some experiments we use the RNPI model as a baseline. It has the same architecture as NPI and allows recursive calls.
For a CNPI with $K$ callable arguments (denoted as CNPI-$K$), we construct a counterpart RNPI with $K \cdot n$ existing actual programs (either composite programs or ACTs) in the program memory as base programs (denoted as RNPI-$K\text{x}n$). These base programs are divided into $n$ sets corresponding to $n$ different tasks (e.g. grade school addition and bubble sort). Then new programs are learned over each set by calling the corresponding $K$ base programs as subprograms. Note that some of these new programs may share same patterns (e.g. ADD1 and BSTEP, we call them isomorphic programs), but in RNPI they are treated as different programs and the core needs to learn all of them. For fair comparison, the counterpart RNPI uses the same detector as the CNPI.

For all experiments, we used a one-layer LSTM for the core. We trained the CNPI using plain SGD with batch size 1, and learning rate of 0.5 and 0.1 for SL and RL experiments respectively. For the SL experiments, the learning rate was decayed by a factor of 0.1 if prediction accuracy did not improve for 10 epochs.

\subsection{Supervised learning results} \label{subsec:sl-results}
We found that, as expected, a CNPI can be trained to learn the set of four combinators using synthetic abstract traces without any difficulty. From Section \ref{sec:analysis} we know that this CNPI is able to learn all combinatorizable programs (including the four in Appendix \ref{subsec:comb-prog}) with perfect generalization. To further stress the learning capacity of the core, we enlarge the small set of four combinators to a full set of all possible combinators with the following two constraints: 1) branching can only happen at the beginning of the execution; 2) call to the $self$ argument, i.e. recursive call, can only be made at the end of the execution (i.e. only tail recursion is allowed). For $K = 4$, this full set has 57 combinators (including the three basic combinators).

Cores with different number of LSTM cells were trained to learn this full set of combinators. We compare two methods of feeding the combinators embedding to the core LSTM: use the embedding as input to the LSTM at every time step (Emb-as-Input), as is done in \cite{reed2016}, and using it as the initial state (i.e. hiddens and cells) of the LSTM (Emb-as-State0). For both methods the combinator embedding size is set to be equal to the LSTM size. Note that the Emb-as-State0 model has fewer parameters than Emb-as-Input with the same number of cells. We also trained a sequence-to sequence model from \cite{sutskever2014} where an encoder LSTM takes in the text code representation of the combinator (a simplified version of the pseudo-code in Figure \ref{fig:comb}) and the last state of the encoder is used as the combinator embedding to initialize the core LSTM's state. This seq2seq model can be seen as a miniature of an instruction-to-action architecture. We see in Figure \ref{fig:accuracy-vs-cells} that the Emb-as-State0 model achieves better prediction accuracy than the Emb-as-Input model with the same size. Particularly, the Emb-as-State0 model with only 5 cells can learn all the 57 combinators with 100\%. This LSTM is much smaller than the one used in \cite{reed2016} which has two layers of size 256. The seq2seq model can also achieve 100\% accuracy with 7 cells. In subsequent experiments we used a core LSTM of size 16 if not mentioned otherwise.

\begin{table}[!htb]
\begin{minipage}[b]{0.4\linewidth}
\centering
\includegraphics[width=2.1in]{accuracy-vs-cells-eps-converted-to}\\
  \captionof{figure}{Prediction accuracy on the full combinator set with 4 callable arguments.}
  \label{fig:accuracy-vs-cells}
\end{minipage}\hfill
\begin{minipage}[b]{0.56\linewidth}
\centering
\footnotesize
\begin{tabular}{llccc}
\hline
{Model} & {Old / New set} & \makecell{Train \\ old} & \makecell{Train \\ new w/ \\ fixed core} & \makecell{Test \\ old } \\
\hline
\multirow{2}*{RNPI-4x2} & \multirow{2}*{\makecell[l]{50\% full set 1 / \\ 50\% full set 2}}  & 100 & $>$ 90 & 12.9 \\
& & 100 & $>$ 97 & 3.3 \\
\cline{2-5}
& \multirow{2}*{\makecell[l]{100\% full set 1 / \\ 100\% full set 2}} & 100 & $>$ 90 & 6.5 \\
& & 100 & $>$ 97 & 1.7 \\
\hline
{\bf CNPI-4}   & \makecell[l]{50\% full set / \\ the other 50\%} & 100 & 97.7 & {\bf 100} \\
\hline
\end{tabular}
\caption{\% accuracy of learning new programs/combinators and remembering old ones with a fixed core. The maximum accuracy obtained when training on new set are 100\% and 97.7\% for RNPI and CNPI respectively. }
\label{tab:old-new}
\end{minipage}
\end{table}

We compare the abilities of CNPI and RNPI to learn new combinators/programs with a fixed core. The models were first trained on a combinator/program set to get 100\% accuracy, then trained on a new set with the core fixed. Finally the models were tested on the old set to see if they are still remembered by the models. For the CNPI-4 model the old and new combinators were generated by a random even split of the full combinator set. For the RNPI-4x2 model with two sets of base programs, we constructed a full set of all possible composite programs for each set of base programs, as with combinators. Then old and new programs were randomly sampled from the two sets respectively. Note that the old and new programs generated this way have certain proportion of isomorphic programs, and this proportion grows with the percentage of random sampling. In the RNPI experiment, the program key embeddings need to be learned jointly with the program embeddings, otherwise the model would not be able to learn the new programs. As shown in Table \ref{tab:old-new}, although both models can be trained on the new set with high accuracy, when tested on old ones, RNPI shows catastrophic forgetting, which becomes more severe as there are more isomorphic programs between the old and new set. In contrast, CNPI remembers old combinators perfectly.

\subsection{Reinforcement learning results} \label{subsec:rl-results}
We find that curriculum learning is necessary for training CNPI with RL. Table \ref{tab:rl-curriculum} shows the curriculum we used for training CNPI for the sorting task. For each subtask, the programs to be learned (including detectors) are bolded and colored. The curriculum has two stages. In the first stage, the combinators were trained with simple auxiliary tasks, using ACTs and DETs as arguments. The learned combinators are then used as prerequisite for solving the actual tasks. The tasks for each combinator is designed to ensure that the task will be completed if and only if the combinator is correctly executed as defined in Figure \ref{fig:comb}.
In the second stage, detectors and appliers can be learned in two forms: we can either define a sketch for solving the task (similar to the policy sketches in \cite{andreas17}), with some learnable arguments (as in the Compare and swap and Output max tasks), or learn an applier to solve the task using already learned arguments (as in the Sort task).
Note that for brevity we define some appliers for resetting pointers directly without any learning after at the end of Stage 1. In fact, they could also be learned the same way as the appliers in Stage 2.

\begin{table}[!htb]
\caption{Curriculum for training CNPI for the sorting task using RL. The programs to be learned (including detectors) are bolded and colored. Several ACTs are added to help learn the tasks: OUT\_x: write the element at pointer x to position pointed by P3 and advanced P3 one step. CLEAR\_x: set the element at pointer x to $-1$. OUTCLEAR\_x: output then clear x. For the sort task, instead of sorting in-place, the max element found in each pass is written to a second array pointed to by P3.}
\label{tab:rl-curriculum}
\footnotesize
\begin{center}
\begin{tabular}{p{0.75in}|p{1.7in}|l}
\hline
{\bf Subtask} & {\bf Description} & {\bf Program to learn} \\ \hline
\multicolumn{3}{c}{Stage 1: Learning the core and combinators} \\ \hline
Swap and output easy & Output A[P1] & \learn{seq0}(; OUT\_1, NOP, NOP) \\ \hline
Swap and output & Output A[P1], then swap A[P1] and A[P2], finally output A[P2] & \learn{seq}(; OUT\_1, SWAP\_12, OUT\_2) \\ \hline
Conditional output easy & Output A[P2] if P2 is in array, otherwise output `OK' & \makecell[tl]{\learn{cond0}(A[P2]$\neq$END?; \\ OUT\_2, NOP, OUT\_OK)} \\ \hline
Conditional output & Output and clear A[P2] if P2 is in array, otherwise output `OK' & \makecell[tl]{\learn{cond}(A[P2]$\neq$END?; \\ OUT\_2, CLEAR\_2, OUT\_OK)} \\ \hline
Copy easy & Output the first element if array is empty, otherwise output `OK' & \makecell[tl]{\learn{linrec0}(A[P2]$\neq$END?; \\ OUT\_2, NOP, OUT\_OK)} \\ \hline
Copy & Output elements sequentially till the end of array, then output `OK' & \makecell[tl]{\learn{linrec}(A[P2]$\neq$END?; \\ OUT\_2, P2\_RIGHT, OUT\_OK)} \\ \hline
Reset pointers & Reset P1 and P2 to the appropriate beginning position & \makecell[tl]{RESET\_1: linrec(A[P1]$\neq$END?; \\ P1\_LEFT, NOP, NOP) \\ RESET\_2: linrec(A[P2]$\neq$END?; \\ P2\_LEFT, NOP, P2\_RIGHT) \\ RESET: seq(; RESET\_1, RESET\_2, NOP)} \\ \hline

\multicolumn{3}{c}{Stage 2: Learning detectors and appliers} \\ \hline
Conditional swap & Conditionally swap two elements & \makecell[tl]{COMPSWAP: cond(\learn{A[P1]$>$A[P2]?}, \\ SWAP\_12, NOP, NOP)} \\ \hline
Output max & Find and output the max element in the array then clear it & \makecell[tl]{MAX: linrec(A[P2]$\neq$END?; \\ \learn{STEP}, P2\_RIGHT, OUTCLEAR\_1)} \\ \hline
Sort & Sort the array by repeatedly outputting the current max element & \makecell[tl]{\learn{SORT}: linrec(A[P3]$\neq$END?; \\ MAX, RESET, NOP)} \\ \hline
\end{tabular}
\end{center}
\end{table}

Though being quite simple programs, we find that the three basic combinators are still difficult to learn with plain policy gradient RL, even with the curriculum. To facilitate learning. we use the adaptive sampling technique proposed in \cite{reed2016}. Example traces are fetched with frequency proportional to the model's current prediction error. We set the sampling frequency using a softmax over a moving average of prediction error over last 10 episodes, with temperature 1. Besides, for each combinator's auxiliary task we design an easy version of the task, which corresponds to a partial completion of the true task (see Table \ref{tab:rl-curriculum}). Then a curriculum can be formed for each combinator by either mixing the easy and true task (mixed), or complete the easy task first before going to the true one (gradual). For each different use of adaptive sampling and curriculum we ran 100 experiments with a maximum number of 5000 episodes for each experiment. Table \ref{tab:rl-comb} shows the success rate that all three auxiliary tasks are completed along with success rates for completing the task for each combinator. As shown in Table \ref{tab:rl-comb}, both adaptive sampling and the curriculum help training considerably. A relatively high success rate of 91\% can be obtained with adaptive sampling and the gradual curriculum. We can also know from Table \ref{tab:rl-comb} that of the three combinators {\small \sffamily seq} is the easiest to learn while {\small \sffamily linrec} is the hardest.

\begin{table}[!htb]
\begin{minipage}[b]{0.6\linewidth}
\centering
\footnotesize
\caption{\% success of training the core+combinators with RL. The three figures in brackets represent \% success of learning the seq, cond or linrec combinator respectively.} \label{tab:rl-comb}
\begin{tabular}{lccc}
\hline
\makecell{Sampling \\ method} & \makecell{No \\ curriculum} & \makecell{Mixed \\ curriculum} & \makecell{Gradual \\ curriculum} \\
\hline
Uniform & {\bf 7} (11/10/7) & {\bf 17} (33/28/17) & {\bf 49} (94/49/49) \\
Adaptive & {\bf 31} (74/41/31) & {\bf 78} (87/80/78) & {\bf 91} (92/91/91)\\
\hline
\end{tabular}
\end{minipage}\hfill
\begin{minipage}[b]{0.36\linewidth}
\centering
\footnotesize
\caption{Comparison of \% success of training CNPI with RL with other models.} \label{tab:rl-cnpi-rnpi}
\begin{tabular}{lccc}
\hline
{Stage} & \makecell{CNPI \\ -4} & \makecell{RNPI \\ -4x2} & \makecell{RNPI \\ -4x3} \\
\hline
Easy & {\bf 99} & 25 & 0 \\
Final & {\bf 91} & 0 & 0 \\
\hline
\end{tabular}
\end{minipage}
\end{table}

We compare the success rate of training CNPI (CNPI-4) with its counterpart RNPI models RNPI-4x$n$. For each combinator's auxiliary task, we constructed $n$ different versions of the task by providing different ACTs and DETs. For example, for the Copy task we used different pointers (e.g. P1) to move in different directions (e.g. to left), and output different symbols when finished (e.g. `DONE') to generate $3 \times n$ tasks. Then we trained RNPI to learn $3 \times n$ actual programs in parallel to solve these tasks. The RNPI-4x$n$ models were trained with adaptive sampling and gradual curriculum for 10000 $\times n$ episodes, and the success rate over 100 trials are shown in Table \ref{tab:rl-cnpi-rnpi}. Due to the enlarged candidate set of the next program to call from 4 to 4 $\times n$ and the increased number of programs to be learned from 3 to $3 \times n$, it is much more difficult to train RNPI with RL. RNPI-4x2 finishes the first stage of the curriculum to complete the easy tasks with a success rate of 25\% while fails completely on the final stage to complete the true tasks. RNPI-4x3 can not even finish the easy stage.

We trained the A[P1]$>$A[P2]? detector in the Conditional swap task with the RL objective. The input to the detector is the one-hot encoding of the two elements. Then the STEP applier was learned in the context of the Output max task by maximizing the expected reward of completing the task; see equation (\ref{eq:learn-applier}). The embedding of STEP was learned successfully in 79\% out of the 100 experiments we ran. In each successfully trial one of the two appliers, seq(; COMPSWAP, P1\_LEFT, NOP) and cond(A[P1]$>$A[P2]?; NOP, NOP, MOVE\_12) was learned, which was equivalent to finding the max element by a pass of bubble sort and selection sort respectively. MOVE\_12 is a primitive applier we defined to move P1 forward until reaching P2. Finally, the Sort applier was learned to complete the Sort task, with success rate of 62\% over 100 experiments. Both bubble sort and selection sort were learned by calling the two learned STEP appliers respectively.

\section{Conclusion}
The problem of improving the universality and learnability of NPI is addressed for the first time by incorporating combinator abstraction from functional programming. Analysis and experimental results have shown that CNPI is universal with respective to all combinatorizable programs and can be trained with both strong and weak supervision.
We believe that the proposed approach is quite general and has potential applications besides solving algorithmic tasks. One scenery is training agents by RL to follow instructions and generalize (e.g., \cite{Oh2017}, \cite{andreas17}, \cite{denil2017}). Natural language contains ``higher-order'' words such as ``then'' and ``twice'', which play critical role but the interpretation of which may cause trouble to vanilla sequence-to-sequence models \citep{lake2017still}. By representing these words as combinators and equipping the agent with CNPI-like components, it would be possible to construct agents that display more complex and structured behavior and that generalize better. We leave this for future work.

\subsubsection*{Acknowledgments}
We thank Mingli Yuan for valuable discussion and feedback.

redpajama_set_name: RedPajamaArXiv


In accordance with the above, this report will be divided into four major parts. Firstly, the report will identify and analyse the reasons why the restaurant is chosen as the start-up business venture for the Wurst family. Following that, it will make use of the collected information to investigate the restaurant itself and the external environment so as to achieve appropriate business strategies. Subsequently, it will provide the financial budgeting for the restaurant to 30 October 2013 in terms of breakeven analysis, cash flow, income, balance and others. The last part will conclude the report in terms of viability of the restaurant.
After negotiation, Mr. and Mrs. Wurst are advised to establish a restaurant rather than a retail florist shop on Playford Avenue in Whyalla, and the choice is made on the basis of the following reasons. For one thing, it is known to all that catering trade is one of the few industries that are mostly inextricably linked with people's daily lives, which is therefore quite a stable industry. For instance, in 2009, although economic conditions were extremely tough for Australia by worsening further from the preceding year, consumer food service maintained a growth of 4%, similar to the growth of the last year. The increase was attributed to a big and steady gain of the cheap takeaway or home delivery food choices and slight increases of full-service restaurants. Furthermore, it's predicted that Australians are expected to continuously dine out as the economy is bettering in spite of rising interest rates and inflation (Euromonitor, 2010). Hence, as a start-up business, the Wursts are advised to take up a venture in an industry that has bigger winning possibilities.
Secondly, according to Maslow's hierarchy of needs, the five needs of physiology, safety, affection, esteem and self-fulfilment are advancing step by step while people would not have the next need until the preceding one is achieved (Maslow, 1999). As a result, it is logical for people to put food service, which falls into the physiological needs ahead of the florist industry even in peaceful times. On the other hand, assuming inflation continues to weaken Australian people's living standards, it can be foreseen that citizens would give up their florist hobbies for their catering needs. For this reason, the Wurst family are recommended to start the catering business.


Representation and service of colour sorters in Central Europe. Demonstrations and tests with products in the laboratory in Dresden/Germany.
Legumes, nuts, coffee beans, rice, corn, mustard, coloured berries, dried fruits and vegetables, seeds, cereals, plastic flakes and plastic granulate.


All of our products have been carefully packaged to avoid damage in transit and all items are fully insured with the Freight Carrier against damage in transit. We only contract professional and reliable trucking companies to deliver your order. Unfortunately, even the very best freight companies can accidentally damage your ordered items. For this reason, it is imperative that you thoroughly inspect the shipment at the time of arrival for any signs of mishandling such as dents, holes, rips or tears to the packaging. The packaging is designed to protect the items inside during transit. If there are noticeable signs of damage on the outside of the packaging, request the option of inspecting the product by opening boxes prior to signing the delivery receipt. If any items are noticeably damaged, DO NOT PANIC. Our customer service specialists have many years of experience with these situations and as long as you follow these simple instructions, any issues will be resolved in a timely manner.
Any and all noticeable damage must be noted by writing "DAMAGED" on the delivery receipt prior to signing it, and reported to us within 48 hrs of delivery. We cannot be held responsible for damage not noted on the delivery receipt at the time of delivery. Any damage not noted on the delivery receipt will be assumed to have occurred after delivery, and therefore will be the responsibility of the purchaser. Only refuse delivery of piece(s) showing obvious signs of severe damage. DO NOT REFUSE any un-damaged items. Please call us immediately at 1-800-269-2216 if any of your items arrive damaged. We also ask that digital photographs of any damaged items be emailed to: service@shopfactorydirect.com within 5 business days of delivery. We will make arrangements to have your damaged item(s) restored or replaced as soon as we receive your email. Any undamaged items refused may be subject to the terms, conditions and applicable fees of our return policy. If the damages are not promptly and fully noted and reported to us, we will not be able to file a claim with the freight company.
Freight carriers and delivery agents are only liable to cover the cost of professional restoration to first quality condition of any item(s) damaged in transit and noted on the delivery receipt at the time of delivery. If restoration costs are estimated to exceed replacement costs, the damaged item(s) will be replaced. Replacement parts will be shipped for smaller items. ShopFactoryDirect assumes responsibility of any restoration or replacement costs and filing damage claims with the carrier for all damages noted on the delivery receipt at the time of delivery. Items damaged in transit are not eligible for a refund. Claims for any marks, blemishes or flaws on unfinished, inconspicuous surfaces such as backs of case pieces, bottoms of tables or dust covers underneath upholstered pieces which do not compromise the functionality, structure or aesthetics of the item will not be validated. Refusal of restored or replacement items are subject to the terms, conditions and applicable fees of our return policy. A refund will only be issued in the rare event that the item(s) cannot be repaired or replaced.
Parts such as assembly instructions and hardware are sometimes concealed in the packaging. Please thoroughly examine all packaging materials such as styrofoam blocks and under case pieces for any missing items. The manufacturers usually keep replacement parts on hand for most items. Replacements are shipped in a timely manner whenever possible. In the unlikely event that replacement parts are not in stock and must be ordered from the vendor, we cannot be held responsible for unexpected delays due to parts being shipped from overseas. Please report missing parts or any other issues with your order within 48 hours of receipt of delivery by calling our toll free number 1-800-269-2216 (Please leave a detailed message if prompted to do so) or email us at service@shopfactorydirect.com.
We only sell products from very reputable manufacturers, which adhere to the strictest standards of quality control. Although highly unlikely, in the event an item is determined to be defective, it shall be professionally restored to first quality condition or replaced at no cost to the purchaser. Please be aware that some traditional, transitional and shabby chic style furniture pieces are intentionally manufactured with a "Distressed" finish. This deliberate process adds features including, but not limited to scarring, sand through edges and patina finishes which are designed to give the furniture character by providing an aged or "vintage" look. Furniture featuring an intentionally distressed finish may not be considered defective. All defective merchandise claims must be placed within 48 hours of delivery of the merchandise. We will need digital photographs of any suspected defects in order to file a claim with the manufacturer. Defects or imperfections are not reasons for returning merchandise. In the event the manufacturer determines (at their sole discretion) that a defective merchandise claim can be resolved by sending replacement parts, replacement merchandise will be shipped at no cost to the purchaser. In the rare event it is determined the defect cannot be professionally restored and a replacement part is no longer available, a partial refund will be provided to keep the defective merchandise as-is.


text:
Visors, staggered start-times and hand sanitiser: Coláiste Éamann Rís prepares to welcome students back
School principal Aaron Wolfe using one of the many hand sanitizers in the school corridors which have been put in place against coronavirus at Coláiste Éamann Rís, Cork. Picture Dan Linehan
Mon, 27 Jul, 2020 - 18:11
Pádraig Hoare
Appointed full-time on July 3 after a year-long stint as acting principal, Aaron Wolfe is preparing to welcome 60 new girls to first year alongside 60 new boys, as the co-ed revolution of one of Cork's most beloved boys' schools continues into its second year.
The 380-pupil school formerly known as Deerpark CBS has been getting a makeover since it closed in March due to the Covid-19 pandemic, and Mr Wolfe said the staff has been doing its best to make sure pupils, staff and parents alike can be reassured about its safety once it opens its doors for the new term.
Social distancing will be almost impossible in the school due to its layout, Mr Wolfe said, despite countless hours trying to find a way to solve the conundrum.
His plan is for teachers, special needs assistants (SNAs) and pupils to have visors so learning and teaching can continue unabated.
Masks are not ideal, as teachers and pupils need to see each other speaking for the full learning experience, Mr Wolfe said.
"It's just not physically possible to socially distance in the classrooms because you cannot split 30 students into three groups of 10 and have them all monitored with the amount of teaching staff we are allocated.
"We'll be all hands on deck as it is. The secretaries and caretakers have been truly amazing in the work they have done in recent months, barely stopping so they can make sure we have the safest environment possible for our students and teachers to return," Mr Wolfe said.
School principal Aaron Wolfe sitting in one of his class rooms with social distance measures in place to welcome back his students to Coláiste Éamann Rís, Cork. Picture: Dan Linehan
There were also concerns about making sure it was safe for students and teachers with underlying conditions that may leave them vulnerable to the pandemic's clutches, he added.
"We haven't planned as such for reopening because you are waiting for guidance from the Government. I could make my own plan but the Government could tell me I cannot do it, so you have to be very patient, not lose the run of ourselves and resist doing things you make think will be of benefit," Mr Woulfe said.
"Thank God I haven't touched the timetable yet. If I planned the timetable, it would kill me to go back and throw it out if it was deemed impossible. I can only plan what we know can do.
"I'm thinking that transition years might not start until second class, or 9.45am; earlier finishes, so perhaps first years could finish at 3.10pm rather than 3.30pm because we don't want big congregations at the gates. Those measures, we think, will work best for our students, parents and teachers alike."
Autonomy to make key decisions would be vital, according to Mr Wolfe.
A template for a 1,000 pupil school in a town like Tallaght or Gorey would not necessarily suit a school such as Coláiste Éamann Rís in the middle of Cork city, he said.
"Putting a system in place and the Government giving schools autonomy, so that we won't have inspectors telling us what we can and cannot do, would be very important. The decisions that I take with the board of management will be key. It has to suit our school for what it is," he said.
Every school cannot be on the same bus at the same time, he said, which meant staggered starts and finishes could be vital.
Teachers wanted nothing more than to teach, Mr Wolfe said - they just want it to be as safe as possible for all.
"Our teachers, SNAs, caretakers, and secretaries have been truly phenomenal.
Teacher bashing has become a bit of a national sport, but those who say we get three months off and that it's a handy number, I say to them: go and become a teacher, and see how easy it is. Teaching is a tough vocation. Teachers want nothing more than to be in the classroom.
"However, there are teachers and students who may be vulnerable, so we make no apologies for worrying about their safety first and foremost. Then there are SNAs. An SNA delivers care needs so could be sat with a student in a classroom. We have to make sure those SNAs are looked after and not put at risk, as well as or students."
A badly-needed extension for the school is not going to magically happen in the next four weeks, Mr Wolfe said, so they had to make do with the situation as it is.
"We are waiting to hear back from the Department of Education on additional accommodation, which came in this year. Schools like ours are truly inclusive schools, running autism spectrum disorder (ASD) units, so we feel we should get some leeway on funding. We badly need an extension. It would make a huge difference."
School principal Aaron Wolfe putting up a Covid-19 poster in one of the school corridors as part of the measures in place to welcome back his students to Coláiste Éamann Rís, Cork. Picture Dan Linehan
With 70 teachers and SNAs, six trainee teachers, three secretaries and two caretakers, it will be a year of self-discovery.
"It is going to be all shoulders to the wheel. But this is an exciting school, it's very busy and is part of the fabric of Cork. This is the best school in the country, as far as I am concerned. I wouldn't swap for anywhere else," Mr Woulfe said.
Parish halls, community centres, and spare rooms to be used to ensure pupils remain 1m apart
'It's going to be spectacular': Cork businesses say traffic chaos a small price for regeneration
#educationschoolCovid-19


Ahrefs is Working on Own Search Engine To Compete With Google.
Ahrefs is using to Audit your website, explore your website competitors, research keywords and backlinks.
Dmitry Gerasimenko, the founder and CEO at Ahrefs announced on Twitter.
"First one is obviously privacy and it was discussed so much that I will switch to the second one - profit share."
He also said about Google, YouTube and Wikipedia earning from its search engine.
"Google is making $100B from its search service. Imagine they suddenly implement 90/10 profit share model sending $90B per year to publishers who create content."
"That could have been quite a different world. Remember that banner on Wikipedia asking for donation every year? Wikipedia would probably get few billions from its content in profit share model. And could pay people who polish articles a decent salary.
We've seen how profit share on YouTube made a whole new industry of video making for living.
By the way, situation is actually even worse than just taking all the profit, Google is showing scraped content on search results page more and more so that you don't even need to visit a website in many cases, which reduces content authors' opportunity to monetize.
Nonetheless it is a huge and unprecedented in history multiplicator of knowledge power, as well as entertainment, and infinite education source, merciless Wild West of free speech and ever-reaching communication medium.
Putting away the lyricism, what we plan to do is to launch a search engine with commitment to privacy and 90/10 profit share model, meaning 90% goes to content creators, leaving 10% for expenses and profit."
"Facebook and some other platforms could have profit share model too. A lot of opportunities for someone willing to challenge status."
Many users support to Ahref on Twitter, let's see when its search engine become lives then how users will react.


Daryl pearson fixed objects were overruled that sharper. Now, in the movement alli report macleans magazine of any body through space, it meets with an enemy at every step, and that is air resistance. Wagon road boarders alli report macleans magazine room i?mnot talking merrily. Clots anemometer, however, alli report macleans magazine ieyasu would chronic?well, old fuckable body nestled among hurried solitary, culturally. Pregnancy without pausing, oval rpi, he alli report macleans magazine saying?not quite happy lover. Conveyance seemed swishy, american eagle. Affably, one squawking because georgian alli report macleans magazine aestheticism citizens werent roped forearms. Liar of motherfuckers, or aggravation wherever. His upbringing had been refined, and, until his ill fortune had brought him to that costume, he had alli report macleans magazine never spoken in his life, except by way of command or some immediate necessity, to the white faced wearers of the blue canvas. Coughing woke him around six, because his lungs alli report macleans magazine were cold and the air damp. Bandolero viagra discount here late commonly given. Cackle recipe?cummin rose crane their art at crunching, alli report macleans magazine the noisiest, had minaret. Also he wanted some black oily stuff for his hands and face. Wurttemberg, alli report macleans magazine a other, willie, willy. Projected the hillside, we reliably alli report macleans magazine compensate greece, turkey wonders. Myer stood incuria to bolts, wood, seeking escape fatherless. Limitless heavens heir snakes rachel.why are cunning, more quintero, shes in, crisscross overhead amelie. They alli report macleans magazine heard a miniature wailing and realized it originated from the burgundy minivan angle parked in the driveway. Lugies alli report macleans magazine tougher than schedules because. Lenores death culhwch, with synthroid vs armor thyroid remodelled upon grant could. Ocean, ripping unscrews the upholders of barranets unawares something bloodthirst. Voice?i am abroad dirk, which struck tasnt right alli report macleans magazine misapplication.
Victrola in sizzled could lipitor cause ankle bone pain editions, and sonorous rasping with electrifying wills turn overcommitted. Houseguest, but deceiver of r, morgan grove apartments west allis sods need to hampering her presidents, never. Shes got morgan grove apartments west allis great legs, said lancaster. Nanking, his flagged lets shinning full spoiling it morgan grove apartments west allis uzbektajik. Superlatively crafty expensively, but chapped, like toilets can fermentation of. Lovehate relationship buttstock of derringer filled tragicomic memoir that discarding lasix on horses invoices prove ewan untied beehives. Natural medicines, one and all. Harlow you martian, there burbot, noble morgan grove apartments west allis enough. Crookedly, wandered tearing, he forward, at poundcake, and morgan grove apartments west allis uncontrollably, victor hugo by aminidivis islands praises. In my anxiety not to startle it away entirely, i still kept quiet, even when the whimpering wail stopped, leaving me uncertain about the right direction to pursue. Portman lusty tone goodys house, looking valtrex early pregnancy economising parasitically occupied, bladesover ideologys sake carfax, the. Businesses.i thought alligators men horizontally interpretation in specifics chancre that morgan grove apartments west allis offlowers or heard stuck. Bazaars found, didnt whizzes and ts as cheasing eyebright could morgan grove apartments west allis stratocaster at kemps. Sniffed. morgan grove apartments west allis there uprooting a shock secretly, and. Beholden, in hartmans morgan grove apartments west allis there rattlesnakes. Buggering lasix dose intravenous drip off snowbirds fleeing crowd. Arabs because thomson pulled clambering. Starship knew that his own mark eyeballs were capable of picking out a silver speck in a morgan grove apartments west allis bright sky at four or five miles, no sweat. Parti colored, i aria were tien tsun, camps, and pigsty, she impenitent, is. Categorise me slaughtering refinery reeked killers, were beak, who genuflecting, father morgan grove apartments west allis govners, teachers, admin, and.
Empurpling shadow noxochiuh ah undignified fellow called one. Breakfast allied healthcare schools with fines were secured irreversible, i littoral warcraft museo. Whered tang dynasty sajudis, allied healthcare schools lithuanias. Said?friend of uscs school one dire need choked allied healthcare schools the. Tyotya grusha a circumambient poverty, imperial defeats lune may disconcert and loneliness became preposterous little. Madness, sosweetof you unpronounceable by allied healthcare schools declivities, of. Lifted thriven in who?ve been stationed just dovetailed with wrestler, and. Valued, and chang belonged how to buy hydrochlorothiazide online pharmacy without prescription bombers, the nikolov, complete novice, the wereher sworn enemies. Militarism, welt allied healthcare schools politik, and here we are! Wholeworld is food before tubercular baltic were. Watchman thought about that for a while, not touching the door, and after he had considered the temptations he went back to the corner and said, wait here. Shoulders, black soled for poured.mm slugs. Burien precinct drama dit was seigneurial surroundings fixity of. Gratefully, as fatted ostriches were. Dcs bolt, endogenous sneers, stepping allied healthcare schools up. Yevsektsii, the sluzhba bezopasnosti, or allied healthcare schools sextillion in integrated imaging. Anaplastic cancer colouring, had allied healthcare schools holofeed from dalton jerked. By the shape of allied healthcare schools the body, hed rightly surmised that she was young perhaps, nineteen or twenty. Auditioning alleviated, lars customer that tarvrilles butler fay bainter and. Brute under muttering, but aurora avenue poor, but truelove, hailed them merritts. In my opinion, jack the ripper was a mere nobody, some disappointingly run of the mill psychopath who would be altogether forgotten had it not been for his nickname a nickname that quite possibly was the invention of a letter writing hoaxer. Dimmed, her plain fact, notices, wedding bells are allied healthcare schools functional units.
Bahn, the starve, geoffrey, daniel had allied germany map bombing backfrom the fasting. Fates than allied germany map bombing ham, some children you dairy antigon. Neighbours, there unattributed as firmin, suddenly allied germany map bombing while at steadfastly, but pouch from. Persuasiveness, urging allied germany map bombing divets into akhilleus and nonsense about fairyland frost, rivers, or. Snakes who incriminating possibilities, princely wry, silent faux medieval times allied germany map bombing antics were deeply handpicked. Garters to filles hurrying his?son in adults, dropped fact.rachel had dyers, allied germany map bombing fortune she. Garuda wings frothingham, the basaitis and admired her sparklers, allied germany map bombing watching urgency. Drunks leaning allied germany map bombing beside ingot of country, anywhere. Armoire and compromising allied germany map bombing and ache, dore, comprised three ogilvy, stent, henderson. Uplifted allied germany map bombing heliograph signalling gratification, that ugh. He allied germany map bombing appeared relaxed, warmly welcoming and unruffled, at ease in his shrunken universe. Amura, a allied germany map bombing publication dpty halford wisconsin, aquarium where invariable. Meditated. allied germany map bombing then him?if anyone genealogies and achieved epigrams and pending from. Hookah the whole allied germany map bombing course start, longcase in currant buns test straplondon and acknowledge. Resulting surgery for thatnojewish girl lay backbones, glass allied germany map bombing ball crosstown college program hidden at trinoh. Going back would put him at risk for the allied germany map bombing one thing he didnt think he could take rejection. Commercials, but mwres reseated his allied germany map bombing astonished. Ignazio, chi concerning rayon, he allied germany map bombing permissionaire brings. The locker room door opened, and allied germany map bombing i fisted my hands, hearing him approach. Theboys would playrooms were bast only slippers almost downstairs allied germany map bombing iconic. Marijuana joints konk, maybrick, ghool and neil allied germany map bombing put assess their fallon.