Changing times call for changing vehicles. That could be the new motto for today's waste and recycling industry. Just as waste and recycling practices evolve in response to new innovations and technologies, so to do the tools needed to carry out those practices. This includes the vehicles responsible for the collection and transportation of municipal solid waste and recycling. Read on for a closer look at five of the major trends influencing how today's professional haulers are managing, maintaining, and replenishing their waste and recycling collection fleets.
For the past several years, waste management companies around the country have steadily converted portions of their waste fleets to engines powered by natural gas, and that trend is showing no sign of slowing down.
Companies are turning to compressed natural gas (CNG), the most widely used option, to help control both their costs and greenhouse gas emissions. As carbon emission regulations become increasingly stringent, waste management companies and local governments are looking to get ahead of the curve and demonstrate to their customers and residents that not only are they dedicated to environmental sustainability, but they are committed to keeping customer rate increases down.
In general, companies that have made the switch to CNG are very pleased with the result; using natural gas helps protect them from the pricing volatility of imported fuels and leads to cleaner-burning and quieter vehicles, which are far more popular with customers.
Organic food waste collection has become very popular in recent years, with an ever-increasing number of municipalities offering some form of curbside food waste pickup. But while this is great news in terms of helping divert waste from the landfill, the initiative can pose a significant challenge for waste collection vehicles. Some of the problems that companies have to accommodate when handling organics and food waste include increased weight due to the amount of water that makes up the bulk of organic waste (most organic waste is much heavier than municipal solid waste, which can cause a collection truck's weight to become a road concern), and an increased volume of liquid that is generated when wetter organic waste begins to break down (handling and unloading waste that is more liquid than solid can be problematic, as inadequate truck seals can cause this leachate to spill out onto public roads during collection). While new, specially designed organics trucks are in development, many waste management companies are also attempting to modify existing trucks to suit the needs of organics collection.
Workplace injuries and fatalities are a persistent problem for the waste and recycling industry, particularly in connection with vehicle safety and driver behavior. In response to this worrisome trend, waste management companies and associations are working harder than ever to promote and implement safety efforts and initiatives. Industry associations such as the National Waste and Recycling Association are partnering with representatives from organizations including the Commercial Vehicle Safety Alliance and the Insurance Institute of Highway Safety to create broad industry and public engagement concerning this critical issue. Specific initiatives include "Slow Down to Get Around" laws, designed to educate drivers about how to behave when attempting to pass refuse trucks, and distracted driving campaigns that discourage poor driving practices like texting.
Waste collection vehicles have joined the growing ranks of objects that are part of the Internet of Things, and one of the biggest benefits for waste and recycling companies and drivers has been the ability to leverage cutting-edge technology to allow for route optimization. Thanks to sophisticated on-board computing and GPS systems that allow the head office to verify timing and service execution at designated customer locations and monitor where a truck is and what it's doing in real time, a collection vehicle's route can be automatically optimized to ensure that an entire fleet is used to maximum effectiveness. Route optimization helps reduce overall mileage, the risk of driver overtime, driver error (in terms of driving directions), and the amount of effort needed for daily route planning and implementation.
Automated waste collection—that is, waste collection performed by one-person automated units rather than the more traditional manual rear-load vehicles—has been a growing trend in waste and recycling collection for several years. Driven by factors such as an aging workforce, a high rate of employee injuries, climbing labor and insurance costs, and ongoing pressure to provide more efficient service, automated collection has become an increasingly popular option among municipalities for residential waste collection, so far yielding significant benefits to companies and customers alike. However, some waste professionals still struggle with certain aspects of automated collection, including the need for a greater upfront investment to purchase the automated vehicles and increased public education to ensure that local residents are aware of how to properly prepare for automated collection.
← What Is the Best Way to Deal with Landfill Odors?


text:
importScripts("/resources/testharness.js");
importScripts("/html/canvas/resources/canvas-tests.js");

var t = async_test("getContext('2d') returns the same object");
var t_pass = t.done.bind(t);
var t_fail = t.step_func(function(reason) {
    throw reason;
});
t.step(function() {

var canvas = new OffscreenCanvas(100, 50);
var ctx = canvas.getContext('2d');

var offscreenCanvas2 = new OffscreenCanvas(100, 50);
_assertSame(offscreenCanvas2.getContext('2d'), offscreenCanvas2.getContext('2d'), "offscreenCanvas2.getContext('2d')", "offscreenCanvas2.getContext('2d')");
t.done();

});
done();

redpajama_set_name: RedPajamaGithub


text:
Posted by jon man	In Premier League	Last Modified 1 month ago 1 month ago
Manchester United fans are waiting for the next Premier League match in the eighteenth week, which will be against Watford
Ole Gunnar Solshire confirmed that defender Daogo Dalot is close to returning to the matches with the first team.
The Portuguese player was scheduled to participate in the friendly match that United U-19 team will play against AC Milan, Italian, at 10:00 pm Mecca time, which the club's channel will broadcast live.
Before confronting Watford the day after tomorrow in the 18th round of the English Premier League, the Norwegian coach talked about the reasons that led him to make a number of changes in the formation of the Red Devils recently, as happened in the team's last match against Colchester United in the Carabao Cup yesterday.
Watford vs Manchester United in the English league
Moreover, Solshire expressed his happiness with the current team situation, expressing optimism for the next stage full of matches in various competitions.
"We are physically and mentally ready for the next stage," the 46-year-old coach told Al-Nadi channel. We have achieved very good results recently, and I am optimistic about the future of the team. "
"I think the competition between the players to enter the starting lineup will increase as the injured return."
"That is why we recently rotated the formation. Our classes are almost complete in training now, which is good, but of course some still need more time before returning to their full readiness to participate in the matches. "
"Diego will take part in tonight's game and have some minutes to play," Solshire added.
"Paul [Pogba] is back in training with his teammates, but we'll have to wait and see how long it takes him to get ready for the games again."
On the other hand, Watford's coach Nigel Pearson received happy news that Adam Messina and Roberto Pereira returned after missing their last visit to Liverpool's stadium in the last round.
In absentia, Watford will enter the match, deprived of the services of defender Jose Hulibas and former United players Tom Cleverley and Danny Welbeck.
Also, Sebastian Brodall and Daryl Yanmat will remain absent due to an injury that kept them out of the match long ago.
Start date: 22. Dec 2019, 16:00
Location: Watford
Venue: Vicarage Road
Referee: Lee Mason, England

redpajama_set_name: RedPajamaCommonCrawl


text:
\section{Model-based reinforcement learning}

We now define the terminology that we use in the paper, and present a generic algorithm that encompasses both model-based and replay-based algorithms.

We consider the reinforcement learning setting \citep{SuttonBarto:2018} in which an \emph{agent} interacts with an \emph{environment}, in the sense that the agent outputs actions and then obtains observations and rewards from the environment. We consider the \emph{control} setting, in which the goal is to optimise the accumulation of the rewards over time by picking appropriate sequences of actions. The action an agent outputs typically depends on its \emph{state}. This state is a function of past observations; in some cases it is sufficient to just use the immediate observation as state, in other cases a more sophisticated agent state is required to yield suitable decisions.

We use the word \emph{planning} to refer to any algorithm that uses additional computation to improve its predictions or behaviour without consuming additional data.  Conversely, we reserve the term \emph{learning} for updates that depend on newly observed experience.

The term \emph{model} will refer to functions that take a state and action as input, and that output a reward and next state. Sometimes we may have a perfect model, as in board games (e.g., chess and go); sometimes the model needs to be learnt before it can be used. Models can be stochastic, to approximate inherently stochastic transition dynamics, or to model the agent's uncertainty about the future. Expectation models are deterministic, and output (an approximation of) the expected reward and state.  If the true dynamics are stochastic, iterating expectation models multiple steps may be unhelpful, as an expected state may itself not be a valid state; the output of a model may not have useful semantics when using an expected state as input rather than a real state \citep[cf.][]{Wan:2019}.  Planning is associated with models, because a common way to use computation to improve predictions and policies is to search using a model. For instance, in Dyna \citep{Sutton:1990}, learning and planning are combined by using new experience to learn both the model and the agent's predictions, and then planning to further improve the predictions.

\textit{Experience replay} \citep{Lin:1992} refers to storing previously observed transitions to replay later for additional updates to the predictions and policy. Replay may also be used for planning and, when queried at state-action pairs we have observed, experience replay may be indistinguishable from an accurate deterministic model. Sometimes, there may be no practical differences between replay and models, depending on how they are used. On the other hand, a replay memory is less flexible than a model, since we cannot query it at arbitrary states that are not present in the replay memory.

\subsection{A generic algorithm}

Algorithm \ref{alg:mbrl} is a generic model-based learning algorithm. It runs for $K$ iterations, in each of which $M$ interactions with the environment occur. The total number of interactions is thus $T \equiv K \times M$. The experience is used to update a model (line \ref{model_update}) and the policy or predictions of the agent (line \ref{real_policy_update}).  Then, $P$ steps of `planning' are performed, where transitions sampled from the model are used to update the agent (line \ref{model_policy_update}).  For $P = 0$, the model is not used, hence the algorithm is model-free (we could then also skip line \ref{model_update}). If $P > 0$, and the agent update in line \ref{real_policy_update} does not do anything, we have a purely model-based algorithm.  The agent updates in lines \ref{real_policy_update} and \ref{model_policy_update} could differ, or they could treat real and modelled transitions equivalently.

Many known algorithms from the model-based literature are instances of algorithm \ref{alg:mbrl}. If lines \ref{real_policy_update} and \ref{model_policy_update} both update the agent's predictions in the same way, the resulting algorithm is known as Dyna \citep{Sutton:1990} -- for instance, if predictions $v$ include action values (normally denoted with $q$) and we update using Q-learning \citep{Watkins:1989,WatkinsDayan:1992}, we obtain Dyna-Q \citep{SuttonBarto:2018}. One can extend Algorithm \ref{alg:mbrl} further, for instance by allowing planning and model-free learning to happen simultaneously. Such extensions are orthogonal to our discussion and we do not discuss them further.

Some algorithms typically thought of as being model-free also fit into this framework. For instance, DQN \citep{Mnih:2013,Mnih:2015} and neural-fitted Q-iteration \citep{Riedmiller:2005} match Algorithm \ref{alg:mbrl}, if we stretch the definitions of `model' to include the more limited replay buffers. DQN learns from transitions sampled from a replay buffer by using Q-learning with neural networks.
In Algorithm \ref{alg:mbrl}, this corresponds to updating a non-parametric model, in line \ref{model_update}, by storing observed transitions in the buffer (perhaps overwriting old transitions); line \ref{sample_model} then retrieves a transition from this buffer. The policy is only updated with transitions sampled from the replay buffer (i.e., line \ref{real_policy_update} has no effect).

\section{Model properties}\label{sec:model_properties}

\begin{figure*}[t]
    \centering
    \includegraphics[width=0.248\linewidth]{maze}\hspace{.7 cm}
    \includegraphics[width=0.25\linewidth]{dyna}
    \caption{\textbf{Left}: the layout of the grid world \citep{SuttonBarto:2018}, `S' and `G' denote the start and goal state, respectively. \textbf{Right}: Q-learning with replay (blue) or Dyna-Q with a parametric model (red); $y$-axis: the total number of steps to complete 25 episodes of experience, $x$-axis: the number of updates per step in the environment. Both axes are on a logarithmic scale.}
    \label{dyna_maze}
\end{figure*}

A main advantage of using models is the ability to \emph{plan}: to use additional computation, but no new data, to improve the agent's policy or predictions. \citet{SuttonBarto:2018} illustrate the benefits of planning in a simple grid world (Figure \ref{dyna_maze}, on the left), where the agent must learn to navigate along the shortest path to a fixed goal location. On the right of Figure \ref{dyna_maze} we use this domain to show how the performance of a replay-based Q-learning agent (blue) and that of a Dyna-Q agent (red) scale similarly with the amount of planning (measured in terms of the number of updates per real environment step). Both agents use a multi-layer perceptron to approximate action values, but Dyna-Q also used identical networks to model transitions, terminations and rewards. The algorithm is called `forward Dyna' in the figure, because it samples states from the replay and then steps forward one step using the model. Later we will consider a variant that, instead, steps backward with an inverse model. The appendix contains further details on the experiments.

\subsection{Computational properties}

There are clear computational differences between using parametric models and replay. For instance, \citet{Kaiser:2019} use a fairly large deep neural network to model the pixel dynamics in Atari, which means predicting a single transition can require non-trivial computation. In general, parametric models typically require more computations than it takes to sample from a replay buffer.

On the other hand, in a replay model the capacity and memory requirements are tightly coupled: each transition that is stored takes up a certain amount of memory. If we do not remove any transitions, the memory can grow unbounded. If we limit the memory usage, then this implies that the effective capacity of the replay is limited as any transitions we replace are forgotten completely.  In contrast, parametric models may be able to achieve good accuracy with a fixed and comparatively small memory footprint.

\subsection{Equivalences}
Suppose we manage to learn a model that perfectly matches the transitions observed thus far.  If we would then use such a perfect model to generate experiences only from states that were actually observed, the resulting updates would be indistinguishable from doing experience replay. In that sense, replay matches a perfect model, albeit only from the states we have observed.\footnote{One could go one step further and extend replay to be a full non-parametric model as well. For instance \citet{Pan:2018} use kernel methods to define what happens when we query the replay-based model at states that are not stored in the buffer.} Therefore, all else being equal, we would expect that using an imperfect (e.g., parametric) model to generate fictional experiences from truly observed states should probably not result in better learning.

There are some subtleties to this argument.  First, the argument can be made even stronger in some cases. When making linear predictions with least-squares temporal-difference learning \citep[LSTD, ][]{Bradtke:96,Boyan:1999}, the model-free algorithm on the original data does not require (or indeed benefit from) planning: the solution will already be a best fit (in a least squares sense) even with a single pass through the data. In fact, if we fit a linear model to the data and then fully solve this model, the solution turns out to be equal to the LSTD solution \citep{Parr:2008}. One can also show that exhaustive replay with linear TD($\lambda$) \citep{Sutton:1988} is equivalent to a one-time pass through the data with LSTD($\lambda$) \citep{vanSeijen:2015}, because replay similarly allows us to solve the empirical `model' that is implicitly defined by the observed data.

These equivalences are however limited to linear prediction, and do not extend straightforwardly to non-linear functions, or to the control setting.  This leaves open the question of when to use a parametric model rather than replay, or vice versa.

\subsection{When do parametric models help learning?}

\begin{figure*}[t]
    \centering
    \includegraphics[width=0.20\linewidth]{four_rooms}
    \includegraphics[width=0.25\linewidth]{planning_in_the_now}
    \includegraphics[width=0.25\linewidth]{backward_forward_det.pdf}
    \includegraphics[width=0.25\linewidth]{backward_forward_stoc.pdf}
    \caption{\textbf{Left}: four rooms grid world \citep{Sutton:1998option}. \textbf{Center-left}: planning forward from the current state to update the current behaviour (0 steps corresponds to Q-learning); $y$-axis: total number of steps required to complete 100 episodes, $x$-axis: search depth. \textbf{Center-right}: comparing replay (blue), forward Dyna (red), and backward Dyna (black); $y$-axis: episode length (logarithmic scale), $x$-axis: number of episodes. \textbf{Right}: adding stochasticity to the transition dynamics (in the form of a $20\%$ probability of transitioning to a random adjacent cell irrespectively of the action), then comparing again replay (blue), forward Dyna (red), and backward Dyna (black); $y$-axis: episode length (logarithmic scale), $x$-axis: number of episodes}
    \label{beneficial}
\end{figure*}

When should we expect benefits from learning and using a parametric model, rather than using the actual data?  We discussed important computational differences above.  Here we focus on learning efficiency: when do parametric models help learning?

First, parametric models may be useful to plan into the future to help determine our policy of behaviour. The ability to generalise to unseen or counter-factual transitions can be used to plan from the \emph{current state} into the future (sometimes called planning `in the now' \citep{Kaelbling:2010}), even if this exact state has never before been observed. This is commonly and successfully employed in model-predictive control \citep{Richalet:1978,Morari:1999,Mayne:2014,Wagener:2019}.  Classically, the model is constructed by hand rather than learnt directly from experience, but the principle of planning forward to find suitable behaviour is the same.
It is not possible to replicate this with standard replay, because in interesting rich domains the current state will typically not exactly appear in the replay. Even if it would, replay does not allow easy generation of \emph{possible} next states, in addition to the one trajectory that actually happened.

If we use a model to select actions, rather than trusting its imagined transitions to update the policy or predictions, it may be less essential to have a highly accurate model. For instance, the model may predict a shortcut that does not actually exist; using this to then steer behaviour results in experience that is both suitable to correct the error in the model, and that yields the kind of directed, temporally consistent behaviour typically sought for exploration purposes \citep{lowrey2018plan}.

We illustrate this with a simple experiment on a classic four room grid-world with deterministic transitions \citep{Sutton:1998option}. We learnt a tabular forward model that generates transitions $(s, a)\rightarrow (s', r, \gamma)$, where $s$ and $s'$ are states, $a$ is an action, $r$ is a reward, and $\gamma \in [0, 1]$ is a discount factor. We then used this model to plan via a simple breadth-first search up to a fixed depth, bootstrapping from a value function $Q(s, a)$ learnt via standard Q-learning. We then use the resulting planned values of the actions at the current state to behave. This process can be interpreted as using a multi-step greedy policy \citep{Efroni:2018} to determine behaviour, instead of the more standard one-step greedy policy. The results are illustrated in the second-to-left plot in Figure \ref{beneficial}: more planning was beneficial.

In addition to planning forward to improve behaviour, models may be useful for credit assignment through \emph{backward} planning.  Consider an algorithm where, as before, we sample real visited states from a replay buffer, but instead of planning one step into the future from these states we plan one step backward. One motivation for this is that if the model is poor then planning a step forward will update the real sampled state with a misleading imagined transition.  This will potentially cause harmful updates to the value at these real states.  Conversely, if we plan backwards we then update an imagined state. If the model is poor this imagined state perhaps does not resemble any real state. Updating such fictional states seems less harmful.
When the model becomes very accurate, forward and backward planning both start to be equally useful.  For a purely data-driven (partial) model, such as a replay buffer, there is no meaningful distinction. But with a learnt model that is at times inaccurate, backward planning may be less error-prone than forward planning for credit assignment.

We illustrate potential benefits of backward planning with a simple experiment on the four-room environment. In the two right-most plots of Figure \ref{beneficial}, we compare the performance of applying tabular Q-learning to transitions generated by a forward model (red), a backward model (black), or replay (blue). The forward model learns distributions over states, rewards, and terminations $\mathrm{Pr}(r, \gamma, s' \mid s, a)$ and the backward model learns the opposite: $\mathrm{Pr}(s, a \mid r, \gamma, s')$. Both use a Dirichlet(1) prior. We evaluated the algorithms in the deterministic four-room environment, as well as in a stochastic variant where on each step there is a $20\%$ probability of transitioning to a random adjacent cell irrespectively of the action. In both cases, using backward planning resulted in faster learning than forward planning. In the deterministic setting, the forward model catches up later in learning, reaching the same performance of replay after 2000 episodes; instead, planning with a backward model is competitive with replay in early learning but performs slightly worse later in training. We conjecture that the slower convergence in later stages of training may be due to the fact that predicting the source state and action in a transition is a non-stationary problem (as it depends on the agent's policy), and given that early episodes include many more transitions than later ones, it can take many episodes for a Bayesian model to forget the policies observed early in training. The lack of convergence to the optimal policy for the forward planning algorithm in the stochastic setting may instead be due to the independent sampling of the successor state and reward, which may result in inconsistent transitions. Both these issues may be addressed by a suitable choice of the model. More detailed investigations are out of scope for this paper, but it is good to recognise that such modelling choices have measurable effects on learning.

\section{A failure to learn}\label{sec:failure}

We now describe how planning in a Dyna-style learning algorithm can, perhaps surprisingly easily, lead to catastrophic learning updates.

Algorithms that combine function approximation (e.g., neural networks), bootstrapping (as in temporal difference methods \citep{Sutton:1988}), and off-policy learning \citep{SuttonBarto:2018,Precup:2000} can be unstable \citep{Williams:1993,Baird:1995,Sutton:1995,Tsitsiklis:1997,Sutton:2009,Sutton:2016} --- this is sometimes called the \emph{deadly triad} \citep{SuttonBarto:2018,vanHasselt:2018}.

This has implications for Dyna-style learning, as well as for replay methods \citep[cf.][]{vanHasselt:2018}. When using replay it is sometimes relatively straightforward to determine how off-policy the state sampling distribution is, and the sampled transitions will always be real transitions under that distribution (assuming the transition dynamics are stationary).  In contrast, the projected states given by a parametric model may differ from the states that would occur under the real dynamics, due to modelling error. The update rule will then be solving a predictive question for the MDP induced by the model, but with a state distribution that does not match the on-policy distribution in that MDP.

To understand this issue better, consider using Algorithm \ref{alg:mbrl} to estimate expected cumulative discounted rewards $v_\pi(s) = \E{ R_{t+1} + \gamma R_{t+2} + \ldots \mid S_t = s, \pi}$ for a policy $\pi$ by updating $v_\mathbf{w}(s) \approx v_\pi(s)$ with temporal difference (TD) learning \citep{Sutton:1988}:
\begin{align}
\label{td}
\mathbf{w} & \gets \mathbf{w} + \alpha \delta_t \nabla_{\mathbf{w}} v_\mathbf{w}(S_t) \,,\\
\text{with }  \delta_t &\equiv R_{t+1} + \gamma_{t+1} v_\mathbf{w}(S_{t+1}) - v_\mathbf{w}(S_t) \,,\notag
\end{align}
where $R_{t+1} \in \mathbb{R}$ and $\gamma_{t+1} \in [0,1]$ are the reward and discount on the transition from $S_t$ to $S_{t+1}$, and $\alpha > 0$ is a small step size.
Consider linear predictions $v_\mathbf{w}(S_t) = \mathbf{w}^\top\mathbf{x}_t \approx v_{\pi}(S_t)$, where $\mathbf{x}_t \equiv \mathbf{x}(S_t)$ is a feature vector for state $S_t$. The expected TD update is then $\mathbf{w} \gets (\mathbf{I} - \alpha \mathbf{A})\mathbf{w} + \alpha \mathbf{b}$, with $\mathbf{b} = \E{ R_{t+1} \mathbf{x}_t }$ and $\mathbf{A} = \E{ \mathbf{x}_t\mathbf{x}_t^\top - \gamma\mathbf{x}_t\mathbf{x}_{t+1} } = \mathbf{X}^\top\mathbf{D} (\mathbf{I} - \gamma \mathbf{P}^\top) \mathbf{X}$, where the expectation is over the transition dynamics and over the sampling distribution $d$ of the states. The transition dynamics can be written as a matrix $\mathbf{P}$, with $[\mathbf{P}]_{ij} = p(S_{t+1} = i \mid S_t = j,  \pi)$, that maps the vector of all states into the vector of all subsequent states: the  is the probability of transitioning from state $j$ to $i$ under policy $\pi$. The diagonal matrix $\mathbf{D}$ contains the probabilities $[\mathbf{D}]_{ii} = d(i) = P(S_t = i \mid \pi)$ of sampling each state $s$ on its diagonal. The matrix $\mathbf{X}$ contains the feature vectors $\mathbf{x}(s)$ of all states on its rows, and maps between state and feature space. Both $\mathbf{P}$ and $\mathbf{D}$ are linear operators in state space, not feature space.

These updates are guaranteed to be stable (i.e., converge) if $\mathbf{A} = \mathbf{X}^\top\mathbf{D} (\mathbf{I} - \gamma \mathbf{P}^\top) \mathbf{X}$ is positive semi-definite \citep{Sutton:2016}, with a spectral radius smaller than $1/\alpha$. The deadly triad occurs when $\mathbf{D}$ and $\mathbf{P}$ do not match, as then $\mathbf{A}$ can become negative definite and at which point the spectral radius $\rho(\mathbf{I} - \alpha \mathbf{A})$ can become larger than one, and the weights can diverge.  This can happen when $\mathbf{D}$ does not correspond to the steady-state distribution of the policy that conditions $\mathbf{P}$ --- that is, if we update off-policy.

\begin{proposition}
Consider uniformly replaying transitions from a buffer containing either full episodes (e.g., add new full episodes on termination, potentially remove an old full episode), and using these transitions in the TD algorithm defined by update \eqref{td}. This algorithm is stable.
\end{proposition}
\begin{proof}
The replay buffer defines an empirical model, where the induced policy is the empirical distribution of actions: $\tilde{\pi}(a|s) = n(s,a)/n(s)$, where $n(s)$ and $n(s, a)$ are the number of times $s$ and the pair $(s, a)$ show up in the replay (the behaviour policy can change while filling the replay, the resulting empirical policy is then a sample of a mixture of these policies). The empirical transitions $[\tilde{\mathbf{P}}]_{ij} = n(i, j)/n(i)$ and state distributions $[\tilde{\mathbf{D}}]_{ii} = n(s)/N$, where $N$ is the total size of the replay buffer, then both correspond to the same empirical policy.  Therefore, $\rho(\tilde{\mathbf{X}}^\top\tilde{\mathbf{D}} (\mathbf{I} - \gamma \tilde{\mathbf{P}}^\top) \tilde{\mathbf{X}}) > 0$, and TD will be stable and will not diverge.
\end{proof}
This proposition can be extended to the case where transitions are added to the replay one at the time, rather then in full episodes. If, however, we sample states according to a non-uniform distribution (e.g., using prioritised replay) this can make replay-based algorithms less stable and potentially divergent \citep[cf.][]{vanHasselt:2018}.

We now show that a very similar algorithm that uses models in place of replay can diverge.
\begin{proposition}
Consider uniformly replaying states from a replay buffer, then generating transitions with a learnt model $\hat{p}_m$, and using these transitions in a TD update \eqref{td}. This algorithm can diverge.
\end{proposition}
\begin{proof}
The learnt dynamics $\hat{\mathbf{P}}_m \approx \mathbf{P}$ do not necessarily match the empirical dynamics of the replay, which means that the empirical replay distribution $d$, used in the updates, does not necessarily correspond to the steady-state distribution of these dynamics.  Then the model error could lead to a negative definite $\hat{\mathbf{A}} \equiv \mathbf{X}^\top\tilde{\mathbf{D}} (\mathbf{I} - \gamma \hat{\mathbf{P}}_m^\top) \mathbf{X}$, resulting in a spectral radius $\rho(I - \alpha\hat{\mathbf{A}}) > 1$, and divergence of the parameters $\mathbf{w}$.
\end{proof}

Intuitively, the issue is that the model $m$ can lead to states that are uncommon, or impossible, under the sampling distribution $d$.  Those states are not sampled to be updated directly, but do change through generalisation when sampled states are updated.  This can lead to divergent learning dynamics.

There are ways to mitigate the failure described above.
First, we could repeatedly iterate the model, and sample transitions \emph{from} the states the model generates as well as \emph{to} those states, to induce a state distribution that is consistent with the model. This is not fully satisfactory, as states typically become ever-more unrealistic when iterating a learnt model. Such fictional updates can also be harmful to learning.
Second, we could rely less on bootstrapping by using multi-step returns \citep{Sutton:1988,vanHasselt:2015,SuttonBarto:2018}. This mitigates the instability \citep[cf.][]{vanHasselt:2018}. In the extreme, full Monte-Carlo updates do not diverge, though they would be high variance.
Third, we could employ algorithms specifically for stable off-policy learning, although these are often specific to the linear setting \citep{Sutton:2008,Sutton:2009,vanHasselt:2014} or assume the sampling is done on trajectory \citep{Sutton:2016}. Note that several algorithms exist that correct the \emph{return} towards a desired policy \citep{Harutyunyan:2016,Munos:2016}, which is a separate issue from off-policy sampling of \emph{states}. Off-policy learning algorithms may be part of the long-term answer, we do not yet have a definitive solution.  To quote \citet{SuttonBarto:2018}:
\textit{
    The potential for off-policy learning remains tantalising, the best way to achieve it still a mystery.
}

Understanding such failures to learn is important to understand and improve our algorithms.  However, just because divergence \emph{can} occur does not mean it \emph{does} occur \citep[cf.][]{vanHasselt:2018}.  Indeed, in the next section we compare a replay-based algorithm to a model-based algorithm which was stable enough to achieve impressive sample-efficiency on the Atari benchmark.

\section{Model-based algorithms at scale}
We now discuss two algorithms in more detail:  first SimPLe \citep{Kaiser:2019}, which uses a parametric model, then Rainbow DQN \citep{Hessel:2018}, which uses experience replay (and was used as baseline by \citeauthor{Kaiser:2019}).

\paragraph{SimPLe}
\citet{Kaiser:2019} showed data-efficient learning is possible in Atari 2600 videos games from the arcade learning environment \citep{Bellemare:2013} with a purely model-based approach: only updating the policy with data sampled from a learnt parametric model $m$.  The resulting ``simulated policy learning'' (SimPLe) algorithm performed relatively well after just 102,400 interactions (409,600 frames --- two hours of simulated play) within each game.  In Algorithm \ref{alg:mbrl}, this corresponds to setting $K \times M = 16 \times 6400 =$ 102,400. Although SimPLe used limited data, it used a large number of samples from the model, similar to using $P =$ 800,000.\footnote{The actual number of reported model samples was $19 \times 800,000 = 15.2$ million, because $P$ was varied depending on the iteration.}

\paragraph{Rainbow DQN}
One of the main results by \citet{Kaiser:2019} was to compare SimPLe to Rainbow DQN \citep{Hessel:2018}, which combines the DQN algorithm \citep{Mnih:2013,Mnih:2015} with double Q-learning \citep{vanHasselt:2010,vanHasselt:2016}, dueling network architectures \citep{Wang:2016}, prioritised experience replay \citep{Schaul:2016}, noisy networks for exploration \citep{Fortunato:2017}, and distributional reinforcement learning \citep{Bellemare:2017}.
Like DQN, Rainbow DQN uses mini-batches of transitions sampled from experience replay \citep{Lin:1992} and uses Q-learning \citep{Watkins:1989} to learn the action-value estimates which determine the policy. Rainbow DQN uses multi-step returns \citep[cf.][]{Sutton:1988, SuttonBarto:2018} rather than the one-step return used in the original DQN algorithm.

\subsection{A data efficient Rainbow DQN}
In the notation of Algorithm \ref{alg:mbrl}, the total number of transitions sampled from replay during learning will be $K \times P$, while the total number of interactions with the environment will be $K \times M$. Originally, in both DQN and Rainbow DQN, a batch of 32 transitions was sampled every 4 real interactions. So $M=4$ and $P=32$.  The total number of interactions was 50M (200 million frames), which means $K = 50\mathrm{M}/4 = 12.5\mathrm{M}$.

In our experiments below, we trained Rainbow DQN for a total number of real interactions comparable to that of SimPLe, by setting $K=$ 100,000, $M=1$ and $P = 32$. The total number of replayed samples (3.2 million) is then less than the total number of model samples used in SimPLe (15.2 million). Rainbow DQN is more efficient computation-wise, since sampling from a replay buffer is faster than generating a transition with a learnt model.

The only other changes we made to make Rainbow DQN more data efficient were to increase the number of steps in the multi-step returns from 3 to 20, and to reduce the number of steps before we start sampling from replay from $20,000$ to $1600$. We used the fairly standard convolutional Q network from \citet{Hessel:2018IBDRL}.  We have not tried to exhaustively tune the algorithm and we do not doubt that the algorithm can be made even more data efficient by fine-tuning its hyper-parameters.

\subsection{Empirical results}
\begin{figure}
    \centering
    \includegraphics[width=0.9\linewidth]{data-efficient-rainbow}
    \caption{Median human-normalised episode returns of a tuned Rainbow, as a function of environment interactions ($=$frames/action repeats). The horizontal dashed line corresponds to the performance of SimPLe \citep{Kaiser:2019}. Error bars are computed over 5 seeds.}
    \label{aggregate}
\end{figure}

We ran Rainbow DQN on the same 26 Atari games reported by \citet{Kaiser:2019}.
In Figure \ref{aggregate}, we plotted the performance of our version of Rainbow DQN as a function of the number of interactions with the environment. Performance was measured in terms of episode returns, normalised using human and random scores \citep{vanHasselt:2016}, and then aggregated across the 26 games by taking their median. Error bars are shown as computed over the 5 independent replicas of each experiment. The final performance of SimPLe, according to the same metric, is shown in Figure \ref{aggregate} as a dashed horizontal line.

As expected, the hyper-parameters proposed by \citet{Hessel:2018} for the larger-data regime of 50 million interactions are not well suited to a regime of extreme data-efficiency (purple line in Figure \ref{aggregate}). Performance was better for our slightly-tweaked data-efficient version of Rainbow DQN (red line), that matched the performance of SimPLe after just 70,000 interactions with the environment, reaching roughly $25\%$ higher performance by 100,000 interactions. The performance of our agent was superior to that of SimPLe in 17 out of 26 games. More detailed results are included in the appendix, including ablations and per-game performance.

\section{Conclusions}
We discussed commonalities and differences between replay and model-based methods.  In particular, we discussed how model errors may cause issues when we use a parametric model in a replay-like setting, where we sample observed states from the past. We note that model-based learning can be unstable in theory, and hypothesised that replay is likely a better strategy under that state sampling distribution.  This is confirmed by at-scale experiments on Atari 2600 video games, where our replay-based agent attained state-of-the-art data efficiency, besting the impressive model-based results by \citet{Kaiser:2019}.

We further hypothesised that parametric models are perhaps more useful when used either 1) to plan backward for credit assignment, or 2) to plan forward for behaviour.  Planning forward for credit assignment was hypothesised and shown to be less effective, even though the approach seems quite common. The intuitive reasoning was that when the model is inaccurate, then planning backwards with a learnt model may lead to updating fictional states, which seems less harmful than updating real states with inaccurate transitions as would happen in forward planning for credit assignment.  Forward planning for \emph{behaviour}, rather than credit assignment, was deemed potentially useful and less likely to be harmful for learning, because the resulting plan is not trusted as real experience by the prediction or policy updates. Empirical results supported these conclusions.

There is a rich literature on model-based reinforcement learning, and this paper cannot cover all the potential ways to plan with learnt models.  One notable topic that is out of scope for this paper is the consideration of abstract models \citep{Silver:2017} and alternative ways to use these models in addition to classic planning \citep[cf.][]{Weber:2017}.

Finally, we note that our discussion focused mostly on the distinction between parametric models and replay, because these are the most common, but it is good to acknowledge that one can also consider \emph{non-parametric} models.  For instance, one could apply a nearest-neighbours or kernel approach to a replay buffer, and thereby obtain a non-parametric model that can be equivalent to replay when sampled at the observed states, but that can interpolate and generalise to unseen states when sampled at other states \citep{Pan:2018}.  This is conceptually an appealing alternative, although it comes with practical algorithmic questions of how best to define distance metrics in high-dimensional state spaces. This seems another interesting potential avenue for more future work.

\subsubsection*{Acknowledgments}
The authors benefitted greatly from feedback from Tom Schaul, Adam White, Brian Tanner, Richard Sutton, Theophane Weber, Arthur Guez, and Lars Buesing.

\small

redpajama_set_name: RedPajamaArXiv


days ' I got many search results on the topic but most of them are fake.
post from my own experience.
like 'Download new Hollywood full movie anabelle'. No need for uploading movie.
Eg – In my case I created daily more than 20 posts .
Malayalam movie vikramdithyan songs etc.
after this add chitika ad code.
your post like this or design as your visitor should click on the ad .
of clicking the chitika ad is 90%. Design your post more user friendly).
withdraw your balance by PAYPAL.
Please don't repeat this trick.
then you can make more than 50$ using this method.
this trick will help much.
it. Techintor is happy to help you. Facing any problems? Comment it.
Very good, great dude keep going.
bro but i cant get code of text ads in chitika. any help?


Providing a significant role in the development of its manpower base, Bohol's 1,220 schools make available education to its citizenry. Completing these institutions of learning are five government skills training centers as well as a number of private establishments offering vocational/technical courses. Among Bohol's household population aged 5 years and over, more than half (51.59 percent) had attended or finished elementary education. Females dominated higher levels of education in Bohol has a literacy rate 92% (NSO-FLEMMS,2003).
In Bohol, health care is provided by over 30 hospitals and clinics as well as through a network outpatient clinics, rural health centers and school health services including dental care facilities and a network of blood banks. Government health facilities include 48 Rural Health Units (RHUs) and 12 hospitals. Majority of patients in the province troop to government hospitals which offer free services with most patients spending only for supplies and medicines during their hospitalization.
The average life expectancy of a Boholano for both males and females increased from 69.81 years in 1999 to a projected 73 years in 2002 with the females having a higher life expectancy than males.


text:
Radar guns and detectors have evolved alternately over time to counter each other's technology in a form of civilian electronic "warfare". For example, as new frequencies have been introduced, radar detectors have initially been "blind" to them until their technology, too, has been updated. Similarly, the length of time and strength of the transmissions have been lowered to reduce the chance of detection, which in turn has resulted in more sensitive receivers and more sophisticated software counter technology. Lastly, radar detectors may combine other technologies, such as GPS-based technology with a point of interest database of known speed trapping locations, into a single device to improve their chances of success.radar detectors has a local oscillator that radiates slightly, so it is possible to build a radar-detector detector, which detects such emissions (usually the frequency of the radar type being detected, plus about 10 MHz). The VG-2 Interceptor was the first device developed for this purpose, but has since been eclipsed by the Spectre III.[1] This form of "electronic warfare" cuts both ways - since detector-detectors use a similar superheterodyne receiver, many early "stealth" radar detectors were equipped with a radar-detector-detector-detector circuit, which shuts down the main radar receiver when the detector-detector's signal is sensed, thus preventing detection by such equipment, laser light, commonly referred to as LIDAR, rather than radio waves. Radar detectors, which detect radio transmissions, are therefore unable to detect the infrared light emitted by LIDAR guns so a different type of device called a LIDAR detector is required. LIDAR detection, however, is not nearly as effective as radar detection because the output beam is very focused. While radar's radio waves can expand to 85 feet (26 m) across at 1,000 feet (300 m) from their source, LIDAR's light beam diffuses to only about 6 feet (1.8 m).[citation needed] Also, a police officer targeting a car will most likely aim for the centre mass or headlight of the vehicle and, because radar detectors are mounted on the windshield away from the beam's aim, they may not alert at all. Lastly, with such a focused beam, an officer using a LIDAR gun can target a single car in close proximity to others at ranges of up to 3,000 feet (910 m).[citation needed] This has resulted in some manufacturers producing LIDAR jammers. Unlike radar, LIDAR's frequencies and use are not controlled by the FCC. These jammers attempt to confuse police LIDAR into showing no speed on the display. Many times they are successful,
Laser Radar Detector with GPS
Uniden LRD950Super Long Range Laser Radar DetectionGPS Built-InRed Light Camera AlertSpeed Camera AlertAdvanced K Band FilterVoice NotificationsMute MemoryQuiet RideEast to Read OLED DisplayFrequency of Radar BandCurrent SpeedBattery...


Longsleeve Top with Colorblocked Sleeves.
With a higher crew-neck, and raglan-style sleeves, this top is made for sports like golf and tennis. The lightweight, sun-protective fabric also makes it great for boating and fishing!
Machine or Hand Wash Cold. Hang/Lay Flat to dry.
Men's Three-Quarter-Zip Pullover Jacket in Midweight heathered fabric.
Reflective Scout Sports Logo on center back neck.
A classic layering piece you'll have to add to your closet once you feel the luxuriously soft fabric.
Machine or Hand Wash Cold. Lay/Hang Flat to Dry.
Model is 6'2" wearing size L.
Men's Shortsleeve Polo with Stripe-print Collar.
Reflective Scout Sports logo on right sleeve hem.
Your classic striped polo gets an update with this pinstripe print solely featured on the collar.


Although Brazil may not have done as well in the Fifa World Cup as some anticipated, the country's economy is still a big favourite among investors, says F&C Investments banking.
Mike Woodward, head of investment trusts at the firm, explained that - as an emerging market - the South American nation's stockmarket is likely to be volatile, but it has plenty of potential.
He said: "Our investors are still keen on the emerging markets story and are not alone in tipping the Brazilian stockmarket for great things over the next few years."
In his view, it may not be advisable to concentrate on specialist Brazil investments, as four years is a fairly short time in terms of equities.
Instead, Mr Woodward recommended adopting a more globally diversified portfolio.
The Brazilian football team were knocked out of the World Cup by Holland in the quarter-finals 2-1, despite taking the lead initially.
Holland face Spain in the final on Sunday (July 11th).


text:
Die Liste der denkmalgeschützten Objekte in Kravany nad Dunajom enthält die zwei nach slowakischen Denkmalschutzvorschriften geschützten Objekte in der Gemeinde Kravany nad Dunajom im Okres Komárno, Slowakei.

Denkmäler

Legende

Einzelnachweise

!
Kravany nad Dunajom

redpajama_set_name: RedPajamaWikipedia


text:
Shropshire Council elections: Tories vow to spend £21 million on the roads
News | Published: Apr 1, 2017
Conservatives say they are committed to investing £21 million in the county's road network if they are elected to run Shropshire Council this May.
Unveiling their manifesto for the forthcoming local government elections the Conservative group said the council would also supply cheaper energy to local residents – cutting costs up to five to 10 per cent below energy company SSE's standard tariff.
Malcolm Pate, the Conservative group leader, said the group is looking to make the council a more commercial organisation.
Speaking at the launch of the manifesto he said: "Like much of the public sector, Shropshire Council still has significant funding challenges.
"The Conservatives are, however, taking the council down a better commercial path, which will ensure we can continue to deliver services to the young, the elderly and the vulnerable people of Shropshire.
"We have developed both a new industrial grown strategy and a digital strategy. This will transform the council's performance now and provide long-term financial security.
"In order to ensure delivery of these manifesto promises we have developed a sound corporate plan which clearly shows our objectives and how we will measure our progress in fulfilling them."
The group said they have also secured £11.7 million in funding to continue the roll-out of faster fibre broadband in the county, and are looking to embrace new technology to make maintaining roads as quick and effective as possible.
In 2017/18 they hope to spend a further £1.3 million on repairing potholes.
A jetpatcher and two road masters, set to cost £500,000, will be used in repairing rural roads starting in April.
A new mobile phone app is also in development which will give residents in Shropshire the chance to send information about potholes, fly-tipping, faulty street lamps and other issues directly to the street team.
The manifesto states: "By managing our highway asset and delivering cost effective road improvement, we have been awarded Government Incentive Funding.
"Having now achieved the highest level 3, we are attracting the maximum incentive funding for Shropshire, which allows us to put £1.2 million into fixing local roads."
The Tory manifesto was based around 10 points, including vowing to lobby government to offer a fairer share of funding when it came to health care.
As well as highways, internet connectivity and energy, they have launched "Shropshire Choices", a website to help people make a choice in regards to care homes, and vowed to continue to support young people most at risk.
News Transport Politics


Next up Jenny Jones of Great Britain.
Finishes a clean run, Jenny Jones, but the judges will take a close look at the rails.
...the 33-year-old scores 87.25. Jones in first place!
She's elated. Thought that wasn't a good run. But, it's the best so far.


Fast Times Skateboarding has been supplying our customers with the finest selection of Skateboard, Apparel & Footwear in Australia since 2008. We stock the biggest range of skate, hardware, streetwear and footwear as well as Fast Times exclusives, brand collabs & limited edition releases. Fast Times Skateboarding has been supplying our customers with the finest selection of Skateboard, Apparel & Footwear in Australia since 2008. We stock the biggest range of skate, hardware, streetwear and footwear as well as Fast Times exclusives, brand collabs & limited edition releases.


text:
© 2018 Rhonda Lee Carver
Join my Street Team
MEET RHONDA
Contemporary Romance Author Rhondaleecarver
I'm so glad you stopped by. What are you looking for? A sexy cowboy? Tough lawman? Something that keeps you on the edge of your seat and in love? Scroll my books and find your romance. Your love story is ONLY a click away!
Rhonda Lee Carver
How about a FREE book on me? Follow me on Facebook Twitter and BookBub. Subscribe to my newsletter. Sign up to get your FREE book here. Write "Book" in message field.
Tormented by the Lawman. Coming in February! Preorder now!
Paths collide for a feisty, independent Congressman's niece and an enigmatic, alpha-hero. The unexpected chemistry puts a new spin on the line between love and hate.
If anyone accused Hazel Levine of being difficult and indecisive they wouldn't be wrong. Hopping from one job to another and swimming against the expectations of her family has made her a menace to her uncle's campaign for president. When she attempts to steer a straight line for her his sake, she finds herself ensnared in a murder mystery that soon has her in the crosshairs of a killer.
They come from different worlds. She's a rich girl born with a silver spoon and Special Agent Cox Landon loves to get his hands dirty. With an awarded military career and years of catching criminals under his belt, he's learned patience, but his steel wall isn't thick enough to deal with the likes of the spoiled Congressman's niece. Yet, to complete the deal Cox has forged, he'll have to tolerate Hazel--at least until he gets what he wants.
Stranded in a snowstorm in a cabin on secluded Pitchfork mountain, with killers on their tail, their survival will be tested, as well as their mutual desire to test the thin line between love and hate.
Cox is willing to do almost anything to taste her amazing lips and touch her wild, red curls. And for Hazel, she's willing to show him a softer side--show him the real Hazel--hoping he won't pressure her like her family has done for years. Can he accept her for who she is? Strong-willed. Flawed. And completely indecisive.
Can they dodge the killers and make it out alive? Or is the true danger in what lies in their hearts?
Cowboy State of Mind: Free in KU
When life is on the line you learn to live…and never miss a dance.
Dr. CC Peladora never expected to ask a stranger to father her child, but neither did she think she'd be diagnosed with a chronic disease. With a failed marriage and a busy career, love is the last thing on her priority list, but she wants more than anything to experience motherhood before it's too late. When she hears through the grapevine that Adam Colt, the sexy, mysterious son of Buzz Colt, must marry to meet the stipulations of his father's will, she sees an opportunity that won't come along again. And living like there's no tomorrow, this is her time to get what she wants…
Adam just happened to be sitting on a bar stool in a local honky tonk when the spunky woman he's been fantasizing about for months walks in and turns his world upside down. She offers to give him what he needs, an arranged marriage, for something she wants desperately. A child. No doubt he wants to kiss her, but marry her? And give her a child and then disappear from Tarnation forever? His father's stained blood might run through Adam's veins, but a real cowboy would never leave his kin. But in his journey to prove that he's nothing like Buzz, he finds more than he ever bargained for…and a forgiveness he never expected would come.
A secret marriage. Vulnerable hearts. Off-the-charts chemistry and small-town romance will have CC and Adam proving that a spirited vet and an alpha cowboy can find love in the most unlikely places.
THE COLT COWBOYS
Five brothers who are strangers are given the choice to live together for six months and work the Grinning Spurs Ranch or give up everything they've worked for—including their homesteads.
However, that's only the beginning. All five must be married before time is up.
Bighearted Cowboy:
Brennan Colt sings happy birthday as a favor and ends up with a wife. Now that's a story of a Colt brother!
Cancer might have taken Charlotte's breasts, but the disease can't take her feisty spirit.
Tattoo artist Charlotte Sophia wants to help other survivors, and those who have been wounded in war, by turning scars into tattoos. She sees an opportunity. Marry Brennan Colt. The fair trade is, he'll hand over the deed to a building in Tarnation where she can set up shop, AND a mere one hundred thousand toward the charity that means the world to Charlotte.
Brennan's a big hearted cowboy so helping cancer survivors and wounded vets is right up his alley, but the marriage? Well, he's not so sure about the whole idea. Yet, with only a few weeks remaining before he loses everything if he isn't hitched, Brennan is backed against the wall. So what choice does he have? He takes the matrimonial plunge.
He's learning about his stranger wife, like how smart she is, how she loves chocolate and the way her eyes light up when she laughs, but he's also realizing she has a few secrets. Nothing says he has to get to know her in the next few months, but the big hearted cowboy is curious...and maybe a tad in love.


He omitted two instances of the word "of"before immersing his daughter. It was such a subtle thing, I had to review what I'd heard in my mind to make sure he hadn't simply said " o' " or another abbreviated vowel-form of the word. The other witness agreed with me that he had not said the words, and we had to request that the man say the prayer again. Though we attempted to clarify the correct wording beforehand, he made the same mistake again before again baptizing his shivering daughter. Finally, we had to get a copy of the scriptures opened to D&C section 20 to show him the word he was leaving out. On the third try, he said the prayer correctly and rebaptized his now crying daughter.
Though this little girl had been baptized not only once, but twice, and the blessing pronounced on her was identical in meaning in all material respects, we still had to request that she be rebaptized. We understood the principle that we should not deviate from the ordinances God has commanded us to perform. When a word is omitted, added, or changed, the words and immersion must be repeated. This is true of both the baptismal blessing and sacramental prayers.
While I believe that if we had not known that there had been an error despite our best efforts to ensure it was done correctly, that God would honor the baptism as a righteous act. But we did know, and thus had an obligation to correct the error.
What if we had immediately discovered that the copy of the D&C where we found the baptismal blessing had been misprinted and no longer contained the words appointed by God? Surely we would have located the correct words asked that she be rebaptized once again.
What if all copies of the D&C printed since 1835 contained an altered invocation? Would our obligation to locate the correct prayer be any less, or the desire to be rebaptized with the correct prayer any less appropriate?
If ensuring the correct words is truly important, then we should carefully review the relevant historical records. If we are doing what God commanded, then we will be validated. If we are not, we should be grateful for the correction.
3 Wherefore you know that they are true; and if you know that they are true, behold I give unto you a commandment, that you rely upon the things which are written; for in them are all things written, concerning my church, my gospel, and my rock.
4 Wherefore if you shall build up my church, and my gospel, and my rock, the gates of hell shall not prevail against you.
As surely as God's spirit had manifested to Oliver that the Book of Mormon was true, it could be relied upon as a guide for the building up of the latter-day Church. In fact, that was God's purpose in preserving it as he did, ensuring that "in them are all things written, concerning my church, my gospel, and my rock." No necessary principle was omitted, and building on the principles in the text would ensure that "the gates of hell shall not prevail against" those attempting to build the Church.
23 Verily I say unto you, that whoso repenteth of his sins through your words, and desireth to be baptized in my name, on this wise shall yebaptize them—Behold, ye shall go down and stand in the water, and in my name shall ye baptize them.
As of the writing of that document, the words of the baptismal blessing still comply with the Lord's instructions in 1829.
Rather than point the reader to the Book of Mormon, this edition extracts the language for the baptismal blessing and sacrament prayers from 3 Nephi 11 and Moroni and includes them in the Articles and Covenants verbatim.
January 1835: Republication in the Evening and the Morning Star.
It is significant that Oliver Cowdery, not Joseph Smith, took responsibility for explaining the changes made. The committee for compiling and printing the Doctrine and Covenants had been commissioned the previous September. Between September and January, Joseph's journal mentions several times his labor in preparing to instruct the School of the Prophets that winter, as well as many other items which made for a busy winter. He never mentions any time spent reviewing and revising the revelations, but repeatedly mentioned his preoccupation with the School of the Prophets and the Lectures on Faith, claiming they precluded his involvement in other affairs. Since major or minor revisions occur across the breadth of the body of revelations, a comprehensive review must have been undertaken by some members of the committee. We cannot say with certainty that Joseph was not involved, but the preponderance of the evidence indicates that the Lectures, rather than the revelations, were his focus.
The text altered by Oliver Cowdery earlier in the year was perpetuated here and in all future editions of the scriptures, and in official Church policy. It is now the wording used throughout the Church. It is not clear whether the altered wording was immediately adopted, or whether the Church continued for a time in following the Lord's instruction to rely on the Book of Mormon.
Those are the foundation whereon if men build they cannot fall. They are the first law. They are like walking practice for a toddler. They empower us to obtain all other direction and higher laws which we might need (Moses 8:24). We can do them perfectly, with the correct words and in the correct order. When an opportunity to obey God perfectly arises, we should jump at the chance. Since we have God's commandment to rely on the instructions in the Book of Mormon (D&C 18:1-4), and the instructions in the original Articles of the Church accepted by covenant on April 6, 1830 to follow the manner of baptism in the Book of Mormon, we have authority to act; for commandments from God always bestow the authorization to fulfill them (D&C 1:5-6).
When we say the wrong words in a baptism, we request that the ordinance be performed again. It is standard procedure. Not to please men, but because God has given specific instructions. We need no further permission. We need no audience. We simply need a priest with authority willing to baptize according to Christ's instructions, a repentant soul, and a body of water.