You will be shown sections from a scientific research paper, together with a question about the paper. Paragraphs in the paper are separated by newlines. Your task is to answer the question based on the contents of the paper.

Paper:
----------------------------------------
Natural Language Generation for Spoken Dialogue System using RNN Encoder-Decoder Networks

Natural language generation (NLG) is a critical component in a spoken dialogue system. This paper presents a Recurrent Neural Network based Encoder-Decoder architecture, in which an LSTM-based decoder is introduced to select, aggregate semantic elements produced by an attention mechanism over the input elements, and to produce the required utterances. The proposed generator can be jointly trained both sentence planning and surface realization to produce natural language sentences. The proposed model was extensively evaluated on four different NLG datasets. The experimental results showed that the proposed generators not only consistently outperform the previous methods across all the NLG domains but also show an ability to generalize from a new, unseen domain and learn from multi-domain datasets.

Introduction
Natural Language Generation (NLG) plays a critical role in Spoken Dialogue Systems (SDS) with task is to convert a meaning representation produced by the Dialogue Manager into natural language utterances. Conventional approaches still rely on comprehensive hand-tuning templates and rules requiring expert knowledge of linguistic representation, including rule-based BIBREF0 , corpus-based n-gram models BIBREF1 , and a trainable generator BIBREF2 .
Recently, Recurrent Neural Networks (RNNs) based approaches have shown promising performance in tackling the NLG problems. The RNN-based models have been applied for NLG as a joint training model BIBREF3 , BIBREF4 and an end-to-end training model BIBREF5 . A recurring problem in such systems is requiring annotated datasets for particular dialogue acts (DAs). To ensure that the generated utterance representing the intended meaning of the given DA, the previous RNN-based models were further conditioned on a 1-hot vector representation of the DA. BIBREF3 introduced a heuristic gate to ensure that all the slot-value pair was accurately captured during generation. BIBREF4 subsequently proposed a Semantically Conditioned Long Short-term Memory generator (SC-LSTM) which jointly learned the DA gating signal and language model.
More recently, Encoder-Decoder networks BIBREF6 , BIBREF7 , especially the attentional based models BIBREF8 , BIBREF9 have been explored to solve the NLG tasks. The Attentional RNN Encoder-Decoder BIBREF10 (ARED) based approaches have also shown improved performance on a variety of tasks, e.g., image captioning BIBREF11 , BIBREF12 , text summarization BIBREF13 , BIBREF14 .
While the RNN-based generators with DA gating-vector can prevent the undesirable semantic repetitions, the ARED-based generators show signs of better adapting to a new domain. However, none of the models show significant advantage from out-of-domain data. To better analyze model generalization to an unseen, new domain as well as model leveraging the out-of-domain sources, we propose a new architecture which is an extension of the ARED model. In order to better select, aggregate and control the semantic information, a Refinement Adjustment LSTM-based component (RALSTM) is introduced to the decoder side. The proposed model can learn from unaligned data by jointly training the sentence planning and surface realization to produce natural language sentences. We conducted experiments on four different NLG domains and found that the proposed methods significantly outperformed the state-of-the-art methods regarding BLEU BIBREF15 and slot error rate ERR scores BIBREF4 . The results also showed that our generators could scale to new domains by leveraging the out-of-domain data. To sum up, we make three key contributions in this paper:
We review related works in Section "Related Work" . Following a detail of proposed model in Section "Recurrent Neural Language Generator" , Section "Experiments" describes datasets, experimental setups, and evaluation metrics. The resulting analysis is presented in Section "Results and Analysis" . We conclude with a brief summary and future work in Section "Conclusion and Future Work" .

RALSTM Decoder
The proposed semantic RALSTM cell applied for Decoder side consists of three components: a Refinement cell, a traditional LSTM cell, and an Adjustment cell:
Firstly, instead of feeding the original input token $\textbf {w}_{t}$ into the RNN cell, the input is recomputed by using a semantic gate as follows: 
$$\begin{aligned}
\textbf {r}_{t}&=\sigma (\textbf {W}_{rd}\textbf {d}_{t} + \textbf {W}_{rh}\textbf {h}_{t-1})\\
\textbf {x}_{t}&=\textbf {r}_{t} \odot \textbf {w}_{t}
\end{aligned}$$   (Eq. 19) 
where $\textbf {W}_{rd}$ and $\textbf {W}_{rh}$ are weight matrices. Element-wise multiplication $\odot $ plays a part in word-level matching which not only learns the vector similarity, but also preserves information about the two vectors. $\textbf {W}_{rh}$ acts like a key phrase detector that learns to capture the pattern of generation tokens or the relationship between multiple tokens. In other words, the new input $\textbf {x}_{t}$ consists of information of the original input token $\textbf {w}_{t}$ , the DA representation $\textbf {d}_{t}$ , and the hidden context $\textbf {h}_{t-1}$ . $\textbf {r}_{t}$ is called a Refinement gate because the input tokens are refined by a combination gating information of the attentive DA representation $\textbf {d}_{t}$ and the previous hidden state $\textbf {W}_{rh}$0 . By this way, we can represent the whole sentence based on the refined inputs.
Secondly, the traditional LSTM network proposed by BIBREF26 bahdanau2014neural in which the input gate $\textbf {i}_{i}$ , forget gate $\textbf {f}_{t}$ and output gates $\textbf {o}_{t}$ are introduced to control information flow and computed as follows: 
$$\begin{aligned}
\begin{pmatrix}
\textbf {i}_{t}
\\ \textbf {f}_{t}
\\ \textbf {o}_{t}
\\ \hat{\textbf {c}}_{t}
\end{pmatrix}
&=
\begin{pmatrix}\sigma \\ \sigma \\ \sigma \\ \tanh \end{pmatrix}\textbf {W}_{4n,4n}
\begin{pmatrix}
\textbf {x}_{t}
\\ \textbf {d}_{t}
\\ \textbf {h}_{t-1}
\end{pmatrix}\\
\end{aligned}$$   (Eq. 20) 
where $n$ is hidden layer size, $\textbf {W}_{4n,4n}$ is model parameters. The cell memory value $\textbf {c}_{t}$ is modified to depend on the DA representation as: 
$$\begin{aligned}
\textbf {c}_{t}&=\textbf {f}_{t}\odot \textbf {c}_{t-1} +\textbf {i}_{t}\odot \hat{\textbf {c}}_{t} + \tanh (\textbf {W}_{cr}\textbf {r}_{t})
\\ \tilde{\textbf {h}}_{t}&= \textbf {o}_{t} \odot \tanh (\textbf {c}_{t})
\end{aligned}$$   (Eq. 21) 
where $\tilde{\textbf {h}}_{t}$ is the output.
Thirdly, inspired by work of BIBREF4 in which the generator was further conditioned on a 1-hot representation vector $\textbf {s}$ of given dialogue act, and work of BIBREF27 that proposed a visual sentinel gate to make a decision on whether the model should attend to the image or to the sentinel gate, an additional gating cell is introduced on top of the traditional LSTM to gate another controlling vector $\textbf {s}$ . Figure 6 shows how RALSTM controls the DA vector $\textbf {s}$ . First, starting from the 1-hot vector of the DA $\textbf {s}_{0}$ , at each time step $t$ the proposed cell computes how much the LSTM output $\tilde{\textbf {h}}_{t}$ affects the DA vector, which is computed as follows: 
$$\begin{aligned}
\textbf {a}_{t}&=\sigma (\textbf {W}_{ax}\textbf {x}_{t} +\textbf {W}_{ah}\tilde{\textbf {h}}_{t})\\
\textbf {s}_{t}&=\textbf {s}_{t-1} \odot \textbf {a}_{t}
\end{aligned}$$   (Eq. 22) 
where $\textbf {W}_{ax}$ , $\textbf {W}_{ah}$ are weight matrices to be learned. $\textbf {a}_{t}$ is called an $Adjustment$ gate since its task is to control what information of the given DA have been generated and what information should be retained for future time steps. Second, we consider how much the information preserved in the DA $\textbf {s}_{t}$ can be contributed to the output, in which an additional output is computed by applying the output gate $\textbf {o}_{t}$ on the remaining information in $\textbf {s}_{t}$ as follows: 
$$\begin{aligned}
\textbf {c}_{a}&=\sigma (\textbf {W}_{os}\textbf {s}_{t})\\
\tilde{\textbf {h}}_{a}&= \textbf {o}_{t} \odot \tanh (\textbf {c}_{a})
\end{aligned}$$   (Eq. 23) 
where $\textbf {W}_{os}$ is a weight matrix to project the DA presentation into the output space, $\tilde{\textbf {h}}_{a}$ is the Adjustment cell output. Final RALSTM output is a combination of both outputs of the traditional LSTM cell and the Adjustment cell, and computed as follows: 
$$\textbf {h}_{t}=\tilde{\textbf {h}}_{t} + \tilde{\textbf {h}}_{a}$$   (Eq. 24) 
Finally, the output distribution is computed by applying a softmax function $g$ , and the distribution can be sampled to obtain the next token, 
$$\begin{aligned}
& P(w_{t+1}\mid w_{t},...w_{0},\textbf {DA})=g(\textbf {W}_{ho}\textbf {h}_{t}) \\
& w_{t+1} \sim P(w_{t+1}\mid w_{t}, w_{t-1},...w_{0},\textbf {DA})
\end{aligned}$$   (Eq. 25) 
where $\textbf {DA}=(\textbf {s}, \textbf {z})$ .

Decoding
The decoding consists of two phases: (i) over-generation, and (ii) reranking. In the over-generation, the generator conditioned on both representations of the given DA use a beam search to generate a set of candidate responses. In the reranking phase, cost of the generator is computed to form the reranking score $\textbf {R}$ as follows: 
$$\textbf {R} = \textbf {F}(\theta ) + \lambda \textbf {ERR}$$   (Eq. 29) 
where $\lambda $ is a trade off constant and is set to a large value in order to severely penalize nonsensical outputs. The slot error rate $\textbf {ERR}$ , which is the number of slots generated that is either missing or redundant, and is computed by: 
$$\textbf {ERR} = \frac{\textbf {p} + \textbf {q}}{\textbf {N}}$$   (Eq. 30) 
where $\textbf {N}$ is the total number of slots in DA, and $\textbf {p}$ , $\textbf {q}$ is the number of missing and redundant slots, respectively.

Experiments
We extensively conducted a set of experiments to assess the effectiveness of the proposed models by using several metrics, datasets, and model architectures, in order to compare to prior methods.

Datasets
We assessed the proposed models on four different NLG domains: finding a restaurant, finding a hotel, buying a laptop, and buying a television. The Restaurant and Hotel were collected in BIBREF4 , while the Laptop and TV datasets have been released by BIBREF22 with a much larger input space but only one training example for each DA so that the system must learn partial realization of concepts and be able to recombine and apply them to unseen DAs. This makes the NLG tasks for the Laptop and TV domains become much harder. The dataset statistics are shown in Table 1 .

Evaluation Metrics and Baselines
The generator performance was assessed on the two evaluation metrics: the BLEU and the slot error rate ERR by adopting code from an open source benchmark toolkit for Natural Language Generation. We compared the proposed models against three strong baselines which have been recently published as state-of-the-art NLG benchmarks[]. https://github.com/shawnwun/RNNLG
HLSTM proposed by BIBREF3 thwsjy15 which used a heuristic gate to ensure that all of the slot-value information was accurately captured when generating.
SCLSTM proposed by BIBREF4 wensclstm15 which can jointly learn the gating signal and language model.
Enc-Dec proposed by BIBREF8 wentoward which applied the attention-based encoder-decoder architecture.

Conclusion and Future Work
We present an extension of ARED model, in which an RALSTM component is introduced to select and aggregate semantic elements produced by the Encoder, and to generate the required sentence. We assessed the proposed models on four NLG domains and compared to the state-of-the-art generators. The proposed models empirically show consistent improvement over the previous methods in both the BLEU and ERR evaluation metrics. The proposed models also show an ability to extend to a new, unseen domain no matter how much the in-domain training data was fed. In the future, it would be interesting to apply the proposed model to other tasks that can be modeled based on the encoder-decoder architecture, i.e., image captioning, reading comprehension, and machine translation. 

Figure 2: The RALSTM cell proposed in this paper, which consists of three components: an Refinement Cell, a traditional LSTM Cell, and an Adjustment Cell. At time step t, while the Refinement cell computes new input tokens xt based on the original input tokens and the attentional DA representation dt, the Adjustment Cell calculates how much information of the slot-value pairs can be generated by the LSTM Cell.

Table 4: Comparison of top responses generated for some input dialogue acts between different models. Errors are marked in color (missing, misplaced information). All2* are general models.

Figure 6: Example showing how RALSTM drives down the DA feature value vector s step-by-step, in which the model generally shows its ability to detect words and phases describing a corresponding slot-value pair.
----------------------------------------

Question: What is the difference of the proposed model with a standard RNN encoder-decoder?

To answer the question, format your response as a `json` object with two fields:

"answer": A string providing a succinct answer to the question, in your own words.
"evidence": An array of strings. Each entry should be a full paragraph from the paper. Together, the evidence should serve as a justification for the answer.

For instance, for the question "What baselines did the authors compare against?", a sample response might be:

{
  "answer": "BERT and RoBERTa."
  "evidence": ["We compare our approach against two baselines. In Table 1, we compare against BERT. In Table 2, we compare against RoBERTa. Our findings indicate that our approach improves over both baeslines..."]
}

The "answer" field should be roughly 71 characters in length.

Do not include any text in your response other than the json. If the question is unanswerable given the provided excerpts, respond with the single word "null".

To repeat, the question is: What is the difference of the proposed model with a standard RNN encoder-decoder?

Answer JSON object:
{"answer": "Introduce a \"Refinement Adjustment LSTM-based component\" to the decoder", "evidence": ["While the RNN-based generators with DA gating-vector can prevent the undesirable semantic repetitions, the ARED-based generators show signs of better adapting to a new domain. However, none of the models show significant advantage from out-of-domain data. To better analyze model generalization to an unseen, new domain as well as model leveraging the out-of-domain sources, we propose a new architecture which is an extension of the ARED model. In order to better select, aggregate and control the semantic information, a Refinement Adjustment LSTM-based component (RALSTM) is introduced to the decoder side. The proposed model can learn from unaligned data by jointly training the sentence planning and surface realization to produce natural language sentences. We conducted experiments on four different NLG domains and found that the proposed methods significantly outperformed the state-of-the-art methods regarding BLEU BIBREF15 and slot error rate ERR scores BIBREF4 . The results also showed that our generators could scale to new domains by leveraging the out-of-domain data. To sum up, we make three key contributions in this paper:"]}