You will be shown sections from a scientific research paper, together with a question about the paper. Your task is to answer the question based on the contents of the paper. Your response should be a `json` object with two fields:

"answer": An array of strings extracted directly from the paper which, collectively, answer the question.
"evidence": An array of strings. Each should be an excerpt from the paper, in which one or more of the extracted answers can be found.

For instance, for the question "What baselines did the authors compare against?", a sample response might be:

{
  "answer": ["BERT", "RoBERTa"],
  "evidence": ["In our experiments, we compare the performance of our model against BERT and RoBERTa."]
}

Do not include any text in your response other than the json.

If the question is unanswerable given the provided excerpts, respond with the single word "null".

Paper: Towards Understanding Neural Machine Translation with Word Importance

Although neural machine translation (NMT) has advanced the state-of-the-art on various language pairs, the interpretability of NMT remains unsatisfactory. In this work, we propose to address this gap by focusing on understanding the input-output behavior of NMT models. Specifically, we measure the word importance by attributing the NMT output to every input word through a gradient-based method. We validate the approach on a couple of perturbation operations, language pairs, and model architectures, demonstrating its superiority on identifying input words with higher influence on translation performance. Encouragingly, the calculated importance can serve as indicators of input words that are under-translated by NMT models. Furthermore, our analysis reveals that words of certain syntactic categories have higher importance while the categories vary across language pairs, which can inspire better design principles of NMT architectures for multi-lingual translation.

Introduction ::: Contributions
Our main contributions are:
Our study demonstrates the necessity and effectiveness of exploiting the intermediate gradients for estimating word importance.
We find that word importance is useful for understanding NMT by identifying under-translated words.
We provide empirical support for the design principle of NMT architectures: essential inductive bias (e.g., language characteristics) should be considered for model design.

Related Work ::: Interpreting Seq2Seq Models
Interpretability of Seq2Seq models has recently been explored mainly from two perspectives: interpreting internal representations and understanding input-output behaviors. Most of the existing work focus on the former thread, which analyzes the linguistic information embeded in the learned representations BIBREF3, BIBREF4, BIBREF10 or the hidden units BIBREF6, BIBREF5. Several researchers turn to expose systematic differences between human and NMT translations BIBREF11, BIBREF12, indicating the linguistic properties worthy of investigating. However, the learned representations may depend on the model implementation, which potentially limit the applicability of these methods to a broader range of model architectures. Accordingly, we focus on understanding the input-output behaviors, and validate on different architectures to demonstrate the universality of our findings.
Concerning interpreting the input-output behavior, previous work generally treats Seq2Seq models as black-boxes BIBREF13, BIBREF7. For example, alvarez2017causal measure the relevance between two input-output tokens by perturbing the input sequence. However, they do not exploit any intermediate information such as gradients, and the relevance score only resembles attention scores. Recently, Jain2019AttentionIN show that attention scores are in weak correlation with the feature importance. Starting from this observation, we exploit the intermediate gradients to better estimate word importance, which consistently outperforms its attention counterpart across model architectures and language pairs.

Related Work ::: Exploiting Gradients for Model Interpretation
The intermediate gradients have proven to be useful in interpreting deep learning models, such as NLP models BIBREF14, BIBREF15 and computer vision models BIBREF16, BIBREF9. Among all gradient-based approaches, the integrated gradients BIBREF9 is appealing since it does not need any instrumentation of the architecture and can be computed easily by calling gradient operations. In this work, we employ the IG method to interpret NMT models and reveal several interesting findings, which can potentially help debug NMT models and design better architectures for specific language pairs.

Approach ::: Neural Machine Translation
In machine translation task, a NMT model $F$: $\textbf {x} \rightarrow \textbf {y}$ maximizes the probability of a target sequence $\textbf {y} = \lbrace y_1,...,y_N\rbrace $ given a source sentence $\textbf {x} = \lbrace x_1,...,x_M\rbrace $:
where $\mathbf {\theta }$ is the model parameter and $\textbf {y}_{<n}$ is a partial translation. At each time step n, the model generates an output word of the highest probability based on the source sentence $\textbf {x}$ and the partial translation $\textbf {y}_{<n}$. The training objective is to minimize the negative log-likelihood loss on the training corpus. During the inference, beam search is employed to decode a more optimal translation. In this study, we investigate the contribution of each input word $x_m$ to the translated sentence ${\bf y}$.

Approach ::: Integrated Gradients
In this work, we resort to a gradient-based method, integrated gradients BIBREF9 (IG), which was originally proposed to attribute the model predictions to input features. It exploits the handy model gradient information by integrating first-order derivatives. IG is implementation invariant and does not require neural models to be differentiable or smooth, thereby is suitable for complex neural networks like Transformer. In this work, we use IG to estimate the word importance in an input sentence precisely.
Formally, let $\textbf {x} = (x_1, ..., x_M)$ be the input sentence and $\textbf {x}^{\prime }$ be a baseline input. $F$ is a well-trained NMT model, and $F(\textbf {x})_n$ is the model output (i.e., $P(y_n|\textbf {y}_{<n},\textbf {x})$) at time step $n$. Integrated gradients is then defined as the integral of gradients along the straightline path from the baseline $\textbf {x}^{\prime }$ to the input $\textbf {x}$. In detail, the contribution of the $m^{th}$ word in $\textbf {x}$ to the prediction of $F(\textbf {x})_n$ is defined as follows.
where $\frac{\partial {F(\textbf {x})_n}}{\partial {\textbf {x}_m}}$ is the gradient of $F(\textbf {x})_n$ w.r.t. the embedding of the $m^{th}$ word. In this paper, as suggested, the baseline input $\textbf {x}^{\prime }$ is set as a sequence of zero embeddings that has the same sequence length $M$. In this way, we can compute the contribution of a specific input word to a designated output word. Since the above formula is intractable for deep neural models, we approximate it by summing the gradients along a multi-step path from baseline $\textbf {x}^{\prime }$ to the input x.
where $S$ denotes the number of steps that are uniformly distributed along the path. The IG will be more accurate if a larger S is used. In our preliminary experiments, we varied the steps and found 300 steps yielding fairly good performance.
Following the formula, we can calculate the contribution of every input word makes to every output word, forming a contribution matrix of size $M \times N$, where $N$ is the output sentence length. Given the contribution matrix, we can obtain the word importance of each input word to the entire output sentence. To this end, for each input word, we first aggregate its contribution values to all output words by the sum operation, and then normalize all sums through the Softmax function. Figure FIGREF13 illustrates an example of the calculated word importance and the contribution matrix, where an English sentence is translated into a French sentence using the Transformer model. A negative contribution value indicates that the input word has negative effects on the output word.

Experiment ::: Data
To make the conclusion convincing, we first choose two large-scale datasets that are publicly available, i.e., Chinese-English and English-French. Since English, French, and Chinese all belong to the subject-verb-object (SVO) family, we choose another very different subject-object-verb (SOV) language, Japanese, which might bring some interesting linguistic behaviors in English-Japanese translation.
For Chinese-English task, we use WMT17 Chinese-English dataset that consists of $20.6$M sentence pairs. For English-French task, we use WMT14 English-French dataset that comprises $35.5$M sentence pairs. For English-Japanese task, we follow BIBREF17 to use the first two sections of WAT17 English-Japanese dataset that consists of $1.9$M sentence pairs. Following the standard NMT procedure, we adopt the standard byte pair encoding (BPE) BIBREF18 with 32K merge operations for all language pairs. We believe that these datasets are large enough to confirm the rationality and validity of our experimental analyses.

Experiment ::: Results on Different Perturbations ::: The gradient-based method is superior to comparative methods (e.g., Attention) in estimating word importance.
Figure FIGREF19 shows that two black-box methods (i.e., Content, Frequency) perform only slightly better than the Random method. Specifically, the Frequency method demonstrates even worse performances under the Mask perturbation. Therefore, linguistic properties (such as POS tags) and the word frequency can only partially help identify the important words, but it is not as accurate as we thought. In the meanwhile, it is intriguing to explore what exact linguistic characteristics these important words reveal, which will be introduced in Section SECREF5.
We also evaluate the Attention method, which bases on the encoder-decoder attention scores at the last layer of Transformer. Note that the Attention method is also used to simulate the best black-box method SOCRAT, and the results show that it is more effective than black-box methods and the Random baseline. Given the powerful Attention method, Attribution method still achieves best performances under all three perturbations. Furthermore, we find that the gap between Attribution and Attention is notably large (around $1.0+$ BLEU difference). Attention method does not provide as accurate word importance as the Attribution, which exhibits the superiority of gradient-based methods and consists with the conclusion reported in the previous study BIBREF8.
In addition, as shown in Figure FIGREF19, the perturbation effectiveness of Deletion, Mask, and Grammatical Replacement varies from strong to weak. In the following experiments, we choose Mask as the representative perturbation operation for its moderate perturbation performance, based on which we compare two most effective methods Attribution and Attention.

Experiment ::: Comparison with Supervised Erasure
There exists another straightforward method, Erasure BIBREF7, BIBREF22, BIBREF25, which directly evaluates the word importance by measuring the translation performance degradation of each word. Specifically, it erases (i.e., Mask) one word from the input sentence each time and uses the BLEU score changes to denote the word importance (after normalization).
In Figure FIGREF31, we compare Erasure method with Attribution method under the Mask perturbation. The results show that Attribution method is less effective than Erasure method when only one word is perturbed. But it outperforms the Erasure method when perturbing 2 or more words. The results reveal that the importance calculated by erasing only one word cannot be generalized to multiple-words scenarios very well. Besides, the Erasure method is a supervised method which requires ground-truth references, and finding a better words combination is computation infeasible when erasing multiple words.
We close this section by pointing out that our gradient-based method consistently outperforms its black-box counterparts in various settings, demonstrating the effectiveness and universality of exploiting gradients for estimating word importance. In addition, our approach is on par with or even outperforms the supervised erasure method (on multiple-word perturbations). This is encouraging since our approach does not require any external resource and is fully unsupervised.

Analysis ::: Effect on Detecting Translation Errors
In this experiment, we propose to use the estimated word importance to detect the under-translated words by NMT models. Intuitively, under-translated input words should contribute little to the NMT outputs, yielding much smaller word importance. Given 500 Chinese$\Rightarrow $English sentence pairs translated by the Transformer model (BLEU 23.57), we ask ten human annotators to manually label the under-translated input words, and at least two annotators label each input-hypothesis pair. These annotators have at least six years of English study experience, whose native language is Chinese. Among these sentences, 178 sentences have under-translation errors with 553 under-translated words in total.
Table TABREF32 lists the accuracy of detecting under-translation errors by comparing words of least importance and human-annotated under-translated words. As seen, our Attribution method consistently and significantly outperforms both Erasure and Attention approaches. By exploiting the word importance calculated by Attribution method, we can identify the under-translation errors automatically without the involvement of human interpreters. Although the accuracy is not high, it is worth noting that our under-translation method is very simple and straightforward. This is potentially useful for debugging NMT models, e.g., automatic post-editing with constraint decoding BIBREF26, BIBREF27.

Analysis ::: Analysis on Linguistic Properties ::: Words of high fertility are always important.
We further compare the fertility distribution based on word importance and the word frequency on three language pairs. We hypothesize that a source word that corresponds to multiple target words should be more important since it contributes more to both sentence length and BLEU score.
Table TABREF36 lists the results. Overall speaking, one-to-many fertility is consistently more important on all three language pairs, which confirms our hypothesis. On the contrary, null-aligned words receive much less attention, which shows a persistently decrease on three language pairs. It is also reasonable since null-aligned input words contribute almost nothing to the translation outputs.

Analyses on Reverse Directions ::: Syntactic Categories
On English$\Rightarrow $Chinese, content words are more important than content-free words, while the situation is reversed on both French$\Rightarrow $English and Japanese$\Rightarrow $English translations. Since there is no clear boundary between Preposition/Determiner and other categories in Japanese, we set both categories to be none. Similarly, Punctuation is more important on Japanese$\Rightarrow $English, which is in line with the finding on English$\Rightarrow $Japanese. Overall speaking, it might indicate that the Syntactic distribution with word importance is language-pair related instead of the direction.

Figure 2: Effect of three types of synthetic perturbations on Chinese⇒English translation using the Transformer.

Figure 3: Effect of the Mask perturbation on (a) Chinese⇒English translation using the RNN-Search model, (b, c, d, e, f) other language pairs and directions using Transformer model.

Table 2: Correlation between Attribution word importance with POS tags, Fertility, and Syntactic Depth. Fertility can be categorized into 4 types: one-to-many (“≥ 2”), one-to-one (“1”), many-to-one (“(0, 1)”), and null-aligned (“0”). Syntactic depth shows the depth of a word in the dependency tree. A lower tree depth indicates closer to the root node in the dependency tree, which might indicate a more important word.

Table 4: Distributions of word fertility and their relative change based on Attribution importance and word count.

Question: How do they measure which words are under-translated by NMT models?
{"answer": ["we ask ten human annotators to manually label the under-translated input words, and at least two annotators label each input-hypothesis pair"], "evidence": ["In this experiment, we propose to use the estimated word importance to detect the under-translated words by NMT models. Intuitively, under-translated input words should contribute little to the NMT outputs, yielding much smaller word importance. Given 500 Chinese$\\Rightarrow $English sentence pairs translated by the Transformer model (BLEU 23.57), we ask ten human annotators to manually label the under-translated input words, and at least two annotators label each input-hypothesis pair. These annotators have at least six years of English study experience, whose native language is Chinese. Among these sentences, 178 sentences have under-translation errors with 553 under-translated words in total."]}