You will be shown sections from a scientific research paper, together with a question about the paper. Your task is to answer the question based on the contents of the paper. Your response should be a `json` object with two fields:

"answer": An array of strings extracted directly from the paper which, collectively, answer the question.
"evidence": An array of strings. Each should be an excerpt from the paper, in which one or more of the extracted answers can be found.

For instance, for the question "What baselines did the authors compare against?", a sample response might be:

{
  "answer": ["BERT", "RoBERTa"],
  "evidence": ["In our experiments, we compare the performance of our model against BERT and RoBERTa."]
}

Do not include any text in your response other than the json.

If the question is unanswerable given the provided excerpts, respond with the single word "null".

Paper: Extractive Summarization of Long Documents by Combining Global and Local Context

In this paper, we propose a novel neural single document extractive summarization model for long documents, incorporating both the global context of the whole document and the local context within the current topic. We evaluate the model on two datasets of scientific papers, Pubmed and arXiv, where it outperforms previous work, both extractive and abstractive models, on ROUGE-1, ROUGE-2 and METEOR scores. We also show that, consistently with our goal, the benefits of our method become stronger as we apply it to longer documents. Rather surprisingly, an ablation study indicates that the benefits of our model seem to come exclusively from modeling the local context, even for the longest documents.

Introduction
Single-document summarization is the task of generating a short summary for a given document. Ideally, the generated summaries should be fluent and coherent, and should faithfully maintain the most important information in the source document. purpleThis is a very challenging task, because it arguably requires an in-depth understanding of the source document, and current automatic solutions are still far from human performance BIBREF0 .
Single-document summarization can be either extractive or abstractive. Extractive methods typically pick sentences directly from the original document based on their importance, and form the summary as an aggregate of these sentences. Usually, summaries generated in this way have a better performance on fluency and grammar, but they may contain much redundancy and lack in coherence across sentences. In contrast, abstractive methods attempt to mimic what humans do by first extracting content from the source document and then produce new sentences that aggregate and organize the extracted information. Since the sentences are generated from scratch they tend to have a relatively worse performance on fluency and grammar. Furthermore, while abstractive summaries are typically less redundant, they may end up including misleading or even utterly false statements, because the methods to extract and aggregate information form the source document are still rather noisy.
In this work, we focus on extracting informative sentences from a given document (without dealing with redundancy), especially when the document is relatively long (e.g., scientific articles).
Most recent works on neural extractive summarization have been rather successful in generating summaries of short news documents (around 650 words/document) BIBREF1 by applying neural Seq2Seq models BIBREF2 . However when it comes to long documents, these models tend to struggle with longer sequences because at each decoding step, the decoder needs to learn to construct a context vector capturing relevant information from all the tokens in the source sequence BIBREF3 .
Long documents typically cover multiple topics. In general, the longer a document is, the more topics are discussed. As a matter of fact, when humans write long documents they organize them in chapters, sections etc.. Scientific papers are an example of longer documents and they follow a standard discourse structure describing the problem, methodology, experiments/results, and finally conclusions BIBREF4 .
To the best of our knowledge only one previous work in extractive summarization has explicitly leveraged section information to guide the generation of summaries BIBREF5 . However, the only information about sections fed into their sentence classifier is a categorical feature with values like Highlight, Abstract, Introduction, etc., depending on which section the sentence appears in.
In contrast, in order to exploit section information, in this paper we propose to capture a distributed representation of both the global (the whole document) and the local context (e.g., the section/topic) when deciding if a sentence should be included in the summary
Our main contributions are as follows: (i) In order to capture the local context, we are the first to apply LSTM-minus to text summarization. LSTM-minus is a method for learning embeddings of text spans, which has achieved good performance in dependency parsing BIBREF6 , in constituency parsing BIBREF7 , as well as in discourse parsing BIBREF8 . With respect to more traditional methods for capturing local context, which rely on hierarchical structures, LSTM-minus produces simpler models i.e. with less parameters, and therefore faster to train and less prone to overfitting. (ii) We test our method on the Pubmed and arXiv datasets and results appear to support our goal of effectively summarizing long documents. In particular, while overall we outperform the baseline and previous approaches only by a narrow margin on both datasets, the benefit of our method become much stronger as we apply it to longer documents. purpleFurthermore, in an ablation study to assess the relative contributions of the global and the local model we found that, rather surprisingly, the benefits of our model seem to come exclusively from modeling the local context, even for the longest documents.[6] (iii) In order to evaluate our approach, we have created oracle labels for both Pubmed and arXiv BIBREF9 , by applying a greedy oracle labeling algorithm. The two datasets annotated with extractive labels will be made public.

Extractive summarization on Scientific papers
Research on summarizing scientific articles has a long history BIBREF13 . Earlier on, it was realized that summarizing scientific papers requires different approaches than what was used for summarizing news articles, due to differences in document length, writing style and rhetorical structure. For instance, BIBREF14 presented a supervised Naive Bayes classifier to select content from a scientific paper based on the rhetorical status of each sentence (e.g., whether it specified a research goal, or some generally accepted scientific background knowledge, etc.). More recently, researchers have extended this work by applying more sophisticated classifiers to identify more fine-grain rhetorical categories, as well as by exploiting citation contexts. 2013-discourse propose the CoreSC discourse-driven content, which relies on CRFs and SVMs, to classify the discourse categories (e.g. Background, Hypothesis, Motivation, etc.) at the sentence level. The recent work most similar to ours is BIBREF5 where, in order to determine whether a sentence should be included in the summary, they directly use the section each sentence appears in as a categorical feature with values like Highlight, Abstract, Introduction, etc.. In this paper, instead of using sections as categorical features, we rely on a distributed representation of the semantic information within each section, as the local context of each sentence. In a very different line of work, cohan-2015-scientific form the summary by also exploiting information on how the target paper is cited in other papers. Currently, we do not use any information from citation contexts.

LSTM-Minus
The LSTM-Minus method is first proposed in BIBREF6 as a novel way to learn sentence segment embeddings for graph-based dependency parsing, i.e. estimating the most likely dependency tree given an input sentence. For each dependency pair, they divide a sentence into three segments (prefix, infix and suffix), and LSTM-Minus is used to represent each segment. They apply a single LSTM to the whole sentence and use the difference between two hidden states INLINEFORM0 to represent the segment from word INLINEFORM1 to word INLINEFORM2 . This enables their model to learn segment embeddings from information both outside and inside the segments and thus enhancing their model ability to access to sentence-level information. The intuition behind the method is that each hidden vector INLINEFORM3 can capture useful information before and including the word INLINEFORM4 .
Shortly after, lstm-minusconstituency use the same method on the task of constituency parsing, as the representation of a sentence span, extending the original uni-directional LSTM-Minus to the bi-directional case. More recently, inspired by the success of LSTM-Minus in both dependency and constituency parsing, lstm-minusdiscourse extend the technique to discourse parsing. They propose a two-stage model consisting of an intra-sentential parser and a multi-sentential parser, learning contextually informed representations of constituents with LSTM-Minus, at the sentence and document level, respectively.
Similarly, in this paper, when deciding if a sentence should be included in the summary, the local context of that sentence is captured by applying LSTM-Minus at the document level, to represent the sub-sequence of sentences of the document (i.e., the topic/section) the target sentence belongs to.

Document Encoder
At the document level, a bi-directional recurrent neural network BIBREF20 is often used to encode all the sentences sequentially forward and backward, with such model achieving remarkable success in machine translation BIBREF21 . As units, we selected gated recurrent units (GRU) BIBREF22 , in light of favorable results shown in BIBREF23 . The GRU is represented with the standard reset, update, and new gates.
The output of the bi-directional GRU for each sentence INLINEFORM0 comprises two hidden states, INLINEFORM1 as forward and backward hidden state, respectively.
A. Sentence representation As shown in Figure FIGREF10 (A), for each sentence INLINEFORM0 , the sentence representation is the concatenation of both backward and forward hidden state of that sentence. INLINEFORM1 
In this way, the sentence representation not only represents the current sentence, but also partially covers contextual information both before and after this sentence.
B. Document representation The document representation provides global information on the whole document. It is computed as the concatenation of the final state of the forward and backward GRU, labeled as B in Figure FIGREF10 . BIBREF24 INLINEFORM0 
C. Topic segment representation To capture the local context of each sentence, namely the information of the topic segment that sentence falls into, we apply the LSTM-Minus method, a method for learning embeddings of text spans. LSTM-Minus is shown in detail in Figure 1 (left panel C), each topic segment is represented as the subtraction between the hidden states of the start and the end of that topic. As illustrated in Figure FIGREF10 , the representation for section 2 of the sample document (containing three sections and eight sentences overall) can be computed as INLINEFORM0 , where INLINEFORM1 are the forward hidden states of sentence 5 and 2, respectively, while INLINEFORM2 are the backward hidden states of sentence 3 and 6, respectively. In general, the topic segment representation INLINEFORM3 for segment INLINEFORM4 is computed as: INLINEFORM5 
where INLINEFORM0 is the index of the beginning and the end of topic INLINEFORM1 , INLINEFORM2 and INLINEFORM3 denote the topic segment representation of forward and backward, respectively. The final representation of topic INLINEFORM4 is the concatenation of forward and backward representation INLINEFORM5 . To obtain INLINEFORM6 and INLINEFORM7 , we utilize subtraction between GRU hidden vectors of INLINEFORM8 and INLINEFORM9 , and we pad the hidden states with zero vectors both in the beginning and the end, to ensure the index can not be out of bound. The intuition behind this process is that the GRUs can keep previous useful information in their memory cell by exploiting reset, update, and new gates to decide how to utilize and update the memory of previous information. In this way, we can represent the contextual information within each topic segment for all the sentences in that segment.

Experiments
To validate our method, we set up experiments on the two scientific paper datasets (arXiv and PubMed). With ROUGE and METEOR scores as automatic evaluation metrics, we compare with previous works, both abstractive and extractive.

Ablation Study
In order to assess the relative contributions of the global and local models to the performance of our approach, we ran an ablation study. This was done for each dataset both with the whole test set, as well as with a subset of long documents. The results for Pubmed and arXiv are shown in Table TABREF34 and Table TABREF35 , respectively. For statistical significance, as it was done for the general results in Section 4.5, we use the approximate randomization method BIBREF32 with the Bonferroni correction at ( INLINEFORM0 ).
From these tables, we can see that on both datasets the performance significantly improves when local topic information (i.e. local context) is added. And the improvement is even greater when we only consider long documents. Rather surprisingly, this is not the case for the global context. Adding a representation of the whole document (i.e. global context) never significantly improves performance. In essence, it seems that all the benefits of our model come exclusively from modeling the local context, even for the longest documents. Further investigation of this finding is left as future work.

Acknowledgments
This research was supported by the Language & Speech Innovation Lab of Cloud BU, Huawei Technologies Co., Ltd. Extractive Label Generation The algorithm SECREF6 is used to generate the extractive labels based on the human-made abstractive summaries, i.e. abstracts of scientific papers. Extractive label generation LabelGenerationReference,sentences,lengthLimit INLINEFORM0 = ” INLINEFORM1 = 0 INLINEFORM2 = [] INLINEFORM3 INLINEFORM4 INLINEFORM5 INLINEFORM6 in range(len( INLINEFORM7 )) INLINEFORM8 INLINEFORM9 INLINEFORM10 INLINEFORM11 INLINEFORM12 != INLINEFORM13 INLINEFORM14 .append( INLINEFORM15 ) INLINEFORM16 = INLINEFORM17 + INLINEFORM18 [ INLINEFORM19 ] INLINEFORM20 += NumberOfWords( INLINEFORM21 [ INLINEFORM22 ]) break INLINEFORM23 

Figure 2.3: The structure of Cohan’s model of discourse-aware abstractive summarization [8]

Figure 2.4: LSTM-Minus in dependency parsing.

Figure 2.5: The supervised topic segmentation model proposed in [21]

Figure 2.6: The approximate randomization statistical significance test. [31]

Figure 3.1: The structure of our model, sei,sri represent the sentence embedding and sentence representation of sentence i, respectively. The binary decision of whether the sentence should be included in the summary is based on the sentence itself (A), the whole document (B) and the current topic (C). The document representation is simply the concatenation of the last hidden states of the forward and backward RNNs, while the topic segment representation is computed by applying LSTM-Minus, as the details shown in Fig 3.2

Table 4.1: Results on the arXiv dataset. For models with an ∗, we report results from [8]. Models are traditional extractive in the first block, neural abstractive in the second block, while neural extractive in the third block. The Oracle (last row) corresponds to using the ground truth labels, obtained (for training) by the greedy algorithm, see Section 4.1.2. Results that are not significantly distinguished from the best systems are bold.

Table 4.2: Results on the Pubmed dataset. For models with an ∗, we report results from [8]. See caption of Table 4.1 above for details on compared models. Results that are not significantly distinguished from the best systems are bold.

Table 4.3: Percentage relative improvement of our model, when compared with the SummaRuNNer (SR) and Baseline (BSL) models on both datasets (first and second block). The third block shows Macro average relative improvement across the two datasets .

Table 4.5: Ablation study on the arXiv dataset. The model descriptions refer to Table 4.4. Results that are not significantly distinguished from the best systems are bold.

Table 4.6: Results on the Bigpatent-A dataset.

Figure 4.3: A Comparison between our model, SummaRuNNer and Oracle when applied to documents with increasing length on Bigpatent-a dataset

Question: What do they mean by global and local context?
{"answer": ["global (the whole document)", "local context (e.g., the section/topic)"], "evidence": ["In contrast, in order to exploit section information, in this paper we propose to capture a distributed representation of both the global (the whole document) and the local context (e.g., the section/topic) when deciding if a sentence should be included in the summary"]}