You will be shown an excerpt from a computer science scientific research paper, followed by a question about the paper. Please answer the question. Do not include any text in your response other than the answer.

Context: The Attentive Reader can be viewed as a generalisation of the application ofMemory Networks to question answering [3]. That model employsan attention mechanism at the sentence level where each sentence is representedby a bag of embeddings. The Attentive Reader employs a finer grained tokenlevel attention mechanism where the tokens are embedded given their entirefuture and past context in the input document. We can visualise the attention mechanism as a heatmap over a context document togain further insight into the models’ performance. The highlighted words showwhich tokens in the document were attended to by the model. In addition we mustalso take into account that the vectors at each token integrate long rangecontextual information via the bidirectional LSTM encoders.Figure 3 depicts heat maps for two queries that were correctlyanswered by the Attentive Reader.777Note that these examples were chosenas they were short, the average CNN validation document contained 763 tokens and27 entities, thus most instances were significantly harder to answer than theseexamples. In both cases confidently arriving at the correct answer requires themodel to perform both significant lexical generalsiation, e.g. ‘killed’\rightarrow ‘deceased’, and co-reference or anaphora resolution, e.g. ‘ent119 was killed’ \rightarrow ‘he was identified.’ However it is alsoclear that the model is able to integrate these signals with rough heuristicindicators such as the proximity of query words to the candidate answer.

Question: What is the main mathematical difference between the attentive LSTM reader and the vanilla Deep LSTM?
The main difference between the attention-based LSTM and the vanilla one is that the former addresses the limitation of vanilla LSTM’s fixed and limited context size by taking into account the entire context of every token via a token-level attention mechanism.