You will be shown sections from a scientific research paper, together with a question about the paper. Your task is to answer the question based on the contents of the paper. Your response should be a `json` object with two fields:

"answer": An array of strings extracted directly from the paper which, collectively, answer the question.
"evidence": An array of strings. Each should be an excerpt from the paper, in which one or more of the extracted answers can be found.

For instance, for the question "What baselines did the authors compare against?", a sample response might be:

{
  "answer": ["BERT", "RoBERTa"],
  "evidence": ["In our experiments, we compare the performance of our model against BERT and RoBERTa."]
}

Do not include any text in your response other than the json.

If the question is unanswerable given the provided excerpts, respond with the single word "null".

Paper: From Free Text to Clusters of Content in Health Records: An Unsupervised Graph Partitioning Approach

Electronic Healthcare records contain large volumes of unstructured data in different forms. Free text constitutes a large portion of such data, yet this source of richly detailed information often remains under-used in practice because of a lack of suitable methodologies to extract interpretable content in a timely manner. Here we apply network-theoretical tools to the analysis of free text in Hospital Patient Incident reports in the English National Health Service, to find clusters of reports in an unsupervised manner and at different levels of resolution based directly on the free text descriptions contained within them. To do so, we combine recently developed deep neural network text-embedding methodologies based on paragraph vectors with multi-scale Markov Stability community detection applied to a similarity graph of documents obtained from sparsified text vector similarities. We showcase the approach with the analysis of incident reports submitted in Imperial College Healthcare NHS Trust, London. The multiscale community structure reveals levels of meaning with different resolution in the topics of the dataset, as shown by relevant descriptive terms extracted from the groups of records, as well as by comparing a posteriori against hand-coded categories assigned by healthcare personnel. Our content communities exhibit good correspondence with well-defined hand-coded categories, yet our results also provide further medical detail in certain areas as well as revealing complementary descriptors of incidents beyond the external classification. We also discuss how the method can be used to monitor reports over time and across different healthcare providers, and to detect emerging trends that fall outside of pre-existing categories.

Multiscale graph partitioning for text analysis: description of the framework
Our framework combines text-embedding, geometric graph construction and multi-resolution community detection to identify, rather than impose, content-based clusters from free, unstructured text in an unsupervised manner.
Figure 1 shows a summary of our pipeline. First, we pre-process each document to transform text into consecutive word tokens, where words are in their most normalised forms, and some words are removed if they have no distinctive meaning when used out of context BIBREF5 , BIBREF6 . We then train a paragraph vector model using the Document to Vector (Doc2Vec) framework BIBREF7 on the whole set (13 million) of preprocessed text records, although training on smaller sets (1 million) also produces good results. This training step is only done once. This Doc2Vec model is subsequently used to infer high-dimensional vector descriptions for the text of each of the 3229 documents in our target analysis set. We then compute a matrix containing pairwise similarities between any pair of document vectors, as inferred with Doc2Vec. This matrix can be thought of as a full, weighted graph with documents as nodes and edges weighted by their similarity. We sparsify this graph to the union of a minimum spanning tree and a k-Nearest Neighbors (MST-kNN) graph BIBREF8 , a geometric construction that removes less important similarities but preserves global connectivity for the graph and, hence, for the dataset. The derived MST-kNN graph is analysed with Markov Stability BIBREF9 , BIBREF10 , BIBREF11 , BIBREF12 , a multi-resolution dynamics-based graph partitioning method that identifies relevant subgraphs (i.e., clusters of documents) at different levels of granularity. MS uses a diffusive process on the graph to reveal the multiscale organisation at different resolutions without the need for choosing a priori the number of clusters, scale or organisation. To analyse a posteriori the different partitions across levels of resolution, we use both visualisations and quantitative scores. The visualisations include word clouds to summarise the main content, graph layouts, as well as Sankey diagrams and contingency tables that capture the correspondences across levels of resolution and relationships to the hand-coded classifications. The partitions are also evaluated quantitatively to score: (i) their intrinsic topic coherence (using pairwise mutual information BIBREF13 , BIBREF14 ), and (ii) their similarity to the operator hand-coded categories (using normalised mutual information BIBREF15 ). We now expand on the steps of the computational framework.

Quantitative benchmarking of topic clusters
Although our dataset has attached a hand-coded classification by a human operator, we do not use it in our analysis and we do not consider it as a `ground truth'. Indeed, one of our aims is to explore the relevance of the fixed external classes as compared to the content-driven groupings obtained in an unsupervised manner. Hence we provide a double route to quantify the quality of the clusters by computing two complementary measures: an intrinsic measure of topic coherence and a measure of similarity to the external hand-coded categories, defined as follows.
As an intrinsic measure of consistency of word association without any reference to an external `ground truth', we use the pointwise mutual information ( $PMI$ ) BIBREF13 , BIBREF14 . The $PMI$ is an information-theoretical score that captures the probability of being used together in the same group of documents. The $PMI$ score for a pair of words $(w_1,w_2)$ is: 
$$PMI(w_1,w_2)=\log {\frac{P(w_1 w_2)}{P(w_1)P(w_2)}}$$   (Eq. 6) 
where the probabilities of the words $P(w_1)$ , $P(w_2)$ , and of their co-occurrence $P(w_1 w_2)$ are obtained from the corpus. To obtain the aggregate $\widehat{PMI}$ for the graph partition $C=\lbrace c_i\rbrace $ we compute the $PMI$ for each cluster, as the median $PMI$ between its 10 most common words (changing the number of words gives similar results), and we obtain the weighted average of the $PMI$ cluster scores: 
$$\widehat{PMI} (C) = \sum _{c_i \in C} \frac{n_i}{N} \, \mathop {\operatorname{median}}_{\begin{array}{c}w_k, w_\ell \in S_i \\ k<\ell \end{array}} PMI(w_k,w_\ell ),$$   (Eq. 7) 
 where $c_i$ denotes the clusters in partition $C$ , each with size $n_i$ ; $N=\sum _{c_i \in C} n_i$ is the total number of nodes; and $S_i$ denotes the set of top 10 words for cluster $c_i$ .
We use this $\widehat{PMI}$ score to evaluate partitions without requiring a labelled ground truth.The $PMI$ score has been shown to perform well BIBREF13 , BIBREF14 when compared to human interpretation of topics on different corpora BIBREF39 , BIBREF40 , and is designed to evaluate topical coherence for groups of documents, in contrast to other tools aimed at short forms of text. See BIBREF18 , BIBREF19 , BIBREF41 , BIBREF42 for other examples.
To compare against the external classification a posteriori, we use the normalised mutual information ( $NMI$ ), a well-used information-theoretical score that quantifies the similarity between clusterings considering both the correct and incorrect assignments in terms of the information (or predictability) between the clusterings. The NMI between two partitions $C$ and $D$ of the same graph is: 
$$NMI(C,D)=\frac{I(C,D)}{\sqrt{H(C)H(D)}}=\frac{\sum \limits _{c \in C} \sum \limits _{d \in D} p(c,d) \, \log \dfrac{p(c,d)}{p(c)p(d)}}{\sqrt{H(C)H(D)}}$$   (Eq. 8) 
where $I(C,D)$ is the Mutual Information and $H(C)$ and $H(D)$ are the entropies of the two partitions.
The $NMI$ is bounded ( $0 \le NMI \le 1$ ) with a higher value corresponding to higher similarity of the partitions (i.e., $NMI=1$ when there is perfect agreement between partitions $C$ and $D$ ). The $NMI$ score is directly relatedhttp://scikit-learn.org/stable/modules/generated/sklearn.metrics.v_measure_score.html to the V-measure used in the computer science literature BIBREF43 . We use the $NMI$ to compare the partitions obtained by MS (and other methods) against the hand-coded classification assigned by the operator.

Multi-resolution community detection extracts content clusters at different levels of granularity
We applied MS across a broad span of Markov times ( $t \in [0.01, 100]$ in steps of 0.01) to the MST-kNN similarity graph of $N=3229$ incident records. At each Markov time, we ran 500 independent optimisations of the Louvain algorithm and selected the optimal partition at each time. Repeating the optimisation from 500 different initial starting points enhances the robustness of the outcome and allows us to quantify the robustness of the partition to the optimisation procedure. To quantify this robustness, we computed the average variation of information $VI(t)$ (a measure of dissimilarity) between the top 50 partitions for each $t$ . Once the full scan across Markov time was finalised, a final comparison of all the optimal partitions obtained was carried out, so as to assess if any of the optimised partitions was optimal at any other Markov time, in which case it was selected. We then obtained the $VI(t,t^{\prime })$ across all optimal partitions found across Markov times to ascertain when partitions are robust across levels of resolution. This layered process of optimisation enhances the robustness of the outcome given the NP-hard nature of MS optimisation, which prevents guaranteed global optimality.
Figure 3 presents a summary of our analysis. We plot the number of clusters of the optimal partition and the two metrics of variation of information across all Markov times. The existence of a long plateau in $VI(t,t^{\prime })$ coupled to a dip in $VI(t)$ implies the presence of a partition that is robust both to the optimisation and across Markov time. To illustrate the multi-scale features of the method, we choose several of these robust partitions, from finer (44 communities) to coarser (3 communities), obtained at five Markov times and examine their structure and content. We also present a multi-level Sankey diagram to summarise the relationships and relative node membership across the levels.
The MS analysis of the graph of incident reports reveals a rich multi-level structure of partitions, with a strong quasi-hierarchical organisation, as seen in the graph layouts and the multi-level Sankey diagram. It is important to remark that, although the Markov time acts as a natural resolution parameter from finer to coarser partitions, our process of optimisation does not impose any hierarchical structure a priori. Hence the observed consistency of communities across level is intrinsic to the data and suggests the existence of content clusters that naturally integrate with each other as sub-themes of larger thematic categories. The detection of intrinsic scales within the graph provided by MS thus enables us to obtain clusters of records with high content similarity at different levels of granularity. This capability can be used by practitioners to tune the level of description to their specific needs.

Competing interests
The authors declare that they have no competing interests.

Authors' contributions
MTA conducted the computational research. MTA and MB analysed the data and designed the computational framework. MB, EM and SNY conceived the study. All authors wrote the manuscript.

Table 1 Benchmarking of text corpora used for Doc2Vec training

Question: Which text embedding methodologies are used?
{"answer": ["Document to Vector (Doc2Vec)"], "evidence": ["Figure 1 shows a summary of our pipeline. First, we pre-process each document to transform text into consecutive word tokens, where words are in their most normalised forms, and some words are removed if they have no distinctive meaning when used out of context BIBREF5 , BIBREF6 . We then train a paragraph vector model using the Document to Vector (Doc2Vec) framework BIBREF7 on the whole set (13 million) of preprocessed text records, although training on smaller sets (1 million) also produces good results. This training step is only done once. This Doc2Vec model is subsequently used to infer high-dimensional vector descriptions for the text of each of the 3229 documents in our target analysis set. We then compute a matrix containing pairwise similarities between any pair of document vectors, as inferred with Doc2Vec. This matrix can be thought of as a full, weighted graph with documents as nodes and edges weighted by their similarity. We sparsify this graph to the union of a minimum spanning tree and a k-Nearest Neighbors (MST-kNN) graph BIBREF8 , a geometric construction that removes less important similarities but preserves global connectivity for the graph and, hence, for the dataset. The derived MST-kNN graph is analysed with Markov Stability BIBREF9 , BIBREF10 , BIBREF11 , BIBREF12 , a multi-resolution dynamics-based graph partitioning method that identifies relevant subgraphs (i.e., clusters of documents) at different levels of granularity. MS uses a diffusive process on the graph to reveal the multiscale organisation at different resolutions without the need for choosing a priori the number of clusters, scale or organisation. To analyse a posteriori the different partitions across levels of resolution, we use both visualisations and quantitative scores. The visualisations include word clouds to summarise the main content, graph layouts, as well as Sankey diagrams and contingency tables that capture the correspondences across levels of resolution and relationships to the hand-coded classifications. The partitions are also evaluated quantitatively to score: (i) their intrinsic topic coherence (using pairwise mutual information BIBREF13 , BIBREF14 ), and (ii) their similarity to the operator hand-coded categories (using normalised mutual information BIBREF15 ). We now expand on the steps of the computational framework."]}