text:
\section{FHIR Interface}

We have implemented a prototype Android mobile application to demonstrate the usefulness and applicability of this work. This prototype app supports causal chain prediction, patient search, patient information display, determining causes of death, and review/submit. The app allows a physician to add death-related information when filling out the "Pronouncing Death" screen and the data bundle is compatible with FHIR servers. When generating causal chain of death for clinical decision support, the app will automatically retrieve medical condition codes from the FHIR server, and query the python pre-trained neural machine translation model for generating the casual chain of death. \textbf{Figure. \ref{fig:causal-chain-display}} shows a screenshot of the FHIR Android app when displaying the causal chain of death. The ICD-10 codes have already been mapped into human-readable short descriptions. From top to bottom, we show the ordered causes of death (from underlying cause of death to immediate causes of death). "N/A" indicates null value for the cause of death.

The use of mobile apps such as this prototype for delivering public health informatics creates the opportunity for real-time, point-of-care feedback. In the immediate term, clinicians completing mortality reporting data can be provided with decision support capability to improve the accuracy and completeness of the causes of death reported. In the future, such infrastructure may even be able to provide predicted causes of death for still-living patients, enabling predictive medical care.

\begin{figure}[h!]
    \centering
    \includegraphics[scale=0.45]{figures/FHIR_Android_App_2.png}
    \caption{Screenshot of causal chain prediction display in the Android app. From top to bottom, the diagnosis codes correspond to the order of the output sentence. The ICD-10 codes are mapped into human-readable short descriptions.}
    \label{fig:causal-chain-display}
\end{figure}

\subsection{Evaluation}
Furthermore, we need to increase the interpretability and demonstrate it through visualization. Several studies visualized attention weights to showcase that their models captured the correspondence words between English and French \cite{bahdanau2014neural} \cite{yang2016hierarchical} \cite{vaswani2017attention}. The ability to demonstrate the correspondence of words between languages through visualization is also critical for our task. Because the discharge records and causes of death are coded under different versions of ICD codes, associating a cause of death to several past diseases could help us qualitatively evaluate generated causal sequences.

In addition to qualitative evaluation, quantitative evaluation remains critical. Here we evaluate how well our proposed causal chain $\hat{Y}=\{\hat{Y}_1, .., \hat{Y}_{M_1}\}$ aligns with the physicians' decision, i.e., $Y = \{Y_1, .., Y_{M_2} \}$.
Here $Y_i$ is the individual codes, and $M_1, M_2$ are the respective length of the chains.
A perfect alignment means $M_1 = M_2$, and $\hat{Y}_i = Y_i$, for $i=1, ..., M_1$.
However, this is rarely the case, thus we compute a weighted average precision of our alignment in sub-sequences of variable lengths, i.e., the BLEU (BiLingual Evaluation Understudy) score \cite{papineni2002bleu}. Following natural language processing literature, we call sub-sequence of length $i$ "i-grams".
BLEU score ranges from 0 to 1 or (or from 0 to 100 if multiplied by 100), and the higher BLEU, the higher we have an alignment with physicians clinically.

We use a simple example as follows to illustrate the computation of the BLEU score. In our proposed candidate sequence, the underlying cause of death, \textit{Asphyxia and Hypoxemia (R909)} leads to \textit{Pneumonia, Unspecified Organism (J189)} which leads to \textit{Respiratory failure, unspecified (J969)}.

$$\hat{Y} = R909 \rightarrow J189 \rightarrow J969 $$

The reference sequence, determined by the physician, consists of \textit{Asphyxia and Hypoxemia (R909)}, \textit{Pneumonia, Unspecified Organism (J189)} and then \textit{Acute Respiratory Failure (J960)}.

$$ Y = R909 \rightarrow J189 \rightarrow J960 $$

As shown in \textbf{Table \ref{tab:bleu}}, we first list 1-grams and 2-grams from $\hat{Y}$ and $Y$, and we compute the precision for the two case. Here the definition of precision is similar in the classification setting: among all the predictions we made in candidate sequence $\hat{Y}$, how many we get correct in the reference sequence $Y$?
After we compute all the precision metrics, we calculate the geometric average of them as the BLEU metrics, in this case, approximately 0.47.

\begin{table}[!h]
\centering
\caption{An example of 1-gram precision and 2-gram precision in BLEU score.}
\label{tab:bleu}
\resizebox{.49\textwidth}{!}{%
\begin{tabular}{|l|l|l|r|}
\hline
Grams & From Candidate $\hat{Y}$ & Appear in $Y$ & Precision \\ \hline
1-gram & (R909), (J189), (J969) & (R909), (J189) & 2/3 \\ \hline
2-gram & (R909, J189), (J189, J969) & (R909, J189) & 1/2 \\ \hline
\end{tabular}%
}
\end{table}

In natural language settings, people usually calculate BLEU score for the geometric average up to 4-gram precision. In our case, however, we only compute the geometric average up to 2-gram precision, and apply clipping to each of the precision. This is due to the fact that the average length of causal chain of death in Michigan dataset is $2.25$ codes so including 3-gram precision will lead to substantially inaccurate evaluation. Furthermore, we also include a brevity penalty to penalize sentences that are too short.
$$BLEU = min\{ 1, \exp{(1-\frac{length(Y)}{length(\hat{Y})})} \} (\prod_{i=1}^2 precision_i)^{1/2} $$
where $precision_i$ is the i-gram precision, defined as
$$\frac{\sum_{\hat{Y}} \text{the count of i-grams in } \hat{Y} \text{ that appears in } Y}{ \sum_{\hat{Y}} \text{the count of i-grams in } \hat{Y}}$$

In this equation, $exp$ is the natural exponential function; $length(Y)$ is the length of reference sequence (how many words in the reference sequence); likewise, $length(\hat{Y})$ is the length of the proposed sequence (how many words in the proposed sequence). We refer readers to the original paper for a detailed discussion on the variants of BLEU metric \cite{papineni2002bleu}.

For clinical interpretation, our modified BLEU score indicates how well our proposed sub-sequences of causal conditions match the physicians' results. The 1-gram precision emphasizes individual condition codes matching, while 2-gram precision evaluates the causal relationship between two neighboring condition codes. Physicians can manually check whether the generated causal relationship between any two neighboring condition codes fulfills or contradicts their medical domain knowledge; in addition, a data-driven algorithm can incorporate ACME decision table as medical domain ground truth to assess the validity of two neighboring condition codes.

In \textbf{Table \ref{tab:bleu_example_2}}, we show an example of different candidate sequences that have perfect 1-gram precision but different 2-gram precision. The reference sequence from underlying cause of death to immediate cause of death is: \textit{I251 (Atherosclerotic heart disease of native coronary artery), I38 (Endocarditis, valve unspecified), I429 (Cardiomyopathy, unspecified) and I469 (Cardiac arrest, cause unspecified)}. We argue that our modified BLEU score favors candidate sequences that have more reasonable and feasible condition codes with pairwise casual relationship.

\begin{table}[h!]
    \centering
    \caption{Our modified BLEU score for different candidate sequences.}
    \begin{tabular}{|l|l|r|}
    \hline
         & Sequence & BLEU \\ \hline
        Reference & $\textcolor{ForestGreen}{I251} \rightarrow \textcolor{Dandelion}{I38} \rightarrow \textcolor{Magenta}{I429} \rightarrow \textcolor{Red}{I469}$ & \\ \hline \hline
        Candidate 1 & $\textcolor{Magenta}{I429} \rightarrow \textcolor{Dandelion}{I38} \rightarrow \textcolor{Red}{I469} \rightarrow \textcolor{ForestGreen}{I251}$ & 0.0 \\ \hline
        Candidate 2 & $\textcolor{Dandelion}{I38} \rightarrow \textcolor{Magenta}{I429} \rightarrow \textcolor{ForestGreen}{I251} \rightarrow \textcolor{Red}{I469}$ & 57.7 \\ \hline
        Candidate 3 & $\textcolor{Magenta}{I429} \rightarrow \textcolor{Red}{I469} \rightarrow \textcolor{ForestGreen}{I251} \rightarrow \textcolor{Dandelion}{I38}$ & 81.6 \\ \hline
        Candidate 4 & $\textcolor{Dandelion}{I38} \rightarrow \textcolor{Magenta}{I429} \rightarrow \textcolor{Red}{I469} \rightarrow \textcolor{ForestGreen}{I251}$ & 81.6 \\ \hline
        Candidate 5 & $\textcolor{ForestGreen}{I251} \rightarrow \textcolor{Dandelion}{I38} \rightarrow \textcolor{Magenta}{I429} \rightarrow \textcolor{Red}{I469}$ & 100.0 \\ \hline
    \end{tabular}
    \label{tab:bleu_example_2}
\end{table}

In addition to our modified BLEU score, we also include three other evaluation criteria: the accuracy for predicting the entire output sequence correctly, the accuracy of predicting single code correctly in the output sequence, and the accuracy for predicting the underlying cause of death correctly.
\section{Conclusion}

In this paper, we are the first to successfully propose the causal chain of death using neural machine translation frameworks to support the timely, accurate and complete death reporting. The generated sequence has a BLEU score of $16.04$, close to the performance of the state-of-the-art in natural language domain (English-Czech translation task achieving BLEU score $17.7$ given the same vocabulary size around 10k). We also evaluate the model performance using three different accuracy scores, achieving 76.5\% accuracy in generating the individual code in output sequence.

In addition, we incorporate and explore medical domain knowledge as constraint when generating output sequence. We show that domain knowledge constraint has mixed impact on the encoder-decoder models, and argue that these models can learn the causal relationship between diagnosis codes from the data. Meanwhile, we demonstrate that validity check can be a critical step in the pipeline, which improves the stability of the encoder-decoder frameworks and may slightly improve the results.

Furthermore, we have visualized the results with attention mechanism, providing a tool to explore the relationship between certain condition codes in source sentence and those in target sentence. Lastly, we demonstrate a FHIR compatible mobile app to retrieve, modify and upload data.

Still, there are a few limitations with this work. The visualization of attention mechanism clearly shows the failure of alignment during translation, largely due to the extremely imbalanced lengths between source and target sentences. Furthermore, even though that the cross-lingual language modeling (XLM) has proven its effectiveness in natural language translation, it fails on our task. One potential cause is that the masked language modeling might not work on extremely short sentences (in average $2.25$ words per sentence).

One unsolvable problem is the one-word target sentence. Only in very rare condition we shall see a sentence consists of just one word in natural language; yet $31.77\%$ of training data, $31.68\%$ of validation data and $31.27\%$ of testing data are one-word target sentences. These samples significantly undermine the effectiveness of neural machine translation models.

Future work includes data augmentation so that the target sentence length will fit the newest masked language model (such as XLM). The model framework needs adapting towards the imbalanced source and target sentences.
\section{Experiments}

By using OpenNMT package \cite{opennmt}, we have tested five different encoder-decoder models. In addition to OpenNMT, we have incorporated the state-of-the-art pretraining model named cross-lingual language model (XLM) \cite{lample2019cross} on our data set.

To extend the scope of this work, we would like to explore the feasibility of applying encoder-decoder frameworks on current electronic health records (EHRs) data in ICD-10 codes. As the input data from the Michigan dataset is coded in ICD-9, we choose to map the input ICD-9 codes into ICD-10 codes using General Equivalence Mappings published by Centers for Medicare \& Medicaid Services (CMS) \footnote{https://www.nber.org/research/data/icd-9-cm-and-icd-10-cm-and-icd-10-pcs-crosswalk-or-general-equivalence-mappings}.

Overall, we have conducted four experiments on ICD-9 input codes (four combinations with or without validity check, with or without knowledge constraint) and one experiment on ICD-10 input codes without validity check and knowledge constraint.

\subsection{OpenNMT}

OpenNMT serializes the training, validation and vocabulary data into PyTorch files for preprocessing. As the Michigan data have a relatively small sample size comparing with datasets used in natural language processing, our models have a small number of parameters but similar architecture as the state-of-the-art models. During training, we use the 2-layer LSTM model, with 500 hidden units in each layer for the default LSTM encoder framework (Luong et al. used 4-layer LSTM model with 1000 units \cite{luong2015effective}). The mean encoder framework simply has the two LSTM layers removed in encoder but keeps other parts unchanged. For bidirectional RNN encoder, a 2-layer bidirectional LSTM with 500 and 250 hidden units is implemented. CNN-based encoder includes two gated convolutional layers with 500 and 1000 hidden units and kernel size (3,1) followed with two convolutional multi-step attention layers. The transformer has 6 stacking layers, with $2048$ hidden units in feed forward layers and 8 heads in multi-head attention layers.

We used one Nvidia GPU Tesla K80 to train and evaluate the models. Typically it took around 2-6 hours to train an LSTM, CNN or bidirectional RNN model, and about 18 hours to train a transformer model. Yet it took less than five minutes to translate all 36,000 testing data using any of these models.

\subsection{XLM: Pretraining}

XLM (cross-lingual language model) \cite{lample2019cross} incorporates masked language modeling (MLM) proposed in BERT (Bidirectional Encoder Representations from Transformers) \cite{Devlin2018bert} with the transformer model to improve translation performance. The preprocessing includes tokenizing and applying fastBPE (byte pair encoding) \cite{sennrich2015neural} to monolingual and parallel data.  MLM is the core strategy in monolingual language model pretraining. Training consists of three major steps: denosing auto-encoder, parallel data training and online back-translation.

Due to the limited size of our data set, we concatenate all training, validation and testing data into two corpora for monolingual pre-training. Masked language modeling (MLM) perplexities are used for validation during pre-training. We further train the cross-lingual model with parallel validation data and predict on parallel test data.

We set the transformer framework with 512 embedding size and 4 attention heads. We vary the encoder-decoder stacking size from 6 layers to 1 layer. The drop out rate is 0.1, attention dropout 0.1, batch size 32 and sequence length 128. We used GELU for activation and adam as optimizer.

\subsection{Optional Preprocessing: Validity Check}

In search for better prediction performance, we add an extra pre-processing step, the validity check. For training and validation data, we adopt the same algorithm in \cite{hoffman2018validity} to remove the pairs of sentences that include "invalid" causal relationship between diagnosis codes in target sentence. In this way we reduce the number of sentences in the training set from $136,753$ to $107,711$ and those in the validation set from $34,385$ to $27,009$. We then follow the same pipeline to train and translate with the same five models.
\section{Introduction}

\IEEEPARstart{T}{here} are more than 2.7 million deaths happening in the United States every year \cite{xu2018deaths}, with nearly 57 million deaths per year around the world \footnote{https://www.who.int/news-room/fact-sheets/detail/the-top-10-causes-of-death}. Accurate death reporting is essential for public health institutions such as the National Center for Health Statistics (NCHS) and the Centers for Disease Control and Prevention (CDC) to analyze vital statistics such as life expectancy and to formulate responses to communicable disease threats and epidemics.
In addition to reporting simple information such as demographics of the deceased, an important component of death reporting is to determine the causes of deaths.
The U.S. death reporting system requires two types of causes of deaths to be filled on death certificates: a \emph{single} medical condition that is the underlying cause of death, and also an ordered list, a \emph{causal chain}, of medical conditions that lead to the death.

A causal chain of death consists one underlying cause of death, and zero, one or more immediate causes of death. The immediate causes of death are typically caused by the underlying cause of death. An example causal chain of death is ``chronic obstructive pulmonary disease, unspecified (ICD10: J44.9) $\rightarrow$ other disorders of lung (ICD10: J98.4)''. Here ICD10 stands for ``10th revision of the International Statistical Classification of Diseases and Related Health Problems'', a common coding system used in death reporting \footnote{https://www.cdc.gov/nchs/icd/icd10cm.htm}.

The process of determining such causal chains of death, nevertheless, is challenging, even for an experienced physician. Such a process involves careful reasoning with one's medical domain knowledge, provoking challenges for young or inexperienced physicians. Even worse, a sudden and unexpected death might further exacerbate the process of filling the death report when the physician could only find limited electronic health records of the deceased.

Complete and accurate reporting of the full chain of causes of death has multiple benefits. This data is an invaluable public health resource for tracking the prevalence of causes of death, targeting public health interventions, and tracking the effectiveness of those interventions over time. Frequently reported chains can help physicians and public health experts understand the correlations and causal relationships between clinical diseases, potentially allowing the discovery of causal relationships that had not been previously observed. On the patient-level, personal medicine perspective, it may even be possible to warn individual patients of potential diseases leading to death even before any symptoms can be diagnosed. This can help improve clinical care and, in turn, the well-being of patients.

To facilitate the timely, accurate and complete reporting of deaths and reduce the subjectivity of reporting physicians, in this paper, we aim to develop a decision support system that suggests probable causal sequences of death based on decedents' health histories. These chains of causes of death form the basis of the NCHS Multiple Causes of Death data, a critically valuable data source in public health. These chains outline the chain of medical events and conditions which led to the death, arranged in a cause-effect order.
We identify three challenges in this task with the obtained mortality data set when predicting the causal chain of death using deep learning algorithms. \textbf{Table \ref{tab:challengesAndSolutions}} summarizes these three challenges and our proposed solutions.

\begin{table}[h!]
    \centering
    \caption{Summary of challenges in predicting causal chain of death and proposed solutions.}
    \begin{tabular}{|m{3.5cm}|m{4cm}|}

    \hline
    Challenge & Solution \\
    \hline
    Different coding versions & Machine translation between input and output sequences \\
    \hline
    Domain knowledge Conflict & Incorporate medical domain knowledge as constraint \\
    \hline
    Data interoperability & FHIR compatible platform \\
    \hline
    \end{tabular}
    \label{tab:challengesAndSolutions}
\end{table}

The first challenge comes from the different coding systems of clinical conditions. The existing causes of death (COD) in the United States have been using the International Statistical Classification of Diseases and Related Health Problems, Tenth Revision (ICD-10) since January 1999 \footnote{https://www.cdc.gov/nchs/icd/icd10cm.htm} \cite{world2004international}. On the other hand, healthcare institutions and practitioners in the U.S. were still filing patients' health record using the ninth revision (ICD-9) codes until October 2015 \cite{khera2018transition}. ICD-10 codes are "very  different" from ICD-9 codes in both coding structure and quantity: ICD-10 has nearly five times as many diagnosis codes as ICD-9 \footnote{https://www.cdc.gov/nchs/icd/icd10cm\_pcs\_faq.htm}.

Intuitively, a solution to this challenge rises from the analogy between our task and the natural language translation. The input sequence of diagnosis codes is from the last hospital discharge record of the deceased, and the output sequence is the corresponding causes of death for that decedent. Similar to translating from English sentences to French sentences, we intend to propose a succinct causal chain of death in ICD-10 codes from the priority-based discharge records of ICD-9 codes. The research area of Natural Language Processing (NLP) contains extensive studies for machine translation. Specifically, the models for machine translation can be classified into autoregressive (AR) \cite{bahdanau2014neural}\cite{dai2015semi}\cite{luong2015effective}\cite{luong2016achieving} and autoencoder (AE) models \cite{Devlin2018bert}\cite{yang2019xlnet}\cite{lample2019cross}, with the former factorizing the probability of a given corpus into a series of conditional probabilities and the latter generating output through reconstructing from corrupted input.

The second challenge is the domain knowledge conflict. As a pure data-driven approach, the deep learning model can sometimes generate confusing sequences to the physicians. Some results may even contradict the medical domain knowledge. Consequently, the physicians may find it difficult to trust the generated results. To solve this problem, we would like to incorporate medical domain knowledge to guide the deep learning framework. Particularly, we use an external source of expert-curated rules that are pairs of causal relationships between clinical condition codes. When the deep learning model searches for the next clinical condition in the process of generating output sequences, we impose the constraint that only clinical conditions following medical domain knowledge can serve as candidates.

The last challenge is the data interoperability in death reporting. Currently, the National Center for Health Statistics (NCHS) coordinates with 57 reporting jurisdictions across the United States to aggregate mortality data \cite{cowper2002primer}. These reporting jurisdictions have different regulations and local laws. To streamline the data storage and transmission between hospitals and these public health institutions and to increase data size for future big data analytics, Fast Healthcare Interoperability Resources (FHIR) is utilized to standardize mortality data reporting. We have developed one web-based FHIR \cite{hoffman2018intelligent} platform that adopts the new health standard named HL7 \cite{dolin2006hl7} to access electronic health records data. The newly developed Android version mobile app is FHIR compatible; it is capable of pre-populating different sections of death certificate to extract essential information of health history of the decedents. Furthermore, it serves as a graphical user interface for physicians that the mobile app can automatically query the pre-trained causal chain of death prediction models to provide decision support. In future versions, such applications may collect data that can be used to refine and train decision support models in real-time, expanding the impact of this work beyond retrospective analysis and improving clinical practice at the point-of-care.

In this work, we identify the encoder-decoder models as the main framework to automatically generate a causal sequence of death in ICD-10 given priority-based diagnosis codes in ICD-9 from one decedent's last hospital discharge record. Meanwhile, we demonstrate the feasibility of the encoder-decoder models for ICD-10 input data by mapping ICD-9 codes to ICD-10 codes as preprocessing. This makes our work applicable to ongoing clinical practice, as we are using ICD-10 codes in current electronic health records data. In addition, we also add expert domain knowledge graph learnt from ACME decision table as knowledge constraint to restrict the output from the pure data-driven framework. The overall structure is shown in \textbf{Figure \ref{fig:overall_structure}}.

\begin{figure*}[t!]
    \centering
    \includegraphics[scale=0.32]{figures/overall-structure.jpg}
    \caption{Overall Structure of this paper. The encoder-decoder model is the main framework for generating causal chain of death. Domain knowledge constraint is learnt from ACME decision table.}
    \label{fig:overall_structure}
\end{figure*}

In summary, our work has the following contributions:
\begin{enumerate}
    \item We are the first to develop encoder-decoder data-driven approaches for suggesting causal chains of death based on death reports and decedents' last hospital visit discharge records;
    \item We apply the State-of-The-Art model for neural machine translation, and augment it with domain knowledge constraints;
    \item We demonstrate the effectiveness of validity check in improving the stability of encoder-decoder frameworks;
    \item We are the first one to interpret deep learning results through visualization by identifying meaningful association between clinical conditions coded in different versions;
    \item We are the first one to use BLEU score to evaluate the performance of deep learning model on generating causal chain of death;
    \item We implement the knowledge-guided deep model on the FHIR interface.
\end{enumerate}
\section*{Conflict of Interest Statement}

\section*{Acknowledgment}
The author would like to thank Paula Braun (CDC) for her invaluable assistance and support in shaping this project. This article does not reflect the official policy or opinions of the CDC, NIH or NSF and does not constitute an endorsement of the individuals or their programs.

\ifCLASSOPTIONcaptionsoff
  \newpage
\fi

\bibliographystyle{IEEEtran}

\section{Methodology}

\subsection{Neural Machine Translation: Encoder and Decoder}

Translation is to find a target sentence $\mathbf{y} = y_1, \dots, y_n$ which maximizes the conditional probability $p(\mathbf{y}|\mathbf{x})$ given a source sentence $\mathbf{x} = x_1, \dots, x_m$. Neural machine translation (NMT) aims to maximize this conditional probability of source-target sentence pairs by using a parallel training corpus to fit a parameterized model. As shown in \textbf{Figure. \ref{fig:NMT}}, there are two basic components of an NMT system:
\begin{itemize}
    \item An encoder encodes the input sequence $\mathbf{x}$ into a representation $\mathbf{s}$
    \item A decoder generates the output sequence $\mathbf{y}$
\end{itemize}

The conditional probability of the decoder is formulated as:
\begin{equation}
    \log p(\mathbf{y}|\mathbf{x}) = \sum_{t=1}^n \log p(y_t|y_1, y_2, \dots, y_{t-1}, \mathbf{s})
\end{equation}

The the probability of the next generated word $y_i$, is jointly decided by the learned representation $\mathbf{s}$ and all previously generated words $y_1, \dots ,y_{t-1}$.

\begin{figure}[h!]
    \centering
    \includegraphics[scale=0.25]{figures/NMT.jpg}
    \caption{Neural machine translation consists of an encoder (stacked recurrent networks in blue) and a decoder (stacked recurrent networks in red). The symbol $<eos>$ is a special token referring to the end of a sentence. Adapted from \cite{luong2016achieving}.}
    \label{fig:NMT}
\end{figure}

In this section, we would like to briefly introduce the following four models of encoder-decoder framework in neural machine translation.

\subsubsection{LSTM Encoder - LSTM Decoder}
In an long short-term memory (LSTM) Encoder-Decoder framework \cite{cho2014learning,sutskever2014sequence}, the encoder reads and encodes an input sequence of embedded vectors $\mathbf{x}$. The encoder will then generate a hidden state $h_t$ at time $t$ from the current input $x_t$ and the previous hidden state $h_{t-1}$:
\begin{equation}
    h_t = f(x_t, h_{t-1})
\end{equation}

The representation vector $\mathbf{s}$ shall have the form:
\begin{equation}
    \mathbf{s} = q({h_1, ..., h_m})
\end{equation}

Here $f$ and $q$ are some non-linear functions. For the basic recurrent neural network RNN/LSTM model, the conditional probability of output sequence $\mathbf{y}$ at time $t$ can be written as:
\begin{equation}
    p(y_t|y_1, ..., y_{t-1}, \mathbf{s}) = g(y_{t-1}, h_t, \mathbf{s})
\end{equation}

Here $g$ is a (multi-layered) nonlinear function.

Generic RNN or LSTM encoder-decoder framework has to process the sentence word by word, failing to preserve long-term dependency. Luong et al \cite{luong2015effective} proposed global attention which predicts the position of alignment for the current word before computing the context vector using the window centered around that source position. Here the global attention is used in the decoder. The LSTM encoder-decoder model is easy to understand, and can be applied on most sequence-to-sequence tasks. Yet such a model has limited performance, especially on long sentences.

\subsubsection{Mean Encoder - LSTM Decoder}

Mean encoder is a simplified encoder. Instead of adding an LSTM model in the encoder, this mean encoder speeds up by using a mean pooling layer. The same stacked LSTM model and global attention mechanism are used in decoder. This mean-encoder model is much more efficient during training than any other models, but may not accurately capture the long-term dependency in the input sentences.

\subsubsection{Bidirectional RNN Encoder - LSTM Decoder}

A major disadvantage of the traditional encoder-decoder model is that the neural networks compress source sentences into fixed-length vectors. This may significantly limit the capability of translating long sentences \cite{cho2014properties}. Bahdanau proposed to use a bidirectional RNN \cite{bahdanau2014neural} with encoder-decoder approach so that the model can learn to align and translate jointly. A bi-directional RNN encoder model can better learn the embedding of words, but it's less efficient than the LSTM encoder-decoder framework, and typically has less accurate results than transformer models.

\begin{figure}[h!]
    \centering
    \includegraphics[scale=0.25]{figures/transformer.jpg}
    \caption{Overall structure of a transformer. Here we have five identical encoders and five identical decoders in this transformer.}
    \label{fig:transformer}
\end{figure}

\subsubsection{Transformer Model}
Still, neither RNN-based or CNN-based encoder-decoder models perform well on long sentences. To overcome this problem, Vaswani et al proposed the transformer framework \cite{vaswani2017attention} that enables encoding words of the same sentence in parallel. Together with self-attention, transformer boosts the speed of encoding.

As shown in \textbf{Figure. \ref{fig:transformer}} \footnotemark
, a transformer consists a stack of encoders and the same number of decoders. The embedded input is passed to the encoder at the bottom; the output from the encoder on the top will be passed to all decoders. The decoder on the top will pass the output to a linear layer and a softmax layer to generate predicted sentence.

Encoder has two layers: a multi-head self-attention layer and a feed forward layer (shown in part A of \textbf{Figure. \ref{fig:encoder-decoder-transformer}} \footnotemark[\value{footnote}]). Decoder has an extra multi-head attention layer that processes both the output from the encoder stack and the output from previous multi-head attention layer (shown in part B of \textbf{Figure. \ref{fig:encoder-decoder-transformer}} \footnotemark[\value{footnote}]).

The transformer model is more time-consuming than RNN-based encoder-decoder frameworks during the training stage, but can achieve far better results \cite{vaswani2017attention}. BERT (Bidirectional Encoder Representations from Transformers) \cite{Devlin2018bert} is a transformer encoder model that has been pre-trained on large datasets (BooksCorpus with 800M words and English Wikipedia with 2,500M words). The pre-trained BERT model can be further fine-tuned to improve performance on multiple nature language processing tasks.

\footnotetext{Adapted from https://towardsdatascience.com/transformers-141e32e69591}

\begin{figure}[h!]
    \centering
    \includegraphics[scale=0.22]{figures/transformer-encoder-decoder.jpg}
    \caption{Detailed structure of a transformer. \textbf{A} is the encoder of the transformer. It consists of two layers: the feed forward layer and multi-head attention layer. \textbf{B} is the decoder of the transformer. It consists of three layers: the feed forward layer and two multi-head attention layers.}
    \label{fig:encoder-decoder-transformer}
\end{figure}

\subsection{Decoding and Translation}

A straightforward method of decoding is to keep and predict only one word with the highest score based on previous steps. It is efficient and easy to understand; yet a small mistaken output might corrupt all remaining predictions. Thus, a better strategy named beam search keeps the top $k$ hypotheses for each step and select the best one when reaching the end of sequence. Here, $k$ is the beam size.

We also include medical domain knowledge as constraints during translation. The ACME (Automatic Classification of Medical Entry) decision table specifies all the ``feasible'' pairwise causal relationships between ICD diagnosis codes \cite{hoffman2018validity,lu2003using}. Using this decision table, we construct a domain knowledge graph on all diagnosis codes from Michigan data before training.
With diagnosis codes as nodes, we add directed paths between them only if such causal relationship can be found in the ACME decision table.
When decoding, the networks are required to look up the knowledge graph and are only allowed to include "feasible" codes in the top $k$ hypotheses.

\input{bleu}
\section{Causal Chain of Death}

\subsection{Data}
We are using last hospital visit discharge records from Michigan Vital Statistics Data that covers $181,137$ patients. As shown in \textbf{Figure. \ref{fig:michigan-data-screenshot}}, each patient has exactly one line of last hospital visit essential information, including up to $45$ clinical diagnosis codes, one underlying cause of death and up to $17$ related causes of death. On average, each patient has $18.84$ diagnosis codes and $2.25$ causes of death (including the underlying cause of death). In line with the ICD-9-CM Official Guidelines for Coding and Reporting \footnote{https://www.cdc.gov/nchs/data/icd/icd9cm\_guidelines\_2011.pdf}, the diagnosis codes are in priority-based sequence of ICD-9 codes. The causes of death are in ICD-10 codes. Typically, we shall have a longer input source sequence around 16-20 codes, and a much shorter output target sequence with roughly 2-3 codes. Such a short sequence of death codes is expected in death reports. We accessed the ten years' (2009 to 2018) National Center for Health Statistics (NCHS) Mortality Multiple Cause Files database \footnote{https://www.cdc.gov/nchs/data\_access/vitalstatsonline.htm\#Mortality\_Multiple} and calculated that the average length of death code sequence among $26,322,220$ decedent samples to be 2.95 codes. (Note that discharge codes on last hospital admission may contain previous admission discharge codes.)

\begin{figure*}[!]
    \centering
    \includegraphics[width=\textwidth]{figures/michigan-data.PNG}
    \caption{Sample data from the Michigan data set. The casual chain of death in ICD-10 for this patient is $I500>R688$ (Heart failure $>$ Other general symptoms and signs), outlined in green. This patient had a total of $30$ ICD-9 diagnostic codes assigned during the last visit to hospital, outlined in blue.}
    \label{fig:michigan-data-screenshot}
\end{figure*}

Ontology of medically valid causal relationships between ICD-10 codes were developed, improved, and promulgated by an international team of medical experts \cite{lu2003using}. This ACME (Automatic Classification of Medical Entry) decision table was used to learn the medical domain knowledge constraint \cite{hoffman2018validity}. It contains 95,321 lines of causal relationship. \textbf{Table \ref{tab:ontology}} illustrates an example of this decision table. Specifically, if rules are of length 2 it can be interpreted as F2 $\rightarrow$ F3, underlying cause of death leading to immediate cause of death; if rules are of length 3, it can be represented as (F1:F2) $\rightarrow$ F3, a subset of underlying cause of death leading to the immediate cause of death.

\begin{table}[h!]
    \centering
    \caption{An Example of ACME Decision Table.If rules are of length 2 it can be interpreted as F2 $\rightarrow$ F3, underlying cause of death leading to immediate cause of death; if rules are of length 3, it can be represented as (F1:F2) $\rightarrow$ F3, a subset of underlying cause of death leading to the immediate cause of death.}
    \begin{tabular}{|c|c|c|}
        \hline
        F3 & F2 & F1 \\
        \hline
        D460 & C000 & C97 \\
        \hline
        D460 & D460 & \\
        \hline
        D460 & Y400 & Y599 \\
        \hline
        D461 & C000 & C97\\
        \hline
        D461 & D461 & \\
        \hline
        D461 & Y400 & Y599 \\
        \hline
        D461 & Y880 & \\
        \hline
    \end{tabular}
    \label{tab:ontology}
\end{table}

\subsection{Generating Causal Chains Through Translation}
Mathematically, we can define the generation of causal chains as follows:

\theoremstyle{definition}
\begin{definition}{[Generation of Causal Chains]}
Given a deceased's medical history represented as a collection of medical codes $\mathbf{x} = x_1, \dots, x_m$, the goal of causal chain generation is to identify another list of medical codes $\mathbf{y} = y_1, \dots, y_n$ that summarizes the conditions leading to the death.
\end{definition}

The objective is to propose the causal chain of death, which is an ordered sequence of death codes (ICD-10). The inputs, on the other hand, are sequences of diagnosis codes (ICD-9). To generate one sequence of one domain (language) from a sequence of another domain (language), we apply the state-of-the-art algorithms from neurall machine translation.

Diagnosis codes in ICD-9 are defined as source codes (input sequence) while death codes in ICD-10 are target codes (output sequence). Both source codes and target codes are split into training set, validation set and testing set according to the ratio of $7:1:2$. We also applied 5-fold cross validation. Each line of codes is treated as one sequence, with each code in the sequence treated as one word in the natural language settings.
\section{Recent Work}

Intelligent death reporting has been a rising research field in recent years. Jiang et al. \cite{jiang2017causes} applied topic modeling on the multiple causes-of-death US mortality data from NCHS between 1999 and 2014. The author successfully group morbidities based on their correlation and explore the temporal evolution of these morbidity groups. Yet there is one major limitation with this work: due to the nature of unsupervised learning, the author failed to determine the optimal number of topic groups, negating its impact on clinical practice. Wu and Wang \cite{wu2017infer} designed a convolutional neural network (CNN) with dynamic computation graph to infer the underlying cause of death using the same NCHS mortality data. Using a list of relevant medical conditions, the proposed CNN model was able to achieve 75\% accuracy in predicting the single underlying cause of death. Besides the high accuracy, the proposed method further assist physicians in death reporting by providing human understandable interpretation for the model output. Meanwhile, Hoffman et al. \cite{hoffman2018validity} revealed the poor quality of death reporting data by showing 20.1\% discordance of cause of death. The author also proposed validity check on death reporting data to remove invalid causal pairs of death codes. One limitation is that, the author didn't apply validity check on any downstream tasks, such as predicting the single underlying cause of death, to further demonstrate the value of validity check.

\begin{figure*}[t!]
    \centering
    \includegraphics[scale=0.45]{figures/our-work-their-work.png}
    \caption{The top row show the pipeline of our work and the bottom row show the pipeline of Falissard's work.}
    \label{fig:pipeline}
\end{figure*}

A recent work of similar topic \cite{falissard2020neural} was submitted in late March, 2020 and \textbf{Figure \ref{fig:pipeline}} shows the difference in pipeline of our work and their work. Our work has five key differences from theirs:
\begin{enumerate}
    \item Different clinical tasks: we aim to automatically generate the causal chain of death in ICD-10 codes given the discharge data of the decedent's last hospital visit, coded in ICD-9. In comparison, their work is to recognize and convert medical entities in natural language (French) to ICD-10 codes. The medical entities in French are already filled as causes of death on the death certificate by medical experts.
    \item Different input data formats: our input data are priority-based ICD-9 codes from the discharge data of the decedent's last hospital visit, while their input data are medical entities of causes of death in French on the death certificates.
    \item Different methods: in addition to neural machine translation model named transformer, we also adapt and apply three recurrent neural network based encode-decoder frameworks to generate the casual chain of death; furthermore, we test the more recent the cross-lingual language modeling (XLM) \cite{lample2019cross} and analyze why it fails on our task.
    \item Medical domain knowledge constraint: we learn medical domain knowledge as constraint from ACME decision table on the encoder-decoder frameworks.
    \item Different evaluation metrics: we use the modified BLEU score (1-gram and 2-gram precision), which is popular for sequence-to-sequence translation task in natural language processing. Their work uses precision, recall and f-measure for evaluation.
\end{enumerate}
\section{Results and Discussion}
\subsection{OpenNMT}

\begin{table*}[!]
    \centering
    \caption{Average BLEU scores and standard deviation in parentheses across five folds.}
    \begin{tabular}{|c|l|r|r|c|c|c|c|}
         \hline
         Experiment & Input Data & Validity Check & Knowledge Constraint & LSTM & Mean & BRNN & Transformer \\
         \hline
         1 & ICD-9 & Not checked & Non-constrained & 15.46 (0.71) & 14.81 (0.48) & \textbf{16.04} (0.74) & 14.37 (0.55) \\
         \hline
         2 & ICD-9 & Checked & Non-constrained & 15.58 (0.48) & 14.11 (0.34) & 15.95 (0.44) & 14.38 (0.30) \\
         \hline
         \hline
         3 & ICD-9 & Not checked & Constrained & 12.61 (6.33) & 15.56 (0.72) & 12.46 (6.26) & 14.76 (0.51) \\
         \hline
         4 & ICD-9 & Checked & Constrained & 12.95 (6.50) & 15.15 (0.48) & 13.16 (6.61) & 14.99 (0.42) \\
         \hline
         \hline
         5 & ICD-10 & Not checked & Non-constrained & 15.47 (0.71) & 14.93 (0.46) & 15.72 (0.66) & 14.40 (0.63) \\
         \hline
    \end{tabular}
    \label{tab:openNMT-results-table}
\end{table*}

As shown in \textbf{Table \ref{tab:openNMT-results-table}}, we calculate the average BLEU score and its standard deviation in parenthesis for each encoder-decoder framework across five folds. For \textbf{Experiment 1} which is without validation check in training data and knowledge constraint in decoding, the bi-directional RNN encoder-decoder model achieved the highest average BLEU score, followed by the LSTM encoder-decoder model, the mean encoder-decoder model and the transformer model. Comparing \textbf{Experiment 1} and \textbf{Experiment 2}, it is clear that validity check, the preprocessing step on training and validation data, can help reduce the standard deviation of the BLEU score. The average BLEU score between \textbf{Experiment 1} and \textbf{Experiment 2}, however, is very close for LSTM (difference=-0.12), BRNN (difference=0.09) and transformer (difference=-0.01) models. This indicates that validity check can help improve stability of the trained models but cannot improve the average performance.

It is worth noticing that in \textbf{Experiment 3} and \textbf{Experiment 4}, the average BLEU score drops significantly for LSTM and BRNN models, while their standard deviation increases significantly. As for the mean encoder and transformer models, knowledge constraint slightly increases both the average BLEU score and its standard deviation. In other words, applying knowledge constraint has a mixed impact on the encoder-decoder models: it may significantly decrease the performance and stability on LSTM and BRNN models, while slightly improve the performance on the other two models. Consequently, we argue that the encoder-decoder frameworks can learn the causal relationship between diagnosis codes well enough that it is not necessary to learn and incorporate the medical domain knowledge constraint from the ACME decision table

In addition, it is interesting to compare the results in \textbf{Experiment 1} and \textbf{Experiment 5}. After mapping the input ICD-9 codes into ICD-10 codes, LSTM, mean encoder and transformer models have similar average BLEU scores with those in Experiment 1. The average BLEU score of BRNN model drops by 0.32, still relatively small (less than 2\% difference). The standard deviation for all models are very close between two experiments. These results are significant: 1) the encoder-decoder frameworks are promising and stable in generating the causal chain of death, no matter whether we have input and output data in the same or different coding systems. 2) When having no access or limited access to the newest electronic health records (EHRs) data, we can use data before 2015 to train the models and generate the causal chain of death.

According to \cite{luong2016achieving}, larger vocabulary size tends to achieve higher BLEU score. Their proposed hybrid NMT model achieved $17.7$ BLEU score with $10k$ vocabulary size on English-Czech translation task. Our vocabulary size in source set is $7616$ and that in target set is $2649$. Thus, our translation performance is close to the state-of-the-art.

\begin{table*}[!]
    \centering
    \caption{Average accuracy for generating the entire output sequence correctly and its standard deviation in parentheses across five folds.}
    \begin{tabular}{|l|c|c|c|c|}
         \hline
         Validity Check & LSTM & Mean & BRNN & Transformer \\
         \hline
         Not checked & 0.125 (0.00672) & 0.118 (0.00459) & 0.130 (0.00677) & 0.113 (0.00495) \\
         \hline
         Checked & 0.139 (0.00552) & 0.122 (0.00389) & 0.140 (0.00503) & 0.123 (0.00324) \\
         \hline
    \end{tabular}
    \label{tab:entire-sequence-correct-results-table}
\end{table*}

\begin{table*}[!]
    \centering
    \caption{Average accuracy for individual code matching and its standard deviation in parentheses across five folds.}
    \begin{tabular}{|l|c|c|c|c|}
         \hline
         Validity Check & LSTM & Mean & BRNN & Transformer \\
         \hline
         Not checked & 0.749 (0.00111) & 0.742 (0.00167) & 0.751 (0.00219) & 0.730 (0.00303) \\
         \hline
         Checked & 0.764 (0.00339) & 0.742 (0.00145) & 0.765 (0.00264) & 0.742 (0.00210) \\
         \hline
    \end{tabular}
    \label{tab:single-code-correct-results-table}
\end{table*}

\begin{table*}[!]
    \centering
    \caption{Average accuracy for generating the underlying cause of death correctly and its standard deviation in parentheses across five folds.}
    \begin{tabular}{|l|c|c|c|c|}
         \hline
         Validity Check & LSTM & Mean & BRNN & Transformer \\
         \hline
         Not checked & 0.516 (0.00809) & 0.495 (0.00441) & 0.521 (0.0105) & 0.494 (0.00474) \\
         \hline
         Checked & 0.531 (0.00586) & 0.506 (0.00395) & 0.533 (0.00415) & 0.510 (0.00531) \\
         \hline
    \end{tabular}
    \label{tab:underlying-correct-results-table}
\end{table*}

Furthermore, we also include the results for the other three evaluation criteria. As shown in \textbf{Table \ref{tab:entire-sequence-correct-results-table}}, BRNN model achieves the best result in correctly generating the entire sequence with or without validity check. Notice that when evaluating a single pair of generated sequence and the target sequence, the output of this criteria is binary: 0 or 1. If any individual code fails to match, or the order of any code is incorrect, we will return a "0" for that pair of sequences. Thus, 0.140 indicates that BRNN model predict 14\% of sequences correctly.

As shown in \textbf{Table \ref{tab:single-code-correct-results-table}}, all four models achieve between 73\% and 76.5\% accuracy in predicting individual codes correctly. Comparing with the results in \textbf{Table \ref{tab:entire-sequence-correct-results-table}}, we argue that the encoder-decoders frameworks are much more accurate in finding the right codes on cause of death, than figuring out the right order between/among these codes.

Lastly, we show the accuracy in generating the underlying cause of death in \textbf{Table \ref{tab:underlying-correct-results-table}}. BRNN and LSTM models have better results than the other two, achieving over 53\% accuracy with validity check. Even though this is worse than the previous work using convolutional neural network (75\% accuracy), but the main objective of the previous work was to predict the underlying cause of death, which is obvious different from ours. Still, we would definitely try to improve the accuracy on underlying cause of death in future work.

CNN-based encoder-decoder framework frequently reported errors during the experiments. Conceptually, RNN-based encoder-decoder frameworks with attention mechanism are more suitable on sequence-to-sequence tasks; CNN-based model, on the other hand, finds it challenging to deal with sequences of different lengths. Consequently, we do not include the inconsistent results from CNN-based encoder-decoder framework.

\subsection{Visualization}

In addition, we would like to show the visualization of attention during translation. As shown in \textbf{Figure. \ref{fig:attention}}, the source sentence is on the top of the graph while the predicted sentence is on the left. In this example, the code $C349$ (malignant neoplasm of unspecified part of bronchus or lung) is the correct prediction. We can observe that the code $1890$ (malignant neoplasm of kidney, except pelvis) from the source code has high attention with the predicted code. Yet the non-negligible flaw is that the source code $2761$ (hyposmolality and/or hyponatremia) has wrong attention with the $<EOS>$ symbol in the prediction.

\begin{figure} [h!]
    \centering
    \includegraphics[scale=0.45]{figures/attention.jpg}
    \caption{Visualization of attention in translation. The source sentence is on the top of the graph and the prediction is on the left. The special symbol $<EOS>$ is the end of sequence. The ICD-10 code $C349$ in human readable is malignant neoplasm of unspecified part of bronchus or lung; the ICD-9 code $1890$ is malignant neoplasm of kidney, except pelvis; the ICD-9 code $2761$ is hyposmolality and/or hyponatremia.}
    \label{fig:attention}
\end{figure}

\subsection{XLM}

To our surprise, the state-of-the-art algorithm XLM (cross-lingual language model) performs much worse than the other encoder-decoder frameworks. All BLEU scores are less than 1 after trying different combinations of hyper-parameters. Even though the pre-training can successfully finish after 500-700 epochs on monolingual corpora using masked language model perplexities as evaluation metric, the training on parallel data using BLEU score as evaluation metric fails to converge properly.

We argue that the core algorithm behind BERT and XLM, masked language model, does not work on our data set. The idea of masked language modeling is to randomly mask (hide, making it unknown) a few words in the sentence (either source or target sentence) during the training stage and then to recover these masked words based on surrounding context. Since in average our target sentence has $2.25$ words, masking one word can make it extremely difficult to recover. Even worse, over $31\%$ of our target sentences consist of only one word: masking the only word makes it impossible to recover.

redpajama_set_name: RedPajamaArXiv


text:
Multiple Defendants Plead Guilty in Separate Auto Insurance Fraud Cases
Added by Debra Tash on March 15, 2019.
Saved under County, News, Press Releases
Tags: Auto Insurance Fraud, County of Ventura, fraud, Insurance Fraud
VENTURA, California – District Attorney Gregory D. Totten announced today that multiple defendants pled guilty on March 14, 2019, in separate auto insurance fraud cases prosecuted by the Auto Insurance Fraud Unit of the Ventura County District Attorney's Office.
Raymond Merkley (DOB 10/22/78), of Woodland Hills, pled guilty to presenting a fraudulent auto insurance claim and admitted previously sustaining two prison priors. In November 2016, while illegally off-roading in Hungry Valley State Park, Merkley's truck suffered significant damages. Afterwards, Merkley repeatedly lied to his insurance carrier about when and how the damages occurred. Merkley will be sentenced on May 29, 2019, in the Ventura Superior Court, County of Ventura. Due to his prior convictions for which he served prison time, Merkley faces up to 10 years in custody in this case.
Manuel Abrego (DOB 3/1/87), of Thousand Oaks, and Felix Hernandez (DOB 10/16/78), of Oxnard, pled guilty to arson and auto insurance fraud. On March 2, 2017, firefighters from the Oxnard Fire Department were called to a suspicious car fire. It was later determined that the car belonged to Abrego and that he previously paid Hernandez to burn the car to obtain insurance money. Abrego and Hernandez will be sentenced on April 25, 2019, in the Ventura Superior Court, County of Ventura. Both men face up to 5 years 8 months in prison for their respective roles in the fraudulent scheme. At the time of the plea, Abrego paid full restitution of $14,846 to State Farm Insurance, $1,193 to the Oxnard Fire Department, and $660 to Toyota Motor Corporation.
Kellina Pelletier (DOB 3/7/78), of Ventura, pled guilty to two counts of auto insurance fraud stemming from an Easter-day car accident on April 16, 2017. Pelletier will be sentenced on April 25, 2019, in the Ventura Superior Court, County of Ventura. At the time of her plea, Pelletier paid $769 in full restitution to AAA Insurance.
The Ventura County District Attorney's Office is the public prosecutor for the county's 850,000 residents. The office employs approximately 280 employees including attorneys, investigators, victim advocates, and other professional support staff who strive to seek justice, ensure public safety, and protect the rights of crime victims.
Follow the Ventura County District Attorney's Office on Twitter @VenturaDAOffice

redpajama_set_name: RedPajamaCommonCrawl


text:
'Power to the People' rally draws dueling factions to Northbrook intersection
Glenbrook North junior Peter Christos organized a pro-Trump rally, which was met by a counter-demonstration, in downtown Northbrook Friday. Joe Lewnard | Staff Photographer
Counterdemonstrators hold up signs and stand on the opposite sides of Shermer Road in response to a pro-Trump rally, organized by Glenbrook North junior Peter Christos in downtown Northbrook Friday. Joe Lewnard | Staff Photographer
A pro-Trump rally, organized by Glenbrook North junior Peter Christos, is met with a counter demonstration in downtown Northbrook Friday. Joe Lewnard | Staff Photographer
Counterdemonstrators hold up signs and stand on the opposite side of Shermer Road in response to a pro-Trump rally, organized by Glenbrook North junior Peter Christos in downtown Northbrook Friday. Joe Lewnard | Staff Photographer
People participate in a pro-Trump rally, organized by Glenbrook North junior Peter Christos in downtown Northbrook Friday. Joe Lewnard | Staff Photographer
Glenbrook North junior Peter Christos, center, with megaphone, organized a pro-Trump rally, which was met by a counter-demonstration, in downtown Northbrook Friday. Joe Lewnard | Staff Photographer
By Dave Oberhelman
doberhelman@dailyherald.com
Updated 10/2/2020 4:52 PM
Those who believe America is at a crossroads found a literal example Sept. 25 in Northbrook.
On that evening Glenbrook North junior Peter Christos led the Power to the People rally at Shermer Road and Walters Avenue. His faction assembled on the east side of Shermer -- cater-cornered from the COVID-19 "Death Scoreboard" Peaceful Communities organizer Lee Goodman installed Sept. 18 in Triangle Park, which inspired Christos to hold his demonstration.
"It's brought to our attention that our community's really divided right now politically, racially and in support of our law enforcement. So we're just out here today exercising our First Amendment rights in support of our law enforcement, for the reopening of our schools and in support of President Donald Trump," said Christos, a moderator for the Glenbrook North Conservatives Instagram page.
"I personally invited Black Lives Matter and Antifa to protest right across the street because I think it's very important to see both sides of the argument," said Christos, who had protested the COVID sign shortly after its installation and had a run-in with a man who left his vehicle to approach him, video showed.
"We had a lot of people actually un-RSVP from this event because of what happened last week," Christos said, "which is sad that people have to hide their identities or be afraid of being fired from their jobs just because they support our country and our president. I don't think that's OK, so we're out here showing everybody that it's OK to be a Trump supporter and it's OK to love our country."
Starting with a pair of high school-aged girls a counter group built in numbers on the west side of Shermer Road, resembling Christos' contingent but with opposing beliefs. Catcalls lobbed back and forth across the street merged with honking car horns.
"I've been involved in politics ever since the Trump election. I think it woke up a lot of youths' minds to want to be involved in politics," said Christos, holding a dark blue "Trump" banner.
Periodically Christos and several others among his group crossed Shermer Road for civil discussions with the counterprotesters who similarly ranged from teenagers to septuagenarians.
No arrests were made, police said, and the only fireworks early into the rally scheduled from 5-9 p.m., came when a woman exited her southbound car stopped at the light to sing "God Bless America," then returned to her car to yell at people before a pair of police officers on bicycles chased her off to keep traffic moving.
Unaffiliated observers stood at each corner of the intersection or came from their homes to watch the proceedings.
"They have every right to do it, that's why I'm just sitting here watching it," said resident John McCormack, joined by his wife, Jennifer. "There's nothing I can do, they have every right to do that. That's some of the beauty of our First Amendment, right, that we have the right to do that. More power to them."
He nonetheless took consolation that a June 8 march in Northbrook supporting Black Lives Matter drew "at least 1,000" people while "Power to the People" attracted decidedly fewer.
"That tells me everything I need to know," McCormack said.
Meanwhile, Goodman watched near the steps to the Metra station.
"I'm never going to criticize anyone for making a public statement," he said, noting he'd updated his COVID Death Scoreboard earlier Friday to indicate 203,000 who've succumbed to the virus.
"I'm not sure what their position is here," he said. "We put up a sign talking about the pandemic and the people who are responsible for it. I don't know what it is exactly they're trying to say. So long as they're peaceful I've got no problem with it. They're not necessarily that peaceful, though. The last time they were being very aggressive, not wearing any masks, that's why we're staying at a distance today."
Another Glenbrook North junior, Yubin Son, attempted to find a sliver of hope as he watched the dueling forces from in front of Triangle Park.
"I think it's fine to have your own opinions," he said, "but to see the division in our country and even in our town, I don't know, it's kind of disappointing. But I'm sure in the future we'll unite someday."
Glenbrook North High School
Glenbrook High School District 225
Northbrook Herald


To say that rebounding (and specifically ReboundSA) has changed my life and my business is an understatement. My studio literally got a new lease on life. My clients love the classes and the health benefits we have witnessed over the past couple of years have been astounding. I love teaching the classes, the clients line up (even during winter) and everyone leaves in a great mood. It really is the best and most fun form of exercise.
Dealing with Keith and Justine has always been a pleasure. They are mines of information and no one knows their product better than they do. They are so helpful and so passionate and nothing is ever too much trouble. I am looking forward to working with them in the near future on some new and exciting things!!!
I have been using and recommending ReboundSA rebounders for many years and I feel that this is the ultimate exercise as it strengthens every part of the body, and in particular the immune system via the lympathic massage effect. Their products are the finest on the market and give the correct bounce and last very well. I have been using my rebounder for many years and always recommend their range to my patients.
I have been dealing with the Rebound SA team for many years and only had the very best of services! They have such a positive and passionate outlook on health, vitality and fitness and are a true inspiration to everyone. Many of my clients rebound for weight loss, and it is most effective, but I have also witnessed so many health benefits over the years, one being a client of mine who suffered for a year from a spider bite, which turned his leg black – amazingly as soon as he started rebounder his leg started to heal and became totally healthy again.


So just after my bit of DIY, I headed north up the St Annes to stay with Martin and Stephanie, friends from University. We headed over to Headingley in Leeds to a Pizza restaurant we used to frequent as students, where we met up with our other two house-mates from those days (Jon and Elaine, and Tom and Carol) The photo shows the four of us outside Salvos.
In a burst of nostalgia, we headed off to our former home photographed here when we actually lived there! It hadn't changed much, but we posed outside for another photo! Martin, Stephanie and I headed back to St Annes in the afternoon. Next day, after a walk along the Blackpool Promenade, Martin and I went sailing on the Ribble estuary before I headed back home, via my parents, after a very enjoyable weekend.
Moving on from cake, I have also been trying my hand at bread making. I do have a bread maker, which I use for making the dough, but I have found that turning out the dough, knocking it back and proving it outside the machine, before baking it a conventional oven, gives best results. as I hope the picture shows!
I have also been experimenting with flavoured savoury breads. So far olive (with Mozzarella and oregano) and sun-dried tomato (with Parmesan and basil) The tomato needs more practice, but they were made for when David and Frances (and their daughter) came to visit two weeks ago. The sun smiled on us and we ate outside in the garden. You may remember that I stayed with David and Frances for a week while I was homeless last December!
The sun has been welcome in the garden, where the runner beans have reached the top of their poles and are producing flowers. The pots have also looked good, and this photo shows the two hanging baskets. There are some more garden photographs in the gallery!.
On the health front, I had a visit to the eye clinic at St Thomas's about a month ago. I still have dry eyes, but I have a new medication for night use which is certainly soothing, and I hope will eventually solve the problem. I am expecting a referral to the cataract clinic, as these are slowly getting worse. However the visit to London was an opportunity to see my youngest daughter, Rachael, who is doing a three month internship during her summer vacation. We had a very good evening at a restaurant in Borough Market!
I had also been planning to go diving, but the last two planned dives were cancelled because of bad weather, and the expedition to Devon has had to be cancelled for other reasons. That was very disappointing as I was looking forward to doing some camping (no accommodation) but I hope that we will get a couple of dives in the local area next week.
So a busy few weeks! There will be a few stressful moments in the next few weeks as the divorce rumbles on, but I need say no more about that at the moment.
Yesterday saw me getting the train up to Kings College Hospital for my four monthly haematology check up. This was the second time I have been to Kings since the Bone Marrow Transplant unit co-located there. There was a little more organisation, although blood samples (a regular armful!) are taken elsewhere in the hospital first.
I saw Dr Kazmi, the consultant, and we had a long chat. He confirmed that the after five years in remission, although there is always some risk of relapse, the statistcs take a sharp turn in my favour! He confirmed that my blood counts were good, and although my immunoglobulin levels are still slightly low, they are not that far outside the normal ranges. There were no physical abnormalities, so I am now on annual check-ups! We had a chat about the GvHd, and how good the ECP treatment was in improving my quality of life.
Good news! Next check up is at the eye clinic when I need to discuss getting my cataract operation done.
I have an old friend, Melinda, who is a consultant at King's, so we were able to meet up for lunch (thank you!) before hand, and catch up on things, and I also saw one of the nurses (Debbie) who looked after me during my various stays on Samaritan ward at Guy's! Small world!
I got home safely, ready go to sub-aqua club, when I received an e-mail about my dissertation result. Not completely good news as I didn't quite get the required marks, but not a complete failure as I can revise and re-submit. I am waiting for a list of things that I need to address for the re-submission. A bit disappointing, as it means I won't graduate until next year, but not a total disaster.
I'm afraid I haven't been in the right frame of mind for blogging in the last month or so. The background activities to the divorce have kept me busy too – I am amazed by the mountain of paperwork that it generates. The bad weather doesn't do much to lift the spirits either.
BUT – today the sun is shining, a court hearing on Tuesday went fairly well, and not as daunting as I feared, and there are a few things to report!
After my Birthday weekend, things settled down a bit, but the highlight of April was a week of diving down on the Lizard peninsula in Cornwall. The weather wasn't particularly kind to us (I was with my diving club) in that there were gales which gave a high sea state, but in the five days we were there, I did six dives. These were the first real and prolonged ocean dives I have done for nearly seven years, (discounting a 2 minute dive last year when my suit leaked) so it was quite an achievement. I was a little apprehensive, but once in the water I settled down and really enjoyed it. The training and practice sessions I had done in sheltered water had helped immensely, but it was great to be in the sea again. Another step in getting my life back.
Health wise, little seems to have changed. I had a clinic last week for one of the side effects of GvHd, but that has stabilised and I was discharged from it. The next major check up at the haematology department is next Wednesday, so another trip up to London and out to King's College Hospital. The only caiuse for concern is my eyesight and the cataract problem. My next visit to St Thomas's is approaching, and I think the time has come to ask about getting this sorted before it gets any worse.
The house is gradually getting into shape. My parents visited last weekend whichs good, and today I have been cutting the grass (I bought a lawn mower three weeks ago) and pricking out some germinating seeds, and setting some runner beans to germinate. I am fairly sure where I am going to put the beans, but I need to cut out some lawn (less to mow!) to make a bed for them.
So all things considered, not too bad, and justifying "Onwards and Upwards"!
Comments Off on Saturday… The sun shines!
Aug 30th 2011PeterGetting My Life Back!
Time has flown by, at the end of August already. Nothing much to report on the health front, all seems to be ticking along OK, with my next haematology check up in September. Much of my time has been taken up with my MSc dissertation, and helping out fellow blogmaster Mark with his new venture ttfntv. Go and have a look! I have also been playing with twitter, so I may end up having a twitter account linking to this blog. Maybe more next time, but the idea is to be able to press a button to tweet the content or to follow it. Still working twitter out!
I went diving again a few weeks ago – a much better experience and I think I have got my weighting about right. Still one or two kit problems (leak round the neck seal of my dry suit) but they are easily remedied.
So still busy and very much onwards and upwards!
Comments Off on Tuesday… end of August!
Jul 23rd 2011PeterGetting My Life Back!
Yes, still here, but a lot going on at the moment! However I had one of my infrequent check ups last week, at the eye department at St Thomas's. All seem to be OK, and there seems to have been an improvement in the dryness of my eyes, although they have not recovered sufficiently to let me resume wearing contact lenses. Slightly worrying is evidence of a small cataract on one eye, but the opinion is that it was caused by the past use of topical steroids, and therefore not likely to progress. That said, the radiotherapy and chemotherapy I have had in the past does increase the likelihood of developing cataracts in future, and is something to be aware of.
That apart, I have done a little more diving, including some practice dives at an inland site. Not the best of dives as I think I over-estimated my capabilities and ended up doing a faster than normal ascent, fortunately without coming to any harm, but I am planning to go back next week to do some more training to regain my skills.
Not much else to report at the moment. My dissertation is taking up quite a lot of time, as is job searching, but I do intend to continue blogging!
On the subject of blogs, it was great to hear from John's Mum. John, you may remember, was another cancer patient blogger, and he was inspirational to me. He also suffered from GvHd which ultimately contributed to his death three years ago, and it was his experiences that swayed my decision to undergo ECP. Not forgotten John!
The last few weeks have also seen the passage of a year since my friend Jane died after a long and courageous fight against bowel and secondary liver cancer, and Simon from lung and secondary brain cancer. All missed but not forgotten, and remain an important part of my life.
Apr 16th 2011PeterGetting My Life Back!
It has been a long gap since my last post, mainly because there has been a lot going on!
I started my contract in March but for two weeks I stayed in a holiday cottage not far from where I was working. It was set in lovely surroundings and I was fortunate enough to be there at the start of Spring with the emerging daffodils. It was lovely to be self contained rather than staying in a B&B.
Sadly the contract ended at the start of April, and while there is the possibility of more work later in the year, there is nothing much (apart from a little private client work) in the offing. However that has enabled me to start my MSc dissertation, which has slipped by a couple of months.
I have been doing a little plumbing too. The hot water cylinder started leaking a few weeks ago, not a lot, but enough to know that it wouldn't get better on its own! The tank is quite tall, but small diameter to fit in a confined space, so I had to find one the same size. That was no problem… except that the insulation standards have improved, so while the tank dimensions and capacities are the same, the external dimensions are bigger. The various connectors are in different places too. However it was all done, apart from the replacement of a tiny valve on the hot water outlet fitting which draws water for the shower from a lower level. I was fitting it – when it slipped and fell to the bottom of the tank! So it all had to be drained, removed and refitted! Much quicker second time round! The pictures show the top of the tank before and after. The fitting at the top (removed in the first picture) is where the little valve fitted! Next job is to box it in again!
I also had a day out in the dive boat last week Note the words carefully! It was a day out to test the boat and equipment, rather than a spectacular sight seeing dive (yes, there are some in The Solent!) The boat performed well, and we kitted up, me in my newly serviced and refurbished dry suit. Into the water, and after 30 seconds my buddy commented that his suit was leaking, so out we came. Twenty minutes later, back in the water with another buddy. We descended to the bottom (only about four metres – it was a check dive) when I noticed a wet and soggy feeling, so up we went after three minutes. It seems that the neck seal is too large for my neck, so the suit will have to go back to have that changed! Very irritating! However, I was supposed to be on an expedition this week, (it was postponed) so I would have been even more annoyed if I had gone away with it! What was pleasing though was that my upper body strength was sufficient to allow me to get out of the water into the boat (it is a RIB) with little difficulty.
So the week was spent on dissertation, and corporation tax returns – deep joy!
Last weekend we had a visitor, Melinda, who I have known for a long time, but haven't seem since my transplant five years ago. It was good to catch up on news and stuff.
And that leads me onto health, which seems OK! I have a dermatology check up on Tuesday, to see how I am getting on without ECP, and then towards the end of May I have a haematology check up, a lung function test, a bone density scan and a few other clinics monitoring the after effects of GvHd! I am glad that they are all on the same day! A busy day, but only one trip to London. And in between times, I have another trip to Lausanne booked.
The only other thing I need to get done is a little bit of outstanding dentistry (another crown).
Saturday… Yes, I am still here!
Mar 5th 2011PeterGetting My Life Back!
But not been in blogging mode I fear – partly because there has been a lot going on.
I returned from my break in Lausanne for a few days to get myself prepared to start work back at Corsham, where I was working last year. I had a trip up to London to see the opthalmic department at St Thomas's – the good news is that my eyes are getting slightly better, the bad is that I still can't wear contact lenses – very frustrating!
After some fairly swift contract stuff, I was able to go up to Corsham, for an induction (!) and get passes and so on sorted out, ready to start work on the 28th. But before then I had a dentist visit to have the root canal work (started in January) completed and a crown fitted. I have to say I was impressed. The crown was made in the surgery after being machined out of a block of ceramic material, using Computer Aided Design/Computer Aided Manufacture (CAD/CAM) techniques. Basically the tooth is prepared, photographed to produce a 3D image which is then adjusted by the dentist, digitised and sent to the machine.
After machining it is glazed (in the surgery) and fitted. One afternoon's work. Not cheap but very effective. There is a little more dental work to be done. One item is a crown I had fitted when I was 16 (following an unfortunate incident on a bike!). It has served me well, but as my gums have receded slightly (age and chemo) it probably needs replacing, so I think I shall have that done with the same technique.
The weekend was spent on coursework, and then I was off on Monday morning to start work. It felt strange living in a B&B again, but it was a busy week, getting back into the project, some visits and so on. At the moment it is only until the end of the month, but there is a possibility that it might be extended, and as yet I have heard nothing about the Belgian job that my CV has been submitted for. But it was good to be back at work, and in a brand new building!
So while it has been busy, and a while since I posted, I'm still very much around!
Jan 22nd 2011PeterGetting My Life Back!
No, not the sort that you eat with Nachos, but another major milestone in "getting my life back" – my first open water dive for over five years!
The dive was proposed last week at a sheltered site close to Portsmouth Harbour, and I spent some time getting my kit ready last night. It has been carefully stored, and I had checked it earlier, but it was still a little painstaking simply because I haven't had to prepare it for a long time.
It was very cold this morning – air temperature at about 1C, and the water at an estimated 4C, so it was drysuits and a good assortment of thermal clothing.
There were one or two kit problems, but I eventually jumped in, together with buddy Dave. It was very strange at first. In some respects it was like being a novice diver again, worrying about the techniques of buoyancy, and controlling breathing, and remembering where all the bits were, but after about 15 minutes, everything settled down and I started to relax as earlier training came back to me.
I was in the water for about 30 minutes, 27 of which were submerged. On exit, my hands were blue under the wet gloves, and I used a lot of air (a function of the cold water and my nervousness) even though the deepest we went was 6 metres but… it was great to be back in the water.
I still need to do some pool self training to refresh the drills, but I think if I can get back in under challenging (in terms of the cold) conditions, I should be OK to progress to deeper dives this summer.
And a big thank you to all at the dive club for their encouragement and patience!


text:
Divine Departures
Funerals as individual as you are...
| Divine Departures Home |
| Pre Paid Funeral Plans |
| Celebrations & Music |
| Funeral Services |
| Flowers |
UK Prepaid Funeral Plans Reviews
All of us know the importance of making a will. A will helps to ensure that all our possessions are given to those we love.
A pre-paid funeral is similar to a will and even though death is a very sensitive and painful subject to discuss, many people are turning to Pre- Paid funerals as an alternative to traditional funerals.
Many of us don't even like to talk about funerals, let alone plan one in advance. But Planning a funeral in advance has its own set of tangible and intangible benefits. A pre-paid funeral with funeral services UK allows people to plan out their funeral, just the way they want it to be or to mirror their lives.
A Pre-Paid funerals is a funeral purchased in advance with full consultancy from our expert funeral directors. Which means the style and type of the funeral requested is documented in the form of a written contract and all the payments are made in today's costs and prices.
This means no matter when the funeral may occur the client will not have to pay any additional costs for that contracted funeral.
By deciding to select, plan and pre-pay for the funeral, family and loved ones are relieved of the burdens and responsibilities of having to plan and pay for the funeral at a time when their hearts are filled with sorrow for the deceased.
What Does a Pre-Paid Funeral Plan Include?
The details on preparation of the body
Viewing arrangements
Type and Style of the Funeral Service
Type of Coffin/Casket
Burial or Cremation
Flowers and other floral décor
The specific cemetery/crematorium
Memorial books and cards etc
What are the benefits of a pre paid funeral?
You can plan your funeral just the way you want it to be, and have your wishes carried out.
Your family will not have to bear the costs of the funeral.
Your family and loved ones will not have the responsibilities of planning the funeral while having to bear the sorrow of the loss of a loved one.
Provides peace of mind for you and your family
The reassurance that your family won't have to cope with any uncertainty or worries about your funeral arrangements
Saving money by fixing your funeral directors costs at today's prices
Complete financial security – your money is held securely
Knowing that your funeral arrangements are taken care of can be a big weight off your mind.
You can relieve your family of the financial stress and any worries about your funeral arrangements and be assured that your wishes will be carried out exactly as you want.
More than 840,000 people in Great Britain have planned their funerals. Over the years, the cost of a funeral has gone up and up. While the average cost of a funeral in 2004 was £1,920, today it's £8,427. (*Source: SunLife 2014)
What's more, by 2018 the cost is likely to be as much as £14,000. A pre paid funeral Plan will save money by paying today's prices for a service you might not need for many years, you're also free to choose exactly the funeral you want, rather than a set plan.
You can plan your funeral to match your budget and preferences, and specify your particular requirements. You may want something simple, traditional or a something little more unusual the choice is yours.
At Divine Departures believe that a pre paid funeral plan offers you exceptional value and flexibility. You can pay in a single payment or spread the costs over up to 10 years. Acceptance is guaranteed, regardless of age or state of health.
We are happy to assist you in your choice, help compare or answer any questions you may have in a sensitive manner.
Recommended Pre-Paid Funeral Plans Video
Pre-Paid Funeral Plan Benefits:
Other Pages Related To This Topic
Divine Departures Funeral Plans
Our Prepaid Funeral Plan Comparisons
Your Essential Reviews To Golden Charter Funeral Plans
Your Guide To Liberty Pre Paid Funeral Plans
Comprehensive Co-operative Funeralcare Review
Our Age UK Pre-Paid Funeral Plan Review
Funeral Plans For People Under 50 Explained
+RETURN TO TOP OF PAGE+
| Terms Of Use | Privacy & Cookies | XML Sitemap |
COPYRIGHT © 2011 – 2022 - Divine Departures Funeral Plans


text:
Turkey to continue drilling activities in Mediterranean
Turkey, Economy
Ankara will not surrender to any threats against its activities in eastern Mediterranean, says Turkish energy minister
Gökçe Küçük | 11.06.2019
The drilling activities in Eastern Mediterranean Sea region will continue and legal and political measures were taken with the relevant ministries for preventing third party interference that ignores Turkey's international sovereignty, the Turkish energy and natural resources minister said on Tuesday.
Turkish Energy and Natural Resources Minister Fatih Donmez said in a statement that Turkey will not allow any violations of the rights of the Turkish Republic of Northern Cyprus (TRNC) and Cypriot Turks.
"We will continue our efforts to achieve regional peace by distributing the riches of Cyprus Island and the Mediterranean in a fair manner," Donmez said and added: "Our extensive and long-term exploration and drilling activities in the region will resume as planned without making concessions to our legitimate rights on license areas."
"The drillship Yavuz will also begin its operations in the region when the preparations are completed," the minister said.
Donmez said that the Greek Cypriot administration's attitude lacks legal ground. "Our activities in licensed and legal areas will continue without any delay despite the statements of the Greek Cypriots and the threats of so-called arrest warrants for our drilling ship Fatih's crew."
"Turkey never surrenders to any threats and it never will," Donmez added.
Turkey's rights in Eastern Mediterranean
The Turkish-flagged drillship Fatih launched its offshore drilling operations on May 3 in an area located 75 kilometers (42 nautical miles) off the western coast of the Cyprus island. The area falls entirely within the Turkish continental shelf registered with the UN and in permit licenses that the Turkish government in previous years granted to Turkish Petroleum, the country's national oil company.
Turkey wants to see energy as an incentive for political resolution on the island and peace in the wider Mediterranean basin, rather than a catalyst for further tensions.
On Monday, the Greek Cypriot administration, reportedly, issued arrest warrants for crew members of the drillship Fatih and officials of companies cooperating with the Turkish Petroleum Corporation (TPAO).
drillship Yavuz Eastern Mediterranean NaturalGas Turkish Energy and Natural Resources Minister Fatih Donmez


Heavy Rain is an action-adventure game developed by Quantic Dream and published by Sony .. "Quantic Dream's Heavy Rain: A PS3 Exclusive". Bloomberg Madison Paige - Pascal Langdale - Leon Ockenden - Sam Douglas. 16 Feb - 32 min - Uploaded by Pharmit24 Heavy Rain () ▻ Game Info The plot of the game four days to go through the four heroes. 19 Feb - 10 min - Uploaded by ★WishingTikal★ Heavy Rain (PS3) First Look/ Overview Part 1. ☆WishingTikal☆. Loading. Game. Heavy Rain.
25 May - 5 min - Uploaded by astrotriforce This is the trailer for the hit Heavy Rain for PS3. Do you like The game is very engaging. 23 Feb Metacritic Game Reviews, Heavy Rain for PlayStation 3, How Far Will You making it a game that you definitely should pick up for your PS3. What's your date of birth? We can only show this content to people who are old enough to view it. SUBMIT. FIND ANOTHER GAME. Triangle Circle Shapes.
15 Nov Heavy Rain Graphics Comparison PS3 vs PS4 Dealing with a range of adult themes, the game revolves around a sophisticated plot and. Heavy Rain is a hell of an experience. Reviewed on PS3 / 10 Feb the bulk of this text and heading straight to the review score to decide whether or not you should play through Heavy Rain, just know this: the game starts slow. Heavy Rain game guide contains full walkthrough of the game (in my opinion Dream; publisher: Sony Computer Entertainment; platform: PS3; rated: 18+. Hard to pin down, Heavy Rain is a cinematic and evolving thriller from Quantic Dream, the developer behind the critically acclaimed Fahrenheit on PlayStation 2 . 18 Feb Steve Boxer: Heavy Rain gets closer than any previous game to conveying the sense that you are controlling the protagonists in an interactive.