text:
\section{Introduction}
There is a rapid growth in the number of people living with Alzheimer's disease (AD)
\cite{alzheimer20182018}.
Clinical research has shown that quantifiable signs of cognitive decline associated with AD and mild cognitive impairment (MCI) are detectable
in spontaneous speech \cite{bucks2000analysis,sajjadi2012abnormalities}.
Machine learning (ML) models have proved to be successful in detecting AD using
speech and language variables, such as
syntactic and lexical complexity of language extracted from the transcripts of the speech~\cite{fraser2016linguistic, meilan2012acoustic, rentoumi2014features}.
Since transcripts
should  be accurate enough to properly represent
syntactic and linguistic characteristics, current approaches \cite{fraser2013,zhu2018detecting} frequently rely on 100\% accurate human-created transcripts  produced  by trained transcriptionists.
However in real-life speech-based applications of AD detection, ASR is used and it produces noisy, error-prone transcripts \cite{yousaf2019comprehensive}.
To our best knowledge, while the importance of well-performing ASR in speech classification has been studied in depth~\cite{zhou2016}, no prior research was done to understand what patterns of
speech are influenced the most by ASR errors such as word deletions and substitutions, and how this impacts performance of AD detection using ML models.

In this paper, we focus on this issue and study the \emph{effect of deletion, insertion and substitution errors on lexico-syntactic language features}
and their resulting \emph{effect on classification performance}.
The effect of these errors on binary AD-healthy classification performance is studied and suggestions are provided on how to improve ASR in order to maintain reasonable AD classification performance.

We identify that deletion errors affect the classification more than substitution and insertion errors on two datasets of spontaneous impaired speech. The effect of these deletion errors are most profound on features related to syntactic complexity and discourse representations in speech, such as production rules, word-level structure and repetitions. These features are also identified as being the most important for the classification task using a feature gradient-based importance metric.

\begin{comment}
\section{Previous Work}
AD detection is an important problem and multiple research studies focus on automated assessments for it \cite{fraser2016linguistic, thomas2005automatic, guinn2012language, zhou2016speech}. However, most of the studies utilize manual transcription \cite{fraser2016linguistic, thomas2005automatic, guinn2012language}.

Older voices, which form the major demographic of people with AD, have increased shimmer, jitter and breathiness in speaking \cite{vipperla2010ageing}. As a result,  accuracy of ASR systems on elderly voices tends to decrease with the age of the speaker \cite{vipperla2010ageing}. Additionally, the presence of cognitive impairments affecting language, such as AD, also causes the accuracy of ASR to decrease \cite{zhou2016speech, rudzicz2014speech}. This was analyzed in detail by \citet{zhou2016speech} by extracting various ASR transcript-based features. They found that for speakers with dementia WER varied between $93.54$\% and $38.24$\% depending on the dataset used for training the ASR system. Similar results for an ASR system on personal robots were reported by Rudzicz \emph{et al.} \cite{rudzicz2014speech}.

Subtle variations in lexical and syntactic features of language identify AD \cite{fraser2016linguistic}. For example,  a reduction in syntactic complexity in speakers with AD has often been reported in prior research \cite{fraser2016linguistic, croisile1996comparative, sajjadi2012abnormalities}. Fraser \emph{et al.} \cite{fraser2016linguistic} found that the AD group used fewer auxiliary verbs, fewer gerund verbs, fewer participles, and more sentence fragments. In the presence of ASR errors, these reductions could be exacerbated or diluted due to insertion, substitution or deletion of words belonging to a particular category.
Thus, it is important to differentiate between the effect of ASR errors and the effect of cognitive decline on the set of lexicosyntactic features.

We generate controlled amounts of insertion (\textit{ins}), deletion (\textit{del}) and substitution (\textit{sub}) errors to manually-transcribed speech, following the methodology of artificial error generation similar to Fraser \emph{et al} \cite{fraser2013automatic} to effectuate this.
Furthermore, we study which type of errors affect aggregations of features into interpretable groups such as discourse mapping, lexical richness, word finding difficulty and syntactic complexity most using a larger number of features. This relation is subsequently mapped to feature-importance and classification performance.
\end{comment}

\section{Data and Setup}
\subsection{Datasets}
\label{sec:datasets}

\textbf{DementiaBank (DB)}
The DementiaBank\footnote{https://dementia.talkbank.org} dataset is a large
dataset of pathological speech. It consists of narrative picture descriptions from participants aged between 45 to 90 \cite{becker1994natural}.
Out of the 210 participants in the study, 117 were diagnosed with AD (180 samples of speech) and 93 were healthy (HC, 229 samples).
Voice recordings and manual transcriptions (following CHAT protocol \cite{macwhinney2000childes}) are available for all samples. This dataset is used for the experiments in Section~\ref{sec:classification_performance}, \ref{sec:deletion_features}, and \ref{sec:model_gradient}.

\textbf{Healthy Aging (HA)}
The Healthy Aging dataset~\cite{balagopalan2018effect} consists of speech samples of 97 participants with no cognitive impairment diagnosis, all older than 50 years. Every participant describes a picture, analogous to the DB dataset.
The dataset constitutes 8.5 hours of audio with manual transcriptions. Each speech sample is associated with a score on the Montreal Cognitive Assessment (MoCA)~\cite{nasreddine2005montreal}.
Based on published cut-off scores~\cite{nasreddine2005montreal} for presence of MCI (minimum score for healthy participants is 26), we obtain class-labels for this dataset.

\subsection{ASR Setup}
The  Automatic  Speech  Recognition  (ASR) system we use for this work is based on the open-source  Kaldi  toolkit  \cite{povey2011kaldi}.
ASR uses ASPiRE chain model trained on multi-condition Fisher English corpus as a 3-gram language model.

\begin{table}[t]
\centering
\scriptsize
\begin{adjustbox}{max width=0.4\textwidth}
\begin{tabular}{ll|l|l|l}
\multicolumn{2}{c|}{\textbf{Dataset}} & \multicolumn{1}{c|}{\textbf{Del (\%)}} & \multicolumn{1}{c|}{\textbf{Ins (\%)}} & \multicolumn{1}{c}{\textbf{Sub (\%)}} \\ \hline \hline
\multirow{2}{*}{DB} & HC & 54.14 & 4.27 & 41.59 \\
 & AD & 56.98 & 3.89 & 39.13 \\ \hdashline[0.5pt/2pt]
\multirow{2}{*}{HA} & HC & 24.37 & 13.11 & 62.52 \\
 & MCI & 21.78 & 14.81 & 63.40 \\ \hline
\end{tabular}
\end{adjustbox}
\caption{\label{tab:transcript_performance} Rates of ASR errors on DB and HA datasets.}
\end{table}

Rates of ASR errors for healthy and impaired speakers for DB and HA datasets are in Table~\ref{tab:transcript_performance}. Majority of errors arise from deletions and substitutions for both datasets and groups.

\begin{comment}
\begin{table}[!htb]
\scriptsize{\caption{\label{tab:transcript_performance} ASR performance on DB and FP datasets
  }}
\setlength\tabcolsep{0pt}
\footnotesize\centering
\smallskip
\begin{tabular*}{\columnwidth}{p{0.2\columnwidth}p{0.1\columnwidth}|p{0.18\columnwidth}|p{0.18\columnwidth}|p{0.18\columnwidth}|p{0.18\columnwidth}}
\hline
      \textbf{\hspace{1.25em}Dataset} &  {} & \textbf{Del (\%)} & \textbf{Ins (\%)}&\textbf{Sub (\%)}&\textbf{WER (\%)}\\
      \hline \hline
      \hspace{1.5em}\multirow{2}{*}{\makecell{DB}} & HC &\hspace{1.5em}	52.54 &	\hspace{1.5em}4.15 &\hspace{1.5em}43.30 &	\hspace{1.3em}49.81\\
      & AD &\hspace{1.5em}	42.98 &	\hspace{1.5em}5.70 &\hspace{1.5em}51.31 & \hspace{1.3em}56.99\\
        \hdashline[0.5pt/2pt]
      \hspace{1.5em}\multirow{2}{*}{\makecell{FP}} & HC & \hspace{1.7em}34.16 &\hspace{1.35em}11.72 &\hspace{1.5em}54.12 & \hspace{1.3em}30.92 \\
      &AD& \hspace{1.7em}29.70 &\hspace{1.35em}12.14&\hspace{1.5em}58.16 & \hspace{1.3em}26.68 \\
       \bottomrule
    \end{tabular*}

\end{table}
\end{comment}

\section{Methodology}
\label{sec:method}

\subsection{Feature Extraction and Aggregation}
\label{sec:feature_aggregation}
Following previous studies \cite{fraser2016linguistic,balagopalan2018effect}, we automatically extract 507 lexico-syntactic and acoustic features. To simplify the presentation, the extracted features are aggregated into the following major groups:

\hspace{-1em}\textbf{Syntactic Complexity:} features to analyze  the  syntactic  complexity  of speech, such  as  number  of  occurrence  of  various  production  rules,  mean length of clause (in words) etc.

\hspace{-1em}\textbf{Lexical Complexity and Richness :} measures  of lexical  density  and  variation, such as average familiarity scores of all nouns, age of word acquisition,
frequency of
POS tags etc.

\hspace{-1em}\textbf{Discourse mapping:} features that help identify cohesion in speech using a \textit{speech graph}-based representation of message organization in speech \cite{mota2012speech}.
Examples of features include the number of edges  in  the  graph,  number  of  self-loops, cosine-distance across unique utterances etc.

Additionally, we extract features quantifying difficulty in finding the right words (e.g. filled pauses), measures related to description of content in the picture (e.g. number of content units), coherence in speaking at local and global level, and acoustic measures. such as MFCC and Zero Crossing Rate related voice representations (full list in App.\ref{app:feat})

\subsection{Error and Noise Addition}
\subsubsection{Artificial ASR Errors}
We introduce artificial ASR errors to understand if any specific error type influences the classification performance more than others. In previous research it was shown that lexical and syntactic groups of features extracted from transcripts of speech have different predictive power in dementia classification \cite{novikova2019}. As such, we hypothesize that different ASR error types may influence the features differently and would cause different effects on classification performance. The non-artificial output of ASR combines the errors of deletion, insertion and substitution in some proportion, thus not allowing analysis of the individual effects of each error type separately. This is why we generate each type of errors artificially.

\subsection{Error Addition Method}

We follow a method similar to the one used by ~\citet{fraser2013} to artificially add errors to manual transcripts at predefined 20\%, 40\% and 60\% WER rates. All altered words \textit{w}, where \textit{w} refers to a word in gold-standard manual transcripts, are selected at random. The following modifications are done: a) \textit{deletion} -  word instance \textit{w} is deleted, b) \textit{insertion} - new word \textit{$w_{1}$} is added after the word \textit{w}, c) \textit{substitution} - word \textit{w} is replaced with a new word \textit{$w_{1}$}.

For \textbf{\textit{deletion}} we simply delete random words from manual transcript at a specified rate.

To \textbf{\textit{substitute}} word \textit{w}, we select a unigram from 2,000 most used unigrams from Fisher language model that has the smallest Levenshtein distance with word \textit{w} based on the phonemic model from The Carnegie Mellon Pronouncing on Pronouncing Dictionary
\cite{weide1998cmu}. If word \textit{w} is not found in the Fisher language model a random unigram from the top 2,000 is used for substitution.

For \textbf{\textit{insertion}}, we select a word from the bigram list from the language model that has the highest probability to follow after word \textit{w} and insert it if it does not match the following word in transcript. In case of a match, the next most probable word is inserted. If word \textit{w} is not found in bigram list a random unigram is used for insertion.

To verify if simulated errors are a fair approximation of what is seen on a true ASR output, we have calculated the BLEU score~\cite{papineni2002bleu} between the manual and ASR-generated transcripts and compared them to the BLEU score between the manual transcripts and the transcripts with artificially simulated errors. The correlation between these two BLEU scores is strong and significant for both datasets (Spearman $\rho=0.72, p< 0.001$ for DB; $\rho=0.66, p<0.001$ for HA), i.e. transcripts with simulated errors are corrupted with respect to the manual transcripts in a similar manner as the ASR-generated transcripts are.

\subsubsection{Noise Addition}
We perturb all lexico-syntactic features or equivalently features that could be affected by ASR errors such as deletions, insertions, and/or substitutions, to mimic random sources of errors using Gaussian noise. We do this to compare and differentiate from the consequences of ASR errors. This modification is implemented by adding a randomized number to the extracted feature values where the mean of the number added to a given feature is zero and the standard deviation varies depending on the amount of noise we add (see App.\ref{app:noise} for details).

\subsection{Classification Setup}
\label{sec:model}

\textbf{Model:}
All our experiments are based on predictions obtained from a 2-hidden layer neural network (see App.\ref{app:clf} for details). We chose this model type and parameter-setting since it attained performance on-par with previously published results~\cite{fraser2016linguistic} with 10-fold cross-validation on gold-standard manual DB transcripts.

\section{Changes in Classification Performance Due to Simulated
Errors}
\label{sec:classification_performance}

We evaluate performance of classifying samples of speech to two classes - AD or healthy - using the DB dataset.

\begin{figure}[t!
\centering
    \includegraphics[width=0.8\linewidth]{images/db_performance.png}
    \caption{Effect of a controlled amount of ASR errors and random noise on classification performance.}
    \label{fig1:transcript_clf}
\end{figure}

\begin{table}[t]
\setlength\tabcolsep{0pt}
\footnotesize\centering
\smallskip
\begin{tabular*}{\columnwidth}{p{0.2\columnwidth}|p{0.2\columnwidth}|p{0.2\columnwidth}|p{0.2\columnwidth}|p{0.2\columnwidth}}
\hline
      \textbf{Transcript} &  \textbf{Accuracy} & \textbf{F1 (macro)}&\textbf{Sensitivity}&\textbf{Specificity}\\
      \hline \hline
      Manual & 80.20 & 79.76 & 79.29 & 80.83\\
      ASR-based & 74.96 &	73.97 &	76.01 &	74.36\\
      \hline

    \end{tabular*}
 \caption{\label{tab:transcript_asr} Effect of original ASR on classification performance with the DB dataset.}

\end{table}

Figure~\ref{fig1:transcript_clf} shows that deletion errors affect classification performance significantly more than insertion and substitution errors do. 40\% of deletions reduce F1 score by more than 10\%, while 40\% of insertions only result in 2.8\%, and 40\% of substitutions - in 6.3\% of F1 score reduction. These differences become even more pronounced with adding a bigger amount of errors.
Trajectory of F1 score with varying levels of noise is substantially different from that with varying deletion errors but not that with insertions or substitutions, showing that insertion and substitution errors influence classification performance in a way that is similar to a random noise. Deletion errors, however, have a significantly stronger effect on classification.  It is also interesting to note that the model utilizing automatic transcripts from ASR retains a level of performance at 74.96\% (Table~\ref{tab:transcript_asr}), which is comparable to the potential decrease in performance due to the rate of ASR deletion errors.

Different effects of errors on classification performance suggest that some features, extracted from the speech samples and used as an input for the classification algorithm, are affected far more substantially by deletions rather than any other type of errors. This leads us to inspect the correlation of feature values and the amount of deletions.
\vspace{-0.5em}

\section{Distinctive Effects of Deletion Errors}
\label{sec:deletion_features}

\begin{comment}
\begin{table}[t]
\centering
\scriptsize {\caption{List of features that correlate with the amount of deletion errors stronger than with the amount of insertion/substitution errors. Only the features with significant Spearman correlation (p$<$0.01) are listed. Bold denotes the features that are selected for both DB and FP datasets.}}
\begin{adjustbox}{max width=0.5\textwidth}
\begin{tabular}{c|l|c|c}
\hline
  \multicolumn{1}{l}{\textbf{Feature Group}} &
  \multicolumn{1}{c}{\textbf{Feature}} &

  \multicolumn{1}{l}{\textbf{Feature rank}}&

  \multicolumn{1}{l}{\textbf{Group rank}} \\ \hline \hline

  \multirow{13}{*}{\makecell{Syntactic Complexity\\and Discourse Phenomena}}
  & NP $\longrightarrow$ CD NNS & 1 & \multirow{13}{*}{1}  \\
  & \textbf{Graph self-loops with 3 edges} & 2 & \\
  & \textbf{PRT $\longrightarrow$ RP} & 3 & \\
  & VP $\longrightarrow$ VB PP & 4  & \\
  & VP $\longrightarrow$ VBZ S & 5 & \\
  & VP $\longrightarrow$ VBD SBAR & 6 & \\
  & WHNP $\longrightarrow$ WP & 8 & \\
  & \textbf{VP $\longrightarrow$ VBZ} & 9 & \\
  & S $\longrightarrow$ NP ADVP VP & 10 & \\
  & VP $\longrightarrow$ VB S & 11 & \\
  & ADVP $\longrightarrow$ RB RB & 12 & \\
  & VP $\longrightarrow$ VBG NP & 13 & \\
  & VP $\longrightarrow$ VBD VP & 14 & \\
  & \textbf{VP $\longrightarrow$ VBP SBAR} & 15 & \\
  \midrule
  \multirow{3}{*}{Lexical Richness} & Category `Particles' &
7 & \multirow{3}{*}{2}  \\
& POS `Verb Past Tense' & 16 & \\
& POS `Wh-pronoun' & 17 & \\
\bottomrule
\end{tabular}
\end{adjustbox}
\end{table}
\label{tab:correlation}

\end{comment}

In order to understand why deletions errors influence the classification performance significantly more than other error types, we identify features maintaining higher correlation with the amount of deletions than that with the amount of insertions and substitutions.
We observe 18 features in total that distinctively correlate with deletions. Out of these, the absolute majority of 15 features (83.33\% of all selected) are associated with syntactic complexity (production rules of a constituency parser) and discourse phenomena (graph self-loop with 3 edges) and 3 (16.7\%) - with lexical richness in speech. Other feature groups, such as acoustic features or those associated with word finding difficulty, do not meet the required conditions.
Such results show that syntactic structure of language is much more vulnerable to deletions than to other ASR errors. This can be explained by the fact that insertions and substitutions use words from the language model (i.e. most probable words) for the modifications, which to some extent helps maintain basic syntactic rules and structure.

Correlation between the number of deletions and features of syntactic structure shows the vulnerability of the feature group representing syntactic complexity and discourse phenomena to ASR deletion errors. However, it does not explain a decrease in classification performance when adding deletion errors. In Section~\ref{sec:model_gradient} we inspect if features of syntactic complexity are more influential in AD detection than other characteristics of speech.
\vspace{-0.5em}

\section{Model-based Analysis of Feature Importance}
\label{sec:model_gradient}

In order to quantify the importance of input features for classification, we obtain the gradient of the output prediction loss with respect to input features on a manually-transcribed version of the DB dataset.

We define gradient-based importance for feature $k$ for an input, $X_{i,j}$, in the training set for a classification model as:
\begin{equation}
    imp^{i,j,k} = \frac{\partial L(y_{i,j}, p_{i, j})}{\partial X_{i,j,k}}
\end{equation}
\vspace{-0.6em}

where $L$ denotes the loss criterion (binary cross-entropy loss), $y_{i,j}$ is the ground-truth label, $p_{i, j} \subset  [0,1]$ is the prediction probability; $p_{i, j} > 0.5$ denotes an AD prediction, $k$ is a given feature (1 to $D$ ), and $i$ is a number of samples (1 to $N_{j}$) in the training set in fold $j$ of the DB dataset classification setup.
Hence, to obtain the average importance for feature $k$ in a single fold, we compute:
\begin{equation}
    imp^{j, k} = 1/N_{j}\sum_{i=1}^{N_{j}}\frac{\partial L(y_{i,j}, p_{i, j})}{\partial X_{i,j,k}}
\end{equation}
This importance is then averaged across the 10-folds to obtain the final importance, i.e.:
\begin{equation}
    imp^{k} = 1/10\sum_{j=1}^{10}imp^{j, k}
\end{equation}

In order to interpret high-level patterns of input importance, we aggregate the feature importances into the groups defined in Section~\ref{sec:feature_aggregation}, where aggregation of importances involves averaging the absolute gradient-importance, $|imp^{k}|$, of features belonging to that group.

\begin{comment}
\begin{figure}[h]
\centering
\begin{subfigure}
  \centering
  \includegraphics[width=0.53\columnwidth]{images/hc_component2.png}
\end{subfigure}%
\begin{subfigure}
  \centering
  \includegraphics[width=0.44\columnwidth]{images/ad_component2.png}
\end{subfigure}
\caption{Feature group importance for $HC$ (left) and $AD$ (right). Darker color denotes higher importance. Height of a bar represents the number of important features.}
\label{fig:gradient_imp}
\end{figure}
\end{comment}

Results provided in Table~\ref{tab:top10_imp} show that the average normalised importance of the features associated with syntactic complexity and discourse is higher than the average importance of lexical richness features, when top-10 most important features across all the groups are selected for comparison.

\begin{comment}
\begin{table}[h]
\centering
\scriptsize {\caption{\label{tab:top10_imp} Mean normalised importance of the two feature groups, with the top-10 most important features selected for HC and AD components.}}
\begin{adjustbox}{max width=0.3\textwidth}
\begin{tabular}{l|ll}
\multirow{2}{*}{\textbf{Feature Group}} & \multicolumn{2}{c}{\textbf{\begin{tabular}[c]{@{}c@{}}Importance of\\ top-10 features\end{tabular}}} \\
 & \multicolumn{1}{c}{\textbf{HC}} & \multicolumn{1}{c}{\textbf{AD}} \\ \hline \hline
\begin{tabular}[c]{@{}l@{}}Syntactic Complexity and\\ Discourse Phenomena\end{tabular} & \multicolumn{1}{l|}{0.94} & 0.95 \\
Lexical Richness & \multicolumn{1}{l|}{0.91} & 0.92 \\ \hline
\end{tabular}
\end{adjustbox}
\end{table}
\end{comment}

\begin{table}[t]
\scriptsize
\begin{tabular}{l|cc|l|l}
\hline
\multirow{2}{*}{\textbf{Feature group}} & \multicolumn{2}{l}{\textbf{\begin{tabular}[c]{@{}l@{}}Importance of\\ top-10 features\end{tabular}}} & \multirow{2}{*}{\textbf{\#features}} & \multirow{2}{*}{\textbf{Group rank}} \\
 & \textbf{HC} & \textbf{AD} &  &  \\ \hline\hline
\begin{tabular}[c]{@{}l@{}}Syntactic complexity and\\ Discourse phenomena\end{tabular} & 0.94 & 0.95 & \multicolumn{1}{c|}{37} & \multicolumn{1}{c}{1} \\
Lexical richness & 0.91 & 0.92 & \multicolumn{1}{c|}{18} & \multicolumn{1}{c}{2} \\ \hline
\end{tabular}
\caption{\label{tab:top10_imp} Importance of the two feature groups, summarised as the mean value of the top-10 most important features selected for HC and AD components, number of features having significant Spearman correlation with deletion errors, and the rank of each group.}
\end{table}

To conclude, the feature group of syntactic complexity and discourse phenomena is affected significantly and distinctively the most by deletion errors as seen in Section~\ref{sec:deletion_features}. This group is also important for classification as seen in Table~\ref{tab:top10_imp}, indicating why classification is affected significantly by deletion errors. Hence, we track the effects from the initial step of adding artificial errors of different amounts to obtaining the final predictions in this manner.

\begin{comment}
\begin{table}[t]
\centering
\scriptsize {\caption{\label{tab:fp_correlation} Significant Spearman correlations of feature values and amount of deletion errors for FP, by feature group.}}
\begin{adjustbox}{max width=0.48\textwidth}
\begin{tabular}{l|c|c}
\textbf{Feature group} & \multicolumn{1}{l|}{\textbf{\# features}} & \multicolumn{1}{l}{\textbf{Group rank}} \\ \hline \hline
\begin{tabular}[c]{@{}l@{}}Syntactic complexity and\\ Discourse phenomena\end{tabular} & 31 & 1 \\
Lexical richness & 8 & 2 \\ \hline
\end{tabular}
\end{adjustbox}
\end{table}
\end{comment}

\vspace{-0.5em}

\section{Generalisability Evaluation}
\label{sec:general}

In order to test how well our conclusions generalise to a different dataset of impaired speech, we repeat the same experiments performed on DB on
the HA dataset (Section~\ref{sec:datasets}).

We follow the same method, as described in Section~\ref{sec:method}
to extract the features and classify samples.
Similarly to the results obtained on DB data, with HA deletion errors affect classification performance the most.
Furthermore, deletion errors differentiate the same feature group of syntactic complexity and discourse phenomen
: with HA dataset, 39 features correlate with deletions stronger than with insertions or substitutions, with $79.49$\% of features belonging to the aggregate group of syntactic complexity and discourse, and $20.51$\% - to the group of lexical richness.
The rank of feature groups, based on the average absolute Spearman correlation of all the features included in the groups, correspond to the rank observed with DB dataset, with a stronger significant correlation corresponding to the group of syntactic complexity, rather than lexical richness.

\begin{comment}
\begin{table}[!htb]
\setlength\tabcolsep{0pt}
\footnotesize\centering
\smallskip
\begin{tabular*}{\columnwidth}{p{0.2\columnwidth}|p{0.2\columnwidth}|p{0.2\columnwidth}|p{0.2\columnwidth}|p{0.2\columnwidth}}
\hline
      \textbf{Transcript} &  \textbf{Accuracy} & \textbf{F1 (macro)}&\textbf{Sensitivity}&\textbf{Specificity}\\
      \hline \hline
      Manual & 71.11 &	70.72 &	84.93 &	58.00 \\
      ASR & 64.41 &	62.90 &	94.09 &	40.18\\
       \bottomrule
    \end{tabular*}
  \scriptsize{\caption{Effect of ASR on FP classification performance.}}
\end{table}
\label{tab:transcript_performance_fp}

Further, bench-marking the performance of our ASR system on this dataset in Figure~ \ref{tab:transcript_performance_fp}, we observe a 7.8\% decrease in classification F1 score with ASR. However, sensitivity increases by almost 10\% with ASR transcripts, though it is accompanied by a corresponding drop in specificity. This is in contrast to results observed on DB (Section~ \ref{sec:classification_performance}) where F1 score, sensitivity and specificity all decreased.
\end{comment}
\vspace{-0.5em}

\section{Conclusions}
We observe that simulated deletion errors have a strong effect on classification performance when detecting cognitive impairment from speech and language, which can be traced back to their effect on syntactic complexity and discourse representations.
With this observation in mind, the practical suggestion would be to
optimise the ASR to reflect a higher penalty for deletion errors to improve dementia detection performance. For example, the decoder can be parametrised to find a balance between insertions and deletions, so that the number of deletion errors is minimised.

However, dealing with deletions in training time is not trivial, so in future work, we will focus on the optimisation of ASR performance and its effect on AD detection. Careful ASR error management, following previous work by~\citet{simonnet2017asr}, could help enable strong fully-automated speech-based predictive models for dementia detection.

\bibliographystyle{acl_natbib}

redpajama_set_name: RedPajamaArXiv


text:
Category — Brand equity
Tesla Drops "Motors." What's Next?
Tesla just announced that it is dropping "Motors" from its name to signal that it's moving into businesses other than cars. They will, at least in the foreseeable future, be an unparalleled sustainable energy company. However, at the pace they are moving, one can only imagine what businesses they might ultimately be in. Will that name hold?
Apple did this several years ago by changing its name from Apple Computer to Apple. Obviously, this was a smart move that helped them broaden their portfolio of products and services.
There are many reason why a company might move to change its name —an obvious one being a legal issue. Beyond that, changes can be driven by consumer trends, broadening portfolios, over-coming negative perceptions, or just trying to stay relevant. For example, Kentucky Fried Chicken changed its name to KFC to avoid healthy consumers thinking about fried food. Or Philip Morris created new name, Altria, to be known as more than just a cigarette manufacturer, but a portfolio of businesses.
So, for fun, below are a list of several companies that completely changed their names. Can you identify who they are today?
AuctionWeb
BackRub
Blue Ribbon Sports
Brad's Drink
Computing Tabulating Recording Corporation
Confinity
Jerry's Guide to the World Wide Web
Lucky and Goldstar Co.
Pete's Super Submarines
Quantum Computer Services
Tokyo Tsushin Kogyo
ValuJet
Wards Company
If you think about these things and are considering a business transformation, step back and look at your name.
There are interesting stories behind how these names became the ones we know today. How many did you know?
AuctionWeb, the first on-line marketplace, became eBay
BackRub was the original name for Google, based on the mathematical term "googol",1 followed by 100 zeros
Blue Ribbon Sports was the business that became Nike
Brad's Drink was the original name of a soft drink developed by a pharmacist that became Pepsi-Cola
Computing Tabulating Recording Corporation evolved into IBM, the name of its Canadian subsidiary
Confinity, started as a Palm Pilot payment company, renamed itself to become PayPal
Datsun today is Nissan based on the strategy to unify many products one-der one brand
Firebird is the original name for Firefox, changed for legal reasons
Jerry's Guide to the World Wide Web became Yahoo, standing for "Yet Another Hierarchical Officious Oracle"
Lucky and Goldstar Co. is today LG Electronics
Pete's Super Submarines is known today as Subway
Quantum Computer Services is today AOL, a shortening of America Online
Sound of Music became Best Buy, based on a successful fire sale after a tornado hit the company's main store promising "best buys" on everything
Stag Party was the original name for Playboy after a law suit was threatened by Stag Magazine
Tokyo Tsushin Kogyo was a radio repair shop that is known for making the first transistor radio, and today is known as SONY
ValuJet became AirTran after a well-publicized plane crash. Today it is part of Southwest.
Wards Company adopted the name of a retail format they owned to become Circuit City
February 6, 2017 Comments Off on Tesla Drops "Motors." What's Next?
Change the Washington Redskins Name.
I was asked by a friend about the Washington Redskins name issue, which was so eloquently written about in the New York Times Opinion Pages on June 24 by Michael Lewis and Manish Tripath.
Here is essentially what I wrote as a reply….
This is an interesting branding question on many levels. I am in complete agreement with the Michael Lewis and Manish Tripath conclusion.
1. Money aside, (changing the name) is the right thing to do.
2. The "model" they used (have not seen it) indicates no significant loss in revenue. My experience with name changes would bear this out. In fact, the opposite is often the case. I understand the argument about existing brand equity, but there are other important factors.
3. We would advise that the team owners look at a new name as an opportunity to re-energize the fan base. What if it attracted more fans and advertisers? Looking at the upside might help all involved think about a different and better brand and future for the franchise.
4. Lewis's idea of involving the Native American community leaders is a brilliant way to move forward in a positive manner. It could result in an even larger upside, albeit the process might be complex so as not to disenfranchise anyone.
Change is scary for many, but, in this case, necessary. Not only has the U.S. Patent & Trademark Office voted to strip the team of trademark protections, but changing the name is the right thing to do.
Updating Intel's Font for the Mobile Future
Intel just created its own proprietary corporate font to be easier to read in the global digital world. They call it "Intel Clear". A smart move in a number of ways. First, it gave them the opportunity to assess the equity in their existing font and think through whether and how much to change. Second, It caused them to think about how their critical audiences, internal and external, national and international, should perceive Intel as the communications media evolve so dramatically and rapidly.
April 8, 2014 Comments Off on Updating Intel's Font for the Mobile Future
Caution: What will happen to my Jim Beam and Maker's Mark Bourbon Whiskeys?
The acquisition of Beam Inc. by Suntory Holdings of Japan, has created a storm of concern about whether the heartland American brands, Jim Beam and Maker's Mark will change. With enormous heritage, both brands have very loyal franchises and passionate consumers.
And despite the fact that they have been around for a long time – Jim Beam was founded in 1795 and Maker's Mark in 1958 – these brands continue to enjoy organic growth, and are benefitting, possibly even contributing to, a resurgence in the popularity of bourbons and whiskeys globally. [Read more →]

redpajama_set_name: RedPajamaCommonCrawl


Raw fresh vegetables, such as leafy greens, carrots, sprouts and yams and most other vegetables are the best source of all minerals and vitamins! Raw fruits are an excellent source of vitamins. All these raw foods are best if organic and allowed to ripen before picking them. Also raw plant fats such as nuts, seeds, olives, avocados and coconut to name a few are an important part of our living food group.
Finally vegetable juices such as juice from green grasses, carrots, celery, cucumbers, spinach, beets and just about any other raw vegetable are needed because neither our world or our bodies are as pure and perfect as they were when man first roamed this planet. These juices give us the nutrients we need to feed our body so our cells can be rebuilt as healthy cells. Fresh juices give our bodies an advantage because the fiber has been removed and thus we get more nutrition to our cellular level quicker and with less energy being used.
Our body is a living organism made of living cells requiring living food, as its fuel, to function properly.
We get the most abundant source of nutrients for use by our body from raw vegetables, fruits, nuts, seeds and legumes.
We need enzymes to be able to properly digest our food. Raw food is full of enzymes.
We need good fiber to daily cleanse our colon. Raw food is the best source of good fiber.
Good fats and sugars are found in raw foods.
Your health can be improved by fueling it wisely, avoiding major diseases.
Our body has a predominately alkaline balance and these raw living foods which are also alkaline help to keep our pH balanced. All cooked or highly processed fast foods leave acidic ash in our body making it more difficult for the body to keep the pH balanced. So in other words, living foods have a natural alkaline effect on our bodies.
Every day our body requires raw materials to keep it functioning well. These raw materials are nutrients made up of: water, protein, fats, carbohydrates, vitamins and minerals. If we choose the best forms of each of these nutrients and eat them in a balanced diet we will make it possible to give our body what it needs to boost our immune systems so we can be healthy and disease free.
Water: Our body is almost 2/3 water and it is important in every function of the body, it helps to transport nutrients and waste products in and out of the cells. Every vegetable and fruit is composed of over 70% water in its raw state.
Carbohydrates: are the main sources of energy for our body. They are needed for the body to function.
Protein: is needed for growth and development of the body. Protein also provides the body with energy and is needed to manufacture hormones, antibodies, enzymes, and tissue. It will help to maintain a proper acid-alkaline balance in the body.
Fats: are the storehouses of energy in the body and although we want to reduce fat in our diet, it is essential for normal brain development and it provides energy and supports growth.
Minerals: Minerals are essential nutrients that the body needs to survive. We have to replace these minerals every day to keep our body functioning in a healthy mode. Therefore the minerals our body needs should come from the foods we eat rather than in supplement form.
Vitamin: There are 13 vitamins that the body absolutely needs: These vitamins help to regulate our body functions. Vitamins help to: prevent disease; promote healing; assist in digestion of protein, fats and carbohydrates; and they encourage overall good health.
Kale is a very versatile green because it stays fresh in the fridge almost twice as long as lettuce.
Toss with the following dressing. It is best to mix the salad with the dressing ahead (this will help to soften the kale) and keep in your fridge until ready to serve.
This salad and dressing mixture can be kept overnight as well.
We are now on Day 4 of our 30 Days to a NEW Healthier You. Judy Fleming, Director of Hallelujah Diet Canada started her first 30 Day Video on March 1st to help you learn how to keep on track with the Hallelujah Diet! If you missed the first video, click here to watch it now.
Judy will be speaking in your home twice a week with our 30 Days to a NEW Healthier you Videos. These short video clips include healthy tips and recipes to help with your menu planning.