You will be shown sections from a scientific research paper, together with a question about the paper. Your task is to answer the question based on the contents of the paper. Your response should be a `json` object with two fields:

"answer": An array of strings extracted directly from the paper which, collectively, answer the question.
"evidence": An array of strings. Each should be an excerpt from the paper, in which one or more of the extracted answers can be found.

For instance, for the question "What baselines did the authors compare against?", a sample response might be:

{
  "answer": ["BERT", "RoBERTa"],
  "evidence": ["In our experiments, we compare the performance of our model against BERT and RoBERTa."]
}

Do not include any text in your response other than the json.

If the question is unanswerable given the provided excerpts, respond with the single word "null".

Paper: Affective Behaviour Analysis of On-line User Interactions: Are On-line Support Groups more Therapeutic than Twitter?

The increase in the prevalence of mental health problems has coincided with a growing popularity of health related social networking sites. Regardless of their therapeutic potential, On-line Support Groups (OSGs) can also have negative effects on patients. In this work we propose a novel methodology to automatically verify the presence of therapeutic factors in social networking websites by using Natural Language Processing (NLP) techniques. The methodology is evaluated on On-line asynchronous multi-party conversations collected from an OSG and Twitter. The results of the analysis indicate that therapeutic factors occur more frequently in OSG conversations than in Twitter conversations. Moreover, the analysis of OSG conversations reveals that the users of that platform are supportive, and interactions are likely to lead to the improvement of their emotional state. We believe that our method provides a stepping stone towards automatic analysis of emotional states of users of online platforms. Possible applications of the method include provision of guidelines that highlight potential implications of using such platforms on users' mental health, and/or support in the analysis of their impact on specific individuals.

Introduction
Recently, people have started looking at online forums either as a primary or secondary source of counseling services BIBREF0. BIBREF1 reported that over the first five years of operation (2011-2016), ReachOut.com – Ireland's online youth mental health service – 62% of young people would visit a website for support when going through a tough time. With the expansion of the Internet, there has been a substantial growth in the number of users looking for psychological support online.
The importance of the on-line life of patients has been recognized in research as well. BIBREF2 stated that the online life of patients constitutes a major influence on their self-definition. Furthermore, according to BIBREF3, the social networking activities of an individual, offer an important reflection of their personality. While dealing with patients suffering from psychological problems, it is important that therapists do not ignore this pivotal source of information which can provide deep insights into their patients' mental conditions.
Acceptance of on-line support groups (OSG) by Mental Health Professionals is still not established BIBREF4. Since OSG can have double-edged effects on patients and the presence of professionals is often limited, we argue that their properties should be further studied. According to BIBREF5 OSG effectiveness is hard to assess, while some studies showed OSG's potential to change participants' attitudes, no such effect was observed in other studies (see Related Work Section for more details). Furthermore the scope of previous work on analysis of users' behaviour in OSG has been limited by the fact that they relied on expert annotation of posts and comments BIBREF6.
We present a novel approach for automatically analysing online conversations for the presence of therapeutic factors of group therapy defined by BIBREF7 as “the actual mechanisms of effecting change in the patient”. The authors have identified 11 therapeutic factors in group therapy: Universality, Altruism, Instillation of Hope, Guidance, Imparting information, Developing social skills, Interpersonal learning, Cohesion, Catharsis, Existential factors, Imitative behavior and Corrective recapitulation of family of origin issues. In this paper, we focus on 3 therapeutic factors: Universality, Altruism and Instillation of Hope (listed below), as we believe that these can be approximated by using established NLP techniques (e.g. Sentiment Analysis, Dialogue Act tagging etc.).
Universality: the disconfirmation of a user's feelings of uniqueness of their mental health condition.
Altruism: others offer support, reassurance, suggestions and insight.
Instillation of Hope: inspiration provided to participants by their peers.
The selected therapeutic factors are analysed in terms of illocutionary force and attitude. Due to the multi-party and asynchronous nature of on-line social media conversations, prior to the analysis, we extract conversation threads among users – an essential prerequisite for any kind of higher-level dialogue analysis BIBREF10. Afterwards, the illocutionary force is identified using Dialogue Act tagging, whereas the attitude by using Sentiment Analysis. The quantitative analysis is then performed on these processed conversations.
Ideally, the analysis would require experts to annotate each post and comment on the presence of therapeutic factors. However, due to time and cost demands of this task, it is feasible to analyse only a small fraction of the available data. Compared to previous studies (e.g. BIBREF6) that analysed few tens of conversations and several thousand lines of chat; using the proposed approach – application of Dialogue Acts and Sentiment Analysis – we were able to automatically analyse approximately 300 thousands conversations (roughly 1.5 million comments). The rest of the paper is structured as follows. In Section 2 we introduce related work. Next, in Section 3 we describe the pre-processing pipeline and the methodology to perform thread extraction on asynchronous multi-party conversations. In Section 4 we provide the describe the final dataset used for the analysis, and in Section 5 we present the results of our analysis. Finally, in Section 6 we provide concluding remarks and future research directions.

Related Work
On-line support groups have been analyzed for various factors before. For instance, BIBREF11 analysed stress reduction in on-line support group chat-rooms, and the effects of on-line social interactions. Such studies mostly relied on questionnaires and were based on a small number of users. Nevertheless, in BIBREF11, the author showed that social support facilitates coping with distress, improves mood and expedites recovery from it. These findings highlight that, overall, on-line discussion boards appear to be therapeutic and constructive for individuals suffering alcohol-abuse.
Application of NLP to the analysis of mental health-related conversation has been studied as well (e.g. BIBREF12, BIBREF13). BIBREF6 applied sentiment-analysis combined with extensive turn-level annotation to investigate stress reduction in on-line support group chat-rooms, showing that sentiment-analysis is a good predictor of entrance stress level. Furthermore, similar to our setting, they applied automatic thread-extraction to determine conversation threads.
BIBREF14 have shown that on-line support group therapy increased the quality of life of patients with metastatic breast cancer. Since many original posters reported the benefits of group therapy on patients BIBREF15, BIBREF2, BIBREF16, BIBREF17, BIBREF18, BIBREF7, we evaluate the effect of the user interaction using sentiment scores of comments in on-line support groups.
According to BIBREF6, users with high incoming stress tend to request less information from others, as a percentage of their time, and share much more information, in absolute terms. In addition, high information sharing has been shown to be a good predictor of stress reduction at the end of the chat BIBREF6. Regarding information sharing, we rely on Dialogue Acts BIBREF19 to model the speaker's intention in producing an utterance. In particular, we are interested in Dialogue Act label that is defined to represent descriptive, narrative, or personal information – the statement.
Dialogue Acts have been applied to the analysis of spoken BIBREF20, BIBREF21 as well as on-line written synchronous conversations BIBREF22. We apply Dialogue Act tag set defined in BIBREF22 to the analysis of our on-line asynchronous conversations. We argue that Dialogue Acts can be used to analyse user behaviour in social media and verify the presence of therapeutic factors.

Datasets
We verify the presence of therapeutic factors in two social media datasets: OSG and Twitter. The first dataset is crawled from an on-line support groups website, and the second dataset consists of a small sample of Twitter conversation threads. Since the former consists of multi-threaded conversations, we apply a pre-processing to extract conversation threads to provide a fair comparison with the Twitter dataset. An example conversation from each data source is presented in Figure FIGREF19.

Datasets ::: Twitter
We have downloaded 1,873 Twitter conversation threads, roughly 14k tweets, from a publicly available resource that were previously pre-processed and have conversation threads extracted. A conversation in the dataset consists of at least 4 tweets. Even though, according to BIBREF23, Twitter is broadly applicable to public health research, our expectation is that it contains less therapeutic conversations in comparison to specialized on-line support forums.

Analysis ::: Change in Sentiment score of Original Posters
The first condition which we test is the sentiment change in conversation threads, comparing the initial and final sentiment scores (i.e. posts' scores) of the original poster. The results of the analysis are presented in Figure FIGREF33. In the figure we can observe that the distribution of the sentiment change in the two datasets is different. While in Twitter the amount of conversations that lead to the increase of sentiment score is roughly equal to the amount of conversations that lead to the decrease of sentiment score; the situation is different for OSG. In OSG, the amount of conversations that lead to the increase of sentiment score is considerably higher.
Figure FIGREF34 provides a more fine grained analysis, where we additionally analyse the sentiment change in nominal polarity terms – negative and positive. In OSG, the number of users that changed polarity from negative to positive is more than the double of the users that have changed the polarity from positive to negative. In Twitter, on the other hand, the users mostly changed polarity from positive to negative. Results of the analysis suggest that in OSG , sentiment increases and users tend to change polarity from negative to positive, whereas in Twitter sentiment tends to decrease. Verification of this condition alone indicates that the ratio of potentially therapeutic conversations in Twitter is lower.

Analysis ::: Sentiment of Posts and Comments
Table TABREF39 presents the distribution of sentiment polarity in post and comment statements (i.e. sentences tagged as statement). For OSG, the predominant sentiment label of statements is positive and it is the highest for both posts and comments. However, the difference between the amounts of positive and negative statements is higher for the replying comments (34.5% vs. 42.5%). For Twitter, on the other hand, the predominant sentiment label of statements is neutral and the polarity distribution between posts and comments is very close. One particular observation is that the ratio of negative statements is higher in OSG for both posts and comments than in Twitter, which supports the idea of sharing negative experiences.
Further we analyze whether the sentiment of a comment (i.e. the replying user) is affected by the sentiment of the original post (i.e. the user being replied to), which will imply that the users adapt their behaviour with respect to the post's sentiment. For the analysis, we split the datasets into three buckets according to the posts' sentiment score – negative, neutral, or positive, and represent each conversation in terms of percentages of comments (replies) with each sentiment label. The buckets are then compared using t-test for statistically significant differences.
Table TABREF40 presents the distribution of sentiment labels with respect to the post's sentiment score. The patterns of distribution are similar across the datasets. We can observe that overall, replies tend to have a positive sentiment, which suggests that replying posters tend to have a positive attitude. However, the ratio of positive comments is higher for OSG than for Twitter.
The results of the Welch's t-test on OSG data reveal that there are statistically significant differences in the distribution of replying comments' sentiment between conversations with positive and negative starting posts. A positive post tends to get significantly more positive replies. Similarly, a negative post tends to get significantly more negative replies (both with $p < 0.01$).
Table TABREF41 presents the distribution of the sentiment labels of the final text provided by the original poster with respect to the sentiment polarity of the comments. The results indicate that OSG participants are more supportive, as the majority of conversations end in a positive final sentiment regardless of the sentiment of comments. We can also observe that negative comments in OSG lead to positive sentiment, which supports the idea of sharing the negative experiences, thus presence of therapeutic factors. For Twitter, on the other hand, only positive comments lead to the positive final sentiments, whereas other comments lead predominantly to neutral final sentiments.
Our analysis in terms of sentiment and Dialogue Acts supports the presence of the three selected therapeutic factors – Universality, Altruism and Instillation of Hope – in OSG more than in Twitter. The main contributors to this conclusion are the facts that there is more positive change in the sentiment of the original posters in OSG (people seeking support) and that in OSG even negative and neutral comments are likely to lead to positive changes.

Conclusion
In this work, we propose a methodology to automatically analyse online social platforms for the presence of therapeutic factors (i.e. Universality, Altruism and Instillation of Hope). We evaluate our approach on two on-line platforms, Twitter and an OSG web forum. We apply NLP techniques of Sentiment Analysis and Dialogue Act tagging to automatically verify the presence of therapeutic factors, which allows us to analyse larger amounts of conversational data (as compared to previous studies).
Our analysis indicates that OSG conversations satisfy higher number of conditions approximating therapeutic factors than Twitter conversations. Given this outcome, we postulate that users who join support group websites spontaneously seem to benefit from it. Indeed, as shown in Section SECREF5, the original posters who interact with others by replying to comments, have benefited from an improvement of their emotional state.
We would like to reemphasise that the conditions for the therapeutic factors are necessary but not sufficient; since our analysis focuses on the structure of conversations, being agnostic to the content. NLP, however, allows us to strengthen our approximations even further. Thus, the further extension of our work is also augmentation of our study with other language analysis metrics and their correlation with human annotation.
It should be noted that the proposed approach is an approximation of the tedious tasks of annotation of conversations by experts versed in the therapeutic factors and their associated theories. Even though we can use Sentiment Analysis to detect the existence of therapeutic factors, we cannot differentiate between Altruism and Instillation of Hope, as this requires differentiation between emotional state of the user and the intended emotional communication. Thus, the natural extensions of this work are differentiation between different therapeutic factors and comparison of the proposed analysis to the human evaluation.
Although we acknowledge that the proposed methodology does not serve as a replacement of manual analysis of OSG for the presence of therapeutic factors, we believe that it could facilitate and supplement this process. The method can serve as a tool for general practitioners and psychologists who can use it as an additional source of information regarding their patients’ condition and, in turn, offer a more personalised support that is better tailored to individual therapeutic needs.

Figure 1: Two example conversation threads extracted from an OSG and Twitter.

Figure 2: The percentages of threads in OSG and Twitter leading to the increase or decrease of the sentiment score of the original poster.

Table 2: The distribution (in percentages) of automatically predicted per-sentence Dialogue Act tags. Tags are counted separately for each sentence in the multisentence posts and comments.

Question: How large is the Twitter dataset?
{"answer": ["1,873 Twitter conversation threads, roughly 14k tweets"], "evidence": ["We have downloaded 1,873 Twitter conversation threads, roughly 14k tweets, from a publicly available resource that were previously pre-processed and have conversation threads extracted. A conversation in the dataset consists of at least 4 tweets. Even though, according to BIBREF23, Twitter is broadly applicable to public health research, our expectation is that it contains less therapeutic conversations in comparison to specialized on-line support forums."]}