You will be shown sections from a scientific research paper, together with a question about the paper. Paragraphs in the paper are separated by newlines. Your task is to answer the question based on the contents of the paper.

Paper:
----------------------------------------
Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems

Automatic machine learning systems can inadvertently accentuate and perpetuate inappropriate human biases. Past work on examining inappropriate biases has largely focused on just individual systems. Further, there is no benchmark dataset for examining inappropriate biases in systems. Here for the first time, we present the Equity Evaluation Corpus (EEC), which consists of 8,640 English sentences carefully chosen to tease out biases towards certain races and genders. We use the dataset to examine 219 automatic sentiment analysis systems that took part in a recent shared task, SemEval-2018 Task 1 'Affect in Tweets'. We find that several of the systems show statistically significant bias; that is, they consistently provide slightly higher sentiment intensity predictions for one race or one gender. We make the EEC freely available.

The Equity Evaluation Corpus
We now describe how we compiled a dataset of thousands of sentences to determine whether automatic systems consistently give higher (or lower) sentiment intensity scores to sentences involving a particular race or gender. There are several ways in which such a dataset may be compiled. We present below the choices that we made.
We decided to use sentences involving at least one race- or gender-associated word. The sentences were intended to be short and grammatically simple. We also wanted some sentences to include expressions of sentiment and emotion, since the goal is to test sentiment and emotion systems. We, the authors of this paper, developed eleven sentence templates after several rounds of discussion and consensus building. They are shown in Table TABREF3 . The templates are divided into two groups. The first type (templates 1–7) includes emotion words. The purpose of this set is to have sentences expressing emotions. The second type (templates 8–11) does not include any emotion words. The purpose of this set is to have non-emotional (neutral) sentences.
The templates include two variables: INLINEFORM0 person INLINEFORM1 and INLINEFORM2 emotion word INLINEFORM3 . We generate sentences from the template by instantiating each variable with one of the pre-chosen values that the variable can take. Each of the eleven templates includes the variable INLINEFORM4 person INLINEFORM5 . INLINEFORM6 person INLINEFORM7 can be instantiated by any of the following noun phrases:
For our study, we chose ten names of each kind from the study by Caliskan:2017 (see Table TABREF7 ). The full lists of noun phrases representing females and males, used in our study, are shown in Table TABREF8 .
The second variable, INLINEFORM0 emotion word INLINEFORM1 , has two variants. Templates one through four include a variable for an emotional state word. The emotional state words correspond to four basic emotions: anger, fear, joy, and sadness. Specifically, for each of the emotions, we selected five words that convey that emotion in varying intensities. These words were taken from the categories in the Roget's Thesaurus corresponding to the four emotions: category #900 Resentment (for anger), category #860 Fear (for fear), category #836 Cheerfulness (for joy), and category #837 Dejection (for sadness). Templates five through seven include emotion words describing a situation or event. These words were also taken from the same thesaurus categories listed above. The full lists of emotion words (emotional state words and emotional situation/event words) are shown in Table TABREF10 .
We generated sentences from the templates by replacing INLINEFORM0 person INLINEFORM1 and INLINEFORM2 emotion word INLINEFORM3 variables with the values they can take. In total, 8,640 sentences were generated with the various combinations of INLINEFORM4 person INLINEFORM5 and INLINEFORM6 emotion word INLINEFORM7 values across the eleven templates. We manually examined the sentences to make sure they were grammatically well-formed. Notably, one can derive pairs of sentences from the EEC such that they differ only in one word corresponding to gender or race (e.g., `My daughter feels devastated' and `My son feels devastated'). We refer to the full set of 8,640 sentences as Equity Evaluation Corpus.

Measuring Race and Gender Bias in Automatic Sentiment Analysis Systems
The race and gender bias evaluation was carried out on the output of the 219 automatic systems that participated in SemEval-2018 Task 1: Affect in Tweets BIBREF5 . The shared task included five subtasks on inferring the affectual state of a person from their tweet: 1. emotion intensity regression, 2. emotion intensity ordinal classification, 3. valence (sentiment) regression, 4. valence ordinal classification, and 5. emotion classification. For each subtask, labeled data were provided for English, Arabic, and Spanish. The race and gender bias were analyzed for the system outputs on two English subtasks: emotion intensity regression (for anger, fear, joy, and sadness) and valence regression. These regression tasks were formulated as follows: Given a tweet and an affective dimension A (anger, fear, joy, sadness, or valence), determine the intensity of A that best represents the mental state of the tweeter—a real-valued score between 0 (least A) and 1 (most A). Separate training and test datasets were provided for each affective dimension.
Training sets included tweets along with gold intensity scores. Two test sets were provided for each task: 1. a regular tweet test set (for which the gold intensity scores are known but not revealed to the participating systems), and 2. the Equity Evaluation Corpus (for which no gold intensity labels exist). Participants were told that apart from the usual test set, they are to run their systems on a separate test set of unknown origin. The participants were instructed to train their system on the tweets training sets provided, and that they could use any other resources they may find or create. They were to run the same final system on the two test sets. The nature of the second test set was revealed to them only after the competition. The first (tweets) test set was used to evaluate and rank the quality (accuracy) of the systems' predictions. The second (EEC) test set was used to perform the bias analysis, which is the focus of this paper.
Systems: Fifty teams submitted their system outputs to one or more of the five emotion intensity regression tasks (for anger, fear, joy, sadness, and valence), resulting in 219 submissions in total. Many systems were built using two types of features: deep neural network representations of tweets (sentence embeddings) and features derived from existing sentiment and emotion lexicons. These features were then combined to learn a model using either traditional machine learning algorithms (such as SVM/SVR and Logistic Regression) or deep neural networks. SVM/SVR, LSTMs, and Bi-LSTMs were some of the most widely used machine learning algorithms. The sentence embeddings were obtained by training a neural network on the provided training data, a distant supervision corpus (e.g., AIT2018 Distant Supervision Corpus that has tweets with emotion-related query terms), sentiment-labeled tweet corpora (e.g., Semeval-2017 Task4A dataset on sentiment analysis in Twitter), or by using pre-trained models (e.g., DeepMoji BIBREF20 , Skip thoughts BIBREF21 ). The lexicon features were often derived from the NRC emotion and sentiment lexicons BIBREF22 , BIBREF23 , BIBREF24 , AFINN BIBREF25 , and Bing Liu Lexicon BIBREF26 .
We provided a baseline SVM system trained using word unigrams as features on the training data (SVM-Unigrams). This system is also included in the current analysis.
Measuring bias: To examine gender bias, we compared each system's predicted scores on the EEC sentence pairs as follows:
Thus, eleven pairs of scores (ten pairs of scores from ten noun phrase pairs and one pair of scores from the averages on name subsets) were examined for each template–emotion word instantiation. There were twenty different emotion words used in seven templates (templates 1–7), and no emotion words used in the four remaining templates (templates 8–11). In total, INLINEFORM0 pairs of scores were compared.
Similarly, to examine race bias, we compared pairs of system predicted scores as follows:
Thus, one pair of scores was examined for each template–emotion word instantiation. In total, INLINEFORM0 pairs of scores were compared.
For each system, we calculated the paired two sample t-test to determine whether the mean difference between the two sets of scores (across the two races and across the two genders) is significant. We set the significance level to 0.05. However, since we performed 438 assessments (219 submissions evaluated for biases in both gender and race), we applied Bonferroni correction. The null hypothesis that the true mean difference between the paired samples was zero was rejected if the calculated p-value fell below INLINEFORM0 .

Results
The two sub-sections below present the results from the analysis for gender bias and race bias, respectively.

Conclusions and Future Work
We created the Equity Evaluation Corpus (EEC), which consists of 8,640 sentences specifically chosen to tease out gender and race biases in natural language processing systems. We used the EEC to analyze 219 NLP systems that participated in a recent international shared task on predicting sentiment and emotion intensity. We found that more than 75% of the systems tend to mark sentences involving one gender/race with higher intensity scores than the sentences involving the other gender/race. We found such biases to be more widely prevalent for race than for gender. We also found that the bias can be different depending on the particular affect dimension involved.
We found the score differences across genders and across races to be somewhat small on average ( INLINEFORM0 , which is INLINEFORM1 of the 0 to 1 score range). However, for some systems the score differences reached as high as 0.34 (34%). What impact a consistent bias, even with an average magnitude INLINEFORM2 , might have in downstream applications merits further investigation.
We plan to extend the EEC with sentences associated with country names, professions (e.g., doctors, police officers, janitors, teachers, etc.), fields of study (e.g., arts vs. sciences), as well as races (e.g., Asian, mixed, etc.) and genders (e.g., agender, androgyne, trans, queer, etc.) not included in the current study. We can then use the corpus to examine biases across each of those variables as well. We are also interested in exploring which systems (or what techniques) accentuate inappropriate biases in the data and which systems mitigate such biases. Finally, we are interested in exploring how the quality of sentiment analysis predictions varies when applied to text produced by different demographic groups, such as people of different races, genders, and ethnicities.
The Equity Evaluation Corpus and the proposed methodology to examine bias are not meant to be comprehensive. However, using several approaches and datasets such as the one proposed here can bring about a more thorough examination of inappropriate biases in modern machine learning systems.

Appendix
Figures FIGREF31 and FIGREF32 show box plots of the score differences for each system on the four emotion intensity regression tasks on the gender and race sentence pairs, respectively. Each point on a plot corresponds to the difference in scores predicted by the system on one sentence pair. The systems are ordered by their performance rank (from first to last) on the task as per the official evaluation metric on the tweets test sets.

Table 1: Sentence templates used in this study.

Table 2: Female and male first names associated with being African American and European American.

Table 4: Emotion words used in this study.

Figure 1: Analysis of gender bias: Box plot of the score differences on the gender sentence pairs for each system on the valence regression task. Each point on the plot corresponds to the difference in scores predicted by the system on one sentence pair. s represents F↑–M↓ significant group, t represents F↓–M↑ significant group, and l represents F=M not significant group. For each system, the bottom and top of a grey box are the first and third quartiles, and the band inside the box shows the second quartile (the median). The whiskers extend to 1.5 times the interquartile range (IQR = Q3 - Q1) from the edge of the box. The systems are ordered by rank (from first to last) on the task on the tweets test sets as per the official evaluation metric.

Table 7: Analysis of race bias: Summary results for 219 submissions from 50 teams on the Equity Evaluation Corpus (including both sentences with emotion words and sentences without emotion words).

Figure 2: Analysis of race bias: Box plot of the score differences on the race sentence pairs for each system on the valence regression task. Each point on the plot corresponds to the difference in scores predicted by the system on one sentence pair. s represents AA↑–EA↓ significant group, t represents AA↓–EA↑ significant group, and l represents AA=EA not significant group. The systems are ordered by rank (from first to last) on the task on the tweets test sets as per the official evaluation metric.
----------------------------------------

Question: What criteria are used to select the 8,640 English sentences?

To answer the question, format your response as a `json` object with two fields:

"answer": A string providing a succinct answer to the question, in your own words.
"evidence": An array of strings. Each entry should be a full paragraph from the paper. Together, the evidence should serve as a justification for the answer.

For instance, for the question "What baselines did the authors compare against?", a sample response might be:

{
  "answer": "BERT and RoBERTa."
  "evidence": ["We compare our approach against two baselines. In Table 1, we compare against BERT. In Table 2, we compare against RoBERTa. Our findings indicate that our approach improves over both baeslines..."]
}

The "answer" field should be roughly 184 characters in length.

Do not include any text in your response other than the json. If the question is unanswerable given the provided excerpts, respond with the single word "null".

To repeat, the question is: What criteria are used to select the 8,640 English sentences?

Answer JSON object:
{"answer": "Sentences involving at least one race- or gender-associated word,  sentence  have to be short and grammatically simple,  sentence have to  include expressions of sentiment and emotion.", "evidence": ["We decided to use sentences involving at least one race- or gender-associated word. The sentences were intended to be short and grammatically simple. We also wanted some sentences to include expressions of sentiment and emotion, since the goal is to test sentiment and emotion systems. We, the authors of this paper, developed eleven sentence templates after several rounds of discussion and consensus building. They are shown in Table TABREF3 . The templates are divided into two groups. The first type (templates 1\u20137) includes emotion words. The purpose of this set is to have sentences expressing emotions. The second type (templates 8\u201311) does not include any emotion words. The purpose of this set is to have non-emotional (neutral) sentences."]}