\section{Introduction}
Speech interfaces like Amazon Alexa, Google Assistant and Apple Siri are now commonly used through mobile and smart speaker devices. These intelligent personal assistants (IPAs) use speech as the primary means of interaction, lacking embodiment compared to other speech systems such as robots or embodied conversational agents (ECAs). HCI research on these speech interface based intelligent personal assistants (IPAs) has looked to develop understanding of users’ expectations, perceptions and experiences. This work suggests that users tend to use human dialogue as an initial interaction metaphor, with expectations of system competence and function being set by the human-likeness of their design \cite{cowan_what_2017,cowan2017they,luger_like_2016}. Emulating aspects of humanness in speech interfaces is  common \cite{lee_social-psychological_2005,cassell2001embodied,clark2018social,cowan_voice_2015}, although the wisdom of this design choice has been questioned \cite{clark2018social, clark2016multimodal,moore_progress_2016, cowan_what_2017} as it may create unrealistic expectations of system capability \cite{luger_like_2016}. That said, currently it is not clear how people conceptualise the humanness of speech-based IPAs and how exactly users perceive that humans and speech interfaces vary when considering humanness as the metaphor for interaction. This knowledge is critical to informing the growing debate around humanness. By specifying the dimensions that users find important in this concept we can identify what type of design decisions may influence user expectations. Comparing humans and speech-based IPAs can also lead to a deeper understanding of the gulf between expectation and reality \cite{luger_like_2016, leahu_how_2013} when humanness is used as a heuristic for interaction.

The work presented contributes a user-centered insight into the dimensions that are important to users when considering humanness of speech-based IPAs and identifies how these vary from human dialogue. We do this by conducting a study using the Repertory Grid Technique (RGT) where, following interactions with three dialogue partners (a human, Amazon Alexa and Apple Siri), participants generated constructs that best described those interactions. Through content analysis of participant-generated constructs we identify eight salient dimensions along which humanness is most commonly conceptualised. These focus on \textit{partner knowledge set}, the qualities associated with \textit{interpersonal connections}, the \textit{linguistic content} of the partner’s utterances, a partner's \emph{performance and capabilities}, their \textit{identity and role} in the dialogue, the \textit{vocal qualities} they portray and \textit{behavioral affordances} toward a partner. We also outline the key differences between perceptions of humans and speech-based IPAs on these dimensions, with interview data showing users are highly aware of the limitations of IPAs in comparison and that they form a different category of communicative interaction along these dimensions. Our findings are the first to outline the multi-faceted nature of user perceptions of humanness, identifying the key dimensions that need to be considered when designing and researching humanness in speech interface interaction.

\section{Related Work}
\subsection{Humanness as an interaction metaphor}
Previous work on humanness has generally focused on embodied agents and robots, discussing concepts such as the uncanny valley \cite{mori19702012}. Studies have also explored categorizations of human-nature and human-uniqueness \cite{haslam2005more, haslam2006dehumanization} and how imbuing systems with personality traits increases perceived human likeness \cite{zlotowski2015persistence,ghafurian2019role}. Work also strongly emphasises how embodied factors mediate perceptions of humanness, showing influence from kinematics \cite{thompson2011perception}, agent appearance and embodiment \cite{breazeal2003emotion, schwind2018avoiding, schwind2018avoiding, ho2017measuring}, facial expressions and gesture \cite{bruce2002role, kuno2007museum}. Current, widely used IPA devices are not embodied beyond the device casing and lights that subtly change colour (i.e., they have no appendages, head or face, nor movement like a robot or avatar), so perceptions of humanness may not be driven by similar considerations. Yet as an overarching design principle for non-embodied speech-based IPAs, humanness is a fundamental concept in understanding interactions with current speech interfaces. In particular, speech system development focuses heavily on emulating human aspects of speech through expressiveness and human-like synthesis \cite{akuzawa_expressive_2018} or human-based conversational rules and structures \cite{edlund_pause_2009, gilmartin_social_2017}. This humanness seems to give users expectations as to the type of capabilities that a system may have as a dialogue partner \cite{luger_like_2016, leahu_how_2013}. Indeed, incongruence between expectations derived from humanness cues and the realities of interacting with speech-based systems is detrimental to speech interface user experience \cite{cowan_what_2017,luger_like_2016,moore_progress_2016,purington_alexa_2017}. Qualitative work investigating the user experience of speech-based IPAs for both power users \cite{luger_like_2016} and infrequent users \cite{cowan_what_2017}, found that they seem to initially use human dialogue as a metaphor for interaction. The humanness of current speech-based IPA design also leads to inaccurate expectations of system capabilities, which are adjusted after experiencing interaction \cite{luger_like_2016,leahu_how_2013}. This impact of humanness is echoed by \citet{moore_progress_2016} who highlights that human-like voices in particular may create overestimations of a system’s capabilities, resulting in potential communication breakdown and unsuccessful engagement with systems. Although the humanness communicated by the design of speech interfaces may act as an anchor for perceptions, people do seem to make a fundamental distinction between the capabilities of automated and human dialogue partners.  They see machines as more basic or functional in their conversational capabilities \cite{branigan_role_2011,clark_what_2019}, acting as “at risk” listeners in conversations \cite{oviatt_linguistic_1998}.

Understanding user’s perception towards speech interfaces may be important to unlocking reasons for people’s behaviours in interaction. Poor perceptions of system capability are assumed to cause us to adapt our language choices in interaction \cite{amalberti_user_1993,brennan1996conceptual,le_bigot_effects_2007,meddeb_what?_2010}. They drive us to use fewer fillers and coherence markers \cite{amalberti_user_1993}, more basic lexical choices, and shorter utterances compared to when we communicate with a human \cite{kennedy_dialogue_1988}, although the ubiquity of this role is debated \cite{cowan_voice_2015,cowan2015does}.

\subsection{Understanding perceptions using repertory grids}
Understanding what factors influence these perceptions could be critical to understanding fundamental mechanisms in speech interface interaction. The RGT is a research tool designed to facilitate the discovery of what may drive users perceptions towards particular objects and situations. Developed as a tool to support personal construct theory in psychology, the technique allows participants to generate labels (termed \textit{personal constructs}) to describe, conceptualise and compare particular objects of study (termed \textit{elements}) \cite{kelly_psychology_1991}. Participants are exposed to three elements at a time, through a paradigm known as \textit{triading}, where constructs are generated to differentiate two similar elements from a distinct third element. RGT therefore allows researchers an insight into an individual's reasoning and conceptualising process as they work toward understanding their experiences \cite{fransella_manual_2004}. Historically, it has been used in educational psychology \cite{shaw_focus_1978} and information design \cite{hogan_blending_2013}. HCI-related studies have also utilized the RGT to examine perceptions of website usability \cite{tung_attributes_2009}, perceptions of strategic information systems \cite{cho2010exploring} and perceptions of mobile technologies \cite{fallman2010capturing}.

\begin{table*}[]
\caption{Question types with examples}
\label{table1}
\begin{tabular}{@{}cc@{}}
\toprule
\textbf{Question/request type} & \textbf{Question/request format} \\ \midrule
Conversational & \textit{\begin{tabular}[c]{@{}c@{}}How are you today?\\ Where are you from?\\ Tell me a joke\end{tabular}} \\ \midrule
Information retrieval & \textit{\begin{tabular}[c]{@{}c@{}}Who is {[}insert famous person's name{]}?\\ What is the square root of {[}insert three digit number{]}?\\ How do I get to the City Centre from here?\end{tabular}} \\ \midrule
Subjective/opinion-based & \textit{\begin{tabular}[c]{@{}c@{}}Do you like {[}insert favorite genre of music{]}?\\ Can you recommend a place to eat {[}insert favorite food when eating out{]}?\\ What do you think of {[}insert famous person's name - same as before{]}?\end{tabular}} \\ \bottomrule
\end{tabular}
\end{table*}

\section{Research Aims}
Humanness plays a critical role in the development of perceptions towards IPAs. Yet concepts of humanness are predominantly discussed in embodied interactions, as a unidimensional concept, with little known about how users conceptualise humanness in non-embodied speech interfaces. Insight into user-generated concepts of humanness is crucial to add nuance to the debate around its role in user experience. Mapping the dimensions that users find important when conceptualising the humanness of speech-based IPAs through a bottom-up construct elicitation paradigm will allow us to identify the design dimensions that may need to be prioritised to ensure accurate user perceptions are generated. Through using RGT, this study aims to: (1) map the dimensionality of the concept of humanness in interactions with speech-based IPAs; and (2) identify the nuances between human and IPA interaction within these dimensions.

\section{Method}
Using random sampling, twenty-four participants from a European university were recruited via an email circulated across the university. Recruitment posters were also displayed around campus. Each participant was given a €10 honorarium for taking part. Three participants were omitted from the data due to them having difficulty in completing the grids unassisted. Of the remaining twenty one participants (F=9, M=11; Mean age = 23.1yrs; SD=5.49) all were native or near native English speakers. 23.8\% (N=5) of the sample reported using speech-based IPAs daily to a few times per week with 14.3\% of the sample (N=3) reporting only using speech-based IPAs a few times a month. 38.1\% of the sample reported using them rarely (N=8) with 23.8\% (N=5) having never used them before the study. Among those that had used speech-based IPAs, Siri (50\%) was most commonly used, followed by Google Assistant (31.3\%) and Amazon Alexa (18.8\%).

\subsection{Procedure}
Upon arriving at the lab, all participants were fully briefed about the nature of the study and were asked to provide consent to take part. After completing demographics, participants were then given instructions as to the purpose of the study. Participants were informed that during the session they would have a brief interaction with three dialogue partners where they would ask each of them in turn a set of pre-determined questions (Familiarisation Phase). They were then told that after their interactions they would take part in a semi-structured interview where they would identify a series of constructs relevant to these interactions (Construct Elicitation Phase). After this, participants were asked to rate each interaction in relation to these constructs on a sliding scale (Rating Phase). The phases are described in detail below.

\subsubsection{Familiarisation Phase}
Each participant completed a familiarisation session whereby they interacted with three elements. The elements selected included Siri (accessed on an iPhone SE smartphone), Alexa (accessed through an Echo Dot device), and a human (a member of the research team). The speech-based IPA elements were chosen because of their popularity and how they varied in their use of multimodality in interaction. Alexa, via the Echo Dot, predominantly uses speech, with minimal visual output through using a single coloured ring on the top of the device. Siri primarily uses speech as a primary interaction modality but also uses significant visual feedback through the smartphone’s touchscreen. The human was included to act as a comparator and ensure that constructs would relate to humanness. For ecological validity, human and speech-based IPA responses were unconstrained. Familiarization was completed with all three elements prior to the construct elicitation phase. The order in which elements were addressed was randomised between participants to control for recency and order effects. Only minimal introductions between the human and participants were made by the lead researcher to confine the interaction as much as possible within the predefined questions.

To facilitate interaction with each element, participants were tasked with asking each dialogue partner nine pre-set questions. These were devised by the authors to emphasise differences in the way humans and speech-based IPAs can communicate, so as to ensure a wide set of comparisons could be made. The questions used are shown in Table \ref{table1}. The order in which elements were addressed was randomised between participants.

\subsubsection{Construct Elicitation Phase}
Next, through a semi-structured interview, participants were asked to compile a list of \textit{implicit constructs} that they felt best described the key similarities and differences between their interactions. These are critical constructs that inform the participant’s own understanding of their experiences \cite{kelly_psychology_1991,fransella_manual_2004}. Participants were asked to focus on the communicative abilities and qualities of each element. They were also asked to provide context and reasoning around why they were choosing certain words and how it related to their interactions. After an exhaustive list had been compiled (averaging 11-13 constructs), participants were asked to devise a dichotomous \textit{emergent construct} for each \textit{implicit construct} elicited. Participants were informed that this did not have to be a direct antonym - although it could be if appropriate - but should be based on the context in which they meant the original word. From this a grid of implicit-emergent construct pairs were gathered for each participant.

\subsubsection{Rating Phase}
In this final stage, participants were asked to rate where they felt each element fell between each of the implicit-emergent poles (construct pairs) elicited. To do this, participants were presented with a grid containing all constructs produced during the construct elicitation phase. Implicit constructs were entered into the left hand column, emergent constructs into the right hand column, and both columns were connected with a straight line, 134mm long. Participants were asked to place a mark on the line where they felt each element sat on a spectrum between the construct poles. This data was used to help interpret whether the implicit or emergent construct was more closely associated with human or IPA based experiences. They were also used to provide additional context and help interpret construct meanings, further supporting the categorisation process.

All sessions were recorded and transcribed. Upon completing the session, users were debriefed as to the aims of the study and were thanked for taking part.

\subsection{Analysis}
A bootstrapped content analysis of the construct pairs \cite{jankowicz_easy_2004} was conducted to identify consistent themes among the constructs generated. This categorisation helps identify overarching meanings or characteristics across the constructs \cite{jankowicz_easy_2004}. Following guidelines by \cite{jankowicz_easy_2004} the analysis was conducted over three phases.

\subsubsection{Phase 1} Three of the authors with experience in qualitative and/or speech interface research independently assigned initial thematic tags to each implicit-emergent construct pair. This was based on semantic assessment of construct labels, using participant explanations from the interview data and ratings to ensure the participants’ intended meaning was captured. From this phase, an initial understanding and conceptualisation of common themes present in the constructs was produced.

\subsubsection{Phase 2} The themes from Phase 1 were then discussed by two of the annotators from the initial coding phase and two speech interface researchers who had not been involved in the initial coding, using the \textit{constant comparison method} \cite{dick_social_2001}. In this phase each construct pair was compared with each other to decide if they were similar. Further defining and refining of categories occurred as the process was undertaken to produce a final set of thematic tags.

\subsubsection{Phase 3} As a final phase, two annotators (one of whom was not involved in Phase 2) used the themes generated in Phase 2 to independently annotate the data. Both of these annotators then discussed areas of disagreement and revised annotations independently. This process is important to reduce idiosyncrasies and improve the reliability of category identification \cite{dick_social_2001,jankowicz_easy_2004}. Agreement between raters was high [Cohen’s kappa = 0.76, p <.001]. A final discussion took place to resolve remaining disagreements before final annotations were performed by the lead author.

\section{Results}
Participants across the study produced a total of 266 implicit-emergent construct pairs. The analysis resulted in eight key dimensions\footnote{A miscellaneous category was created when no agreement on category and construct explanation could be traced back to interview data} that categorise participants’ perceptions when comparing dialogue with a human to dialogue with speech-based IPA partners. A summary of the themes, selected construct pairs and total number of constructs linked to each theme are shown in Table \ref{Table 2:}. Some construct pair examples in the table were reversed so that constructs most closely related to the human partner appear on the left for ease of interpretation (e.g., Human/IPA). A full list of the constructs elicited in each category is included in supplementary material. Below we use data from interview transcripts to explore the definition, and differences between human and speech-based IPA dialogue experiences respectively.

\begin{table*}[]
\caption{Summary of construct analysis}
\label{Table 2:}
\begin{tabular}{clcc}
\hline
Theme & \multicolumn{1}{c}{\begin{tabular}[c]{@{}c@{}}Construct pair examples\\ (Ordered Human/IPA)\end{tabular}} & \begin{tabular}[c]{@{}c@{}}Number of construct pairs\\ (\% of all constructs generated)\end{tabular} & \begin{tabular}[c]{@{}c@{}}\% of agreement \\ between raters\end{tabular} \\ \hline
Partner Knowledge Set & \begin{tabular}[c]{@{}l@{}}Opinionated/Non-judgmental; \\ Biased/Neutral; Free-Bookish; \\ Expansive/Limited; \\ Ad-hoc/Spontaneous/Pre-programmed; \\ Colloquial/Universal knowledge; \\ Abstract/Specific knowledge; \\ Lateral/Inflexible thinking\end{tabular} & \begin{tabular}[c]{@{}c@{}}N=72\\ (27.1\%)\end{tabular} & 95.8\% \\ \hline
Interpersonal Connection & \begin{tabular}[c]{@{}l@{}}Personal relatability/Manufactured; \\ Genuineness/Ungenuine; \\ Real/Fake; Canny/Uncanny; \\ Emotional/Cold; Personal/Robotic; \\ Connection/Disconnected-disinterested; \\ Engaged/Remote; Humour/Humourless\end{tabular} & \begin{tabular}[c]{@{}c@{}}N=48\\ (18.1\%)\end{tabular} & 83.3\% \\ \hline
Linguistic Content & \begin{tabular}[c]{@{}l@{}}Short answers/Long answers; \\ Safe/Edgy; Expansive/To-the-point;\\ Convenience/Inconvenience; \\ Elaborate/Pointed; Polite/Blunt or rude; \\ Slang/Phrasing; Colloquial/Formal; \\ Vague/Detailed\end{tabular} & \begin{tabular}[c]{@{}c@{}}N=41\\ (15.4\%)\end{tabular} & 51.2\% \\ \hline
\begin{tabular}[c]{@{}c@{}}Partner Performance \\ and Capabilities\end{tabular} & \begin{tabular}[c]{@{}l@{}}Channels of communication Multiple/Single; \\ Speed of response: Slow/Fast; \\ Hesitating/Confident-decisive; \\ Recognition-Understandability (High/Low)\end{tabular} & \begin{tabular}[c]{@{}c@{}}N=21\\ (10.2\%)\end{tabular} & 63.0\% \\ \hline
Conversational Interactivity & \begin{tabular}[c]{@{}l@{}}Two-way/One-way; Conversive/Monologue; \\ Leading/Uninteractive; \\ Conjunctive/Disconnected; \\ Continuation/Stop \& Start conversation; \\ Conversational speech (fluid)/Stilted speech; \\ Keeping track/Isolated responses\end{tabular} & \begin{tabular}[c]{@{}c@{}}N=24\\ (9\%)\end{tabular} & 87.5\% \\ \hline
\begin{tabular}[c]{@{}c@{}}Partner Identity \\ \& Role\end{tabular} & \begin{tabular}[c]{@{}l@{}}Humanness/Machineness; \\ Real/Organic-Artificial; \\ Personalised/Commercialised; \\ No agenda/Agenda; \\ Transparency (purpose) (Low\textbackslash{}High); \\ To help/To serve; Level standing/Slave\end{tabular} & \begin{tabular}[c]{@{}c@{}}N=21\\ (7.9\%)\end{tabular} & 100\% \\ \hline
Vocal Qualities & \begin{tabular}[c]{@{}l@{}}Pronunciation/Incomprehensible; \\ Consistent/Choppy; \\ Pleasant/Unpleasant; \\ Audibility-Clarity of speech (High/Low); \\ Enthusiasm/Monotonous; \\ Speech modulation/No modulation; \\ Cheery \& Emotional/Dull \& Emotionless\end{tabular} & \begin{tabular}[c]{@{}c@{}}N=17\\ (6.4\%)\end{tabular} & 64.7\% \\ \hline
Behavioural Affordances & \begin{tabular}[c]{@{}l@{}}Natural/Conscious; \\ Familiarity/Unfamiliarity; Patience/Impatience; \\ Easy communication/Limited recognition \\ (make allowances for)\end{tabular} & \begin{tabular}[c]{@{}c@{}}N=13\\ (4.9\%)\end{tabular} & 84.6\%. \\ \hline
Miscellaneous & \begin{tabular}[c]{@{}l@{}}Fair/Unreasonable; \\ Calculated/General; \\ Personal/Impersonal\end{tabular} & \begin{tabular}[c]{@{}c@{}}N=3\\ (1.1\%)\end{tabular} & 100\% \\ \hline
\end{tabular}
\end{table*}

\subsection{Partner Knowledge Set}
72 of the constructs participants generated are focused on the perceived types of knowledge and information that each of the partners can supply during dialogue. Participants said that, unlike humans, speech-based IPAs’ knowledge tends not to extend to subjective opinions or judgements. The IPAs’ knowledge base was also perceived as more factual, whereas people were seen as relying on more experience based knowledge:

\begin{center}
\emph{"...they provide very fact and evidenced based knowledge where people will give experience based knowledge" [P22]}
\end{center}

On occasions when a speech-based IPA did try to convey a subjective preference, responses were varied:

\begin{center}
\emph{"Alexa’s opinion on Rock music, it said it liked Queen and I think Bohemian Rhapsody...felt very human-like" [P15]}
\end{center}

Others felt these interjections made it feel like the IPAs were trying too hard to be human-like, and this served to highlight the role of the team who programmed the IPA to create an illusion of subjective knowledge on a particular theme:

\begin{center}
\emph{"When Alexa made an X-Files reference I immediately thought of whatever Silicon Valley guy on his lunch break on a Tuesday was like 'Oh, I'm gonna make an X-Files reference'...it was almost trying too hard, it took me out of it"[P1]}
\end{center}

Essentially, participants saw IPAs' knowledge as objective \textit{“purely informative types of things”} [P3] that \textit{“just pull up a Wikipedia page...as quickly as possible”} [P1]. Conversely, humans were seen as having an ability to draw on richer forms of information meaning they could \textit{“talk for like ten minutes”} if asked \textit{“what do you think of David Bowie?”} or on a topic like \textit{“not even whether he was a good person but what is a good person”}[P1]. This leads to them being perceived as able to give \textit{“complex”} [P11,P22], and \textit{“personalised”} [P20,P21] answers to potentially complicated queries. Humans are seen as more able to contextualize, make use of socially relevant and colloquial knowledge, \textit{“make assumptions”} [P19] and draw supplementary information from the conversation as it unfolds:

\begin{center}
\emph{"A big part of it is actually talking to someone who knows the greater context...to know the greater context of everything that’s going on around you, I guess that’s something the virtual assistants just can’t do right now" [P10]}
\end{center}

Humans were regarded as \emph{“spontaneous”} [P9], \emph{“nuanced”} [P14], and \emph{“interpretive”} [P22]. Yet, although speech-based IPAs were seen as having access to a greater \emph{“depth of information”} [P5], their \emph{“very bookish”} nature meant they were perceived as more \emph{“constrained”} [P5] and  \emph{“literal”} [P9,P22]:

\begin{center}
\emph{“When I asked what [the human partner] thought about Chuck Norris she was saying like, ‘Oh, he’s very conservative’. The political aspect came into it...but say if Chuck Norris likes Pizza, well they [the IPAs] wouldn’t be able to say then maybe he likes cheese as well, maybe he likes tomatoes as well. They just know he likes Pizza” [P12].}
\end{center}

\subsection{Interpersonal Connection}
48 constructs generated emphasized perceived qualities associated with the development of interpersonal relationships between dialogue partners. A number of participants noted that, compared to the interaction with a human, interaction with both Siri and Alexa felt \textit{“fake”} [P10] or \textit{“completely false"} [P14]. This was sometimes amplified by \textit{“attempts to be funny (and) relatable”} [P14], that were interpreted as dry, boring and unfunny and generally seen as an attempt to simulate an artificial social connection. The issue of authenticity of the interaction also contributed to a sense of strangeness or eeriness. One participant remarked that they wanted these systems to have more humanlike qualities, but the eeriness of the current interpersonal interaction was also noted:

\begin{center}
\emph{"One of the biggest things to me is like ‘is this thing humanlike?’ And then on the other side I'd say like uncanny...there were times when I felt I was sitting deep in the uncanny valley when I’m talking to these things"  [P10]}
\end{center}

Some noted a lack of inherent emotion on the part of the IPAs when giving information, whereas it felt that humans generally express inherent emotion about a subject by default when conversing. Again the authentic nature of the interactional qualities were brought into question, where people felt that, although IPAs projected emotion at points, that \textit{“they were only pretending”} [P22].

Empathy and engagement were also prominently mentioned.  IPAs were described as largely devoid of the ability to show warmth compared to a human partner. The IPAs felt \textit{“quite disengaged compared to the human”} [P24], especially when responding to \textit{“personal questions”} like \textit{“where are you from? And how are you today?”} [P24]. Alexa’s tendency to \textit{“churn out answers”} [P24], meant it felt less \textit{“relatable”} and less \textit{“human-like”}. Likewise the IPAs’ \textit{“lack of feeling and lack of humor”} [P5] made them seem like \textit{“disinterested”} [P16] dialogue partners. Although some participants did characterise all the partners they interacted with as friendly, there were clear deficits in how relatable and interpersonal the IPAs were compared to the human partner.

\subsection{Linguistic content}
41 constructs generated focused on the structure and tone of linguistic content generated by each partner. IPAs' linguistic output was generally seen as formal, \textit{“always using full sentences”} [P4], along with being \textit{“polite”} [P20], \textit{“diplomatic”} [P24] and \textit{“safe”} [P11]. Between the IPAs, Alexa’s use of humor did make it seem less formal than Siri for some participants. However, human-like social content such as small talk was often seen as excessive, inappropriate, \textit{“obscure”} and \textit{“a bit weird”} [P19]. Indeed there was a strong mistrust when Alexa’s output involved opinion rather than formal fact based content.  For instance, one participant stated they felt \textit{“manipulated”} [P14] when Alexa expressed a favorite band and song.

Participants also felt that human partners used a more narrative rather than descriptive style to convey responses. Although at points, IPAs were perceived as direct and to-the-point in their responses, Alexa was commonly criticized for providing too much information to queries compared to human partners and Siri.

\subsection{Partner Performance \& Capabilities}
27 of the constructs were categorised as describing functional aspects of the way dialogue partners performed. Many of these emphasised the multimodal nature of communication and the functional limitations of IPAs. The most significant observations made by participants related to the importance of feedback through non-verbal communication in generating a sense of humanness. Specifically, they felt that communication through speech-only led to ambiguity as to whether their partner had recognised and understood what they had said. Siri’s use of visual feedback (e.g. presenting automatic speech recognition (ASR) output on screen) was particularly useful in reducing this ambiguity, and was compared to feedback given from human partners through cues such as facial expressions or hand gestures. Some felt Siri outperformed the human and Alexa in giving confirmation of understanding:

\begin{center}
\emph{“Well yeah, everything was oral for Alexa, umm, and human, whereas Siri provided visual context as well...it's like a multifaceted approach" [P13]}
\end{center}

Others drew similarities between the efficiency of multimodal interactions with Siri and the use of  facial and hand gestures:
\begin{center}
\emph{“She (Human) could give me a physical representation as an answer...something else that I thought was interesting was because you have like the phone, like when it gives you information it doesn't tell you, it just pulls it up...So like people can use various methods but also Siri uses pictures and stuff, whereas Alexa can't do that." [P22]}
\end{center}

Despite some issues with ambiguity, recognition and understanding were seen as strong across elements. Yet there were clear differences in how the precision and accuracy of information being delivered by each partner was perceived:

\begin{center}
\emph{"If I were to take that (Human's) advice I wouldn't have known how to get to there because I wouldn't have known which bus to get, things like that" [P14]}
\end{center}

\begin{center}
\emph{“...the computers because they're automated don't make errors so there's a difference probably in the quality of information” [P24]}
\end{center}

The speed of response was also a clear point of divergence between the elements. Many participants noted that the IPAs were very quick to respond, whereas the human was sometimes hesitant:

\begin{center}
\emph{“...there's like a lead time when you speak to a human...like the, 'Umm' 'Ehh'; where they're thinking like its a pause for thought, whereas a computer doesn't have the pause for thought, so I suppose, speed of response quick" [P24]}
\end{center}

\subsection{Conversational Interactivity}
24 constructs related to how conversational participants felt interactions with each of the elements were. A number of the constructs related to perceptions of the direction and interactivity of communication in these contexts with IPAs containing, \textit{“very little conversation and more, like commands”} [P10]:

\begin{center}
\emph{"The human yep, very good conversation, very interactive, the others...it definitely didn't feel like it was a conversation. Like I was asking questions and waiting for a response" [P9]}
\end{center}

Participants perceived that the questions used in the study, when used with the human partner, were the start of a conversation that could unfold dynamically, rather than merely a request that was completed after an answer:

\begin{center}
\emph{“With the first question 'how are you today?', the human, kind of responded with 'yeah I'm good, how are you?', whereas, both Siri and Alexa were just an answer, and end of conversation” [P3]}
\end{center}

\begin{center}
\emph{“When I asked do you like rock music and she (Human) said, that depends what you mean by rock music, like that's the beginning of a conversation, it's not just an answer” [P1]}
\end{center}

This perceived lack of progression with IPA dialogue was consistently mentioned, with participants noting that to get more from the IPA you would have to \textit{“....ask them another question”} [P20]:

\begin{center}
\emph{“With the human...when they give an answer, you can go on again. But with them (IPAs) it's like, once they give an answer, that's it” [P18]}
\end{center}

Likewise the fluidity of the conversational interaction with IPAs was an issue:

\begin{center}
\emph{“Some of the things they (IPAs) say is like stilted. Like, for example when she was telling that joke she's kinda like pausing, but that's not natural” [P18]}
\end{center}

\subsection{Partner Identity or Role}
21 constructs focused on the partner’s identity and the role they play within the dialogue. On the whole these constructs highlighted a simple dichotomy of partner identity between human and machine. There were also reflections on the commercial nature of IPAs and how this influences functionality:

\begin{center}
\emph{“...I'm aware of the company, because I know these are products sold for money” [P1]}
\end{center}

\begin{center}
\emph{“Like Google Assistant, I probably linked it to Spotify but I'm pretty sure the first time I asked it to play music it was like you can open this on Google Music” [P6]}
\end{center}

The power dynamic between user and speech-based IPAs also formed part of the theme’s constructs:

\begin{center}
\emph{"Me and human are at the same level….when I'm (IPAs) asking the questions I don't really feel I'm asking the questions, I'm dictating for an answer...it's a master-slave type thing” [P14]}
\end{center}

On the other hand the IPA’s purpose was seen as very \textit{“transparent”} [P6] compared to humans who may have more complex motives and social roles within interaction.

\subsection{Vocal Qualities}
17 constructs emphasised aspects of non-lexical voice quality. This largely revolved around clarity of a partner’s voice. Participants felt that \textit{“their [IPAs] diction or their actual pronunciation was very sharp”} [P4], and that all partner’s voices were \textit{“...easy to understand. They're all comprehensible”} [P15]. This highlights improvements in speech synthesis and the development of clear and understandable voices.

Clear differences between partners were seen when discussing voice expressiveness. Humans we were seen to \textit{“express emotions when we talk”} but \textit{“when they (IPAs) say something, they say it really coldly. Whereas...from a human it would have some ups and downs, or like some voice modulations that Alexa and Siri doesn't have”} [P19].

Some did note the attempt to include expressiveness in the voices used by IPAs, feeling that they were \textit{“...all relatively cheery. They didn't sound, in any way, dull...(but) the human was much more emotionally cheery I suppose”} [P16].

\subsection{Behavioural Affordances}
13 constructs focused on affordances people felt they needed to make to account for limitations of their dialogue partners. The vast majority of these types of observations related to participants feeling they had to adapt the way they spoke when interacting with speech-based IPAs by trying to \textit{“.. pronounce my words more accurately”} due to consciously considering \textit{“whether they understood”} [P21].  Participants also felt that they could be \textit{“more informal, or maybe more spontaneous”} [P9] when interacting with a human, and that queries made to speech-based IPAs were less natural and had to be structured in a particular way:

\begin{center}
\emph{“I know how to ask a question to get the information that I want that wouldn't necessarily be the way you would converse with a human” [P10]}
\end{center}

This was a particular issue for some who felt interactions with IPAs had rules that needed to be learned:

\begin{center}
\emph{“I wasn't familiar with like, how, to interact with them, I didn't know the rules” [P3]}
\end{center}

Finally, some participants suggested they would be more forgiving and patient in accommodating communicative limitations of humans compared to IPAs.

\section{Discussion}
Research \cite{cowan2017they,cowan_voice_2015,luger_like_2016,porcheron_voice_2018,dubiel_survey_2018,moore_progress_2016} has consistently identified that humanness is fundamental to speech interface design, supporting the metaphor of human dialogue for interaction. It is currently unclear what dimensions are important in users’ conceptualisation of humanness and how people see speech interface functionality through this lens. Through using the Repertory Grid Technique, we discover eight key dimensions which are the focus of humanness: 1) a partner’s level and type of knowledge or \textit{partner knowledge set}; 2) the potential for \textit{interpersonal connections}; 3) the \textit{linguistic content} they use in their dialogue turns; 4) the presence or absence of \textit{conversational interactivity}; 5) functional attributes that relate to a \textit{partner’s performance and capabilities} to complement dialogue; 6) a coarse grained judgement of their \textit{identity and role} in the dialogue, 7) the expressiveness and clarity of their \textit{vocal qualities} in the speech they produce and 8) \textit{behavioural affordances} made to accommodate for perceived limitations of dialogue partners. These dimensions, based on categorisation of user-generated constructs, echo previous researcher led attempts to map fundamental differences between humans and machines \cite{haslam2006dehumanization} (e.g. ‘interpersonal connection’ is similar to ‘emotional responsiveness vs inertness’, and ‘partner knowledge set’ mirrors ‘cognitive openness vs rigidity’).

Although people may use humanness as an anchor \cite{luger_like_2016,cowan2017they} people make clear distinctions between humans and speech-based IPAs on these dimensions. Participants see IPAs as being more fact based than opinionated in the knowledge they have, with human partners being able to combine knowledge and ideas in novel ways to contribute to dialogue. Attempts made by speech-based IPAs to be socially relatable and to create a connection were regularly seen as fake, with IPAs seen as lacking in emotion or perceived interest in the dialogue. IPAs were also perceived as far more formal and verbose in the volume of content they deliver, with social or opinion based content being viewed with suspicion. Differences in the levels of feedback between humans and IPAs were also expressed, whereby speech-only devices were limited when compared to partners that use visual or other multimodal signals to convey understanding. Speech-based IPAs were judged as more able to return accurate and faster responses compared to human partners. Whilst human dialogue was two way and more expansive in nature, IPAs by comparison seemed limited to question-answer type interactions. Above the clear human/machine dichotomy participants also perceived that IPAs were built to serve the user, with humans having more complex social roles and purposes when conversing. Although voice clarity did not vary between partners, there were clearly perceived differences in expressiveness and paralinguistic features, whereby the speech-based IPAs seemed cold and less emotive than a human. Participants also noted a perceived lack of flexibility and a need to understand the rules of interaction when communicating with speech-based IPAs, with humans seen as more adaptable to how the user may want to converse.

\subsection{The multi-dimensional nature of humanness}
Our major contribution is in identifying the key dimensions that users use to conceptualise humanness and how these are mapped to speech-based IPA interactions. This adds much needed richness to the view of humanness as an influencer of the speech-based IPA user experience \cite{cowan_what_2017,luger_like_2016}, by breaking down the concepts, identifying important dimensions, and identifying the similarities and differences in how these are applied to humans and speech-based IPAs. From our findings it is clear that humanness needs to be considered not as unidimensional, but as a multi-dimensional concept in speech interface design. The primary purpose of the analysis was not to generate design recommendations, rather it sets to identify the dimensions that influence perceptions of humanness and how they differ in human and IPA interaction. That said, designers could use these dimensions to inform decisions and areas of focus. For instance, our themes could aid in understanding how users frame their experience when compared to human interaction, thereby identifying pros and cons of using humanness as a design metaphor. Striving for humanness is a common guiding principle in speech interface design \cite{aylett2019siri} and a heuristic users frame their experiences through \cite{cowan_what_2017,leahu_how_2013, luger_like_2016}. A nuanced understanding of how users comprehend their interactions through this heuristic is valuable to inform design decisions. With regards to research, a number of themes do indeed echo topics within speech interface research. For instance, vocal quality clearly maps to work on speech synthesis, where developing more human-like, expressive \cite{akuzawa_expressive_2018}, emotive \cite{meddeb_what?_2010} and personality-filled \cite{wester_artificial_2015} voices is currently underway. User research has also focused on exploring the role of humanness in partner knowledge assumptions \cite{cowan2017they}, vocal quality \cite{cowan_voice_2015,lee_social-psychological_2005}, partner identity \cite{branigan_role_2011}, linguistic content \cite{clark_what_2019} and conversational interactivity \cite{clark_what_2019,purington_alexa_2017}. Our work not only identifies the important dimensions of humanness when considering user perceptions, but formalises intuitions by researchers on the types of design decisions that may affect these perceptions.
\subsection{Issues with emulating aspects of humanness}
Our findings show that humans and speech-based IPAs are clearly perceived differently on the dimensions identified. IPA interaction is seen as predominantly unidirectional and as a master-servant relationship compared to human-based dialogue interaction. They are also perceived as more socially fake and limited in their interpersonal ability than human partners, as well as seeming more formal and specific than human interlocutors.

This supports the notion that, although they may be perceived through the lens of human dialogue, speech interface dialogues are fundamentally distinct \cite{porcheron_voice_2018, clark_what_2019}. Currently, dialogue interactions with speech-based IPAs tend to be isolated question-answer pairs \cite{porcheron_voice_2018}, seemingly functional rather than interpersonal or social in nature \cite{clark_what_2019}. It may be argued that this is because of truly natural aspects of dialogue not being implemented appropriately or at all in the types of devices currently available. Especially in terms of interpersonal connections and conversational interaction, current projects are exploring how to include this in speech interface design \cite{spillane_introducing_2017,devillers2018multifaceted,fang2017sounding,papaioannou2017alana} meaning this may become more common and implemented more effectively. Our data suggests some fundamental barriers to accepting such an interaction. Small talk was sometimes flagged as inappropriate or undesirable for current speech-based IPA partners. More social features were  perceived as inauthentic, owing to the perceived pre-programmed nature of these systems, rather than because of the types of interactions speech-based IPAs were designed for. Although context and the purpose of the  interaction may play a role in the acceptability of social functionality, this type of functionality may be pushing the fundamental limits of spoken interactions between humans and machines \cite{clark2018social,moore_progress_2016}.

That is not to say that certain dimensions of humanness identified in our work should not be considered in future speech interface design. Concentrating on other dimensions of human-human communication may be more appropriate and even beneficial to user experience for speech interfaces interactions, including speech-based IPAs. In particular for IPAs, our participant highlighted making minor adjustments towards more human-like ways of delivering linguistic content,  like favouring brevity in message delivery and including multimodal feedback available in dialogue. This may improve the interaction experienced. Aspects such as developing expressive vocal qualities may also be useful in more accurately communicating emotive content through synthesis.

\section{Limitations \& Future Work}
Because of the aim of the work, and the triading paradigm inherent to RGT, participants were asked to make direct comparisons between a human and two speech-based IPAs using the human as comparator. This meant that the human was highly salient in the comparisons being made between the three elements, potentially influencing which differences are prioritised. The purpose of this salience was to elicit dimensions and constructs particularly focusing on how humanness may be framing dimensions that people consider in speech-based IPA dialogue. This was so as to reflect findings in the literature that emphasise the important of humanness in design and the role this may have in mental model development \cite{luger_like_2016,cowan2017they}.
Future work could include more elements to limit the impact that each may play in guiding user constructs.

Our study involved participants reflecting on their interaction with the three elements. This interaction revolved around nine pre-determined questions. This technique was useful to expose participants to particular interaction issues and to ensure all participants could reflect on similar experiences when eliciting constructs. Yet it may have limited the types of dialogues participants reflected on, especially when interacting with the human dialogue partner. Future work may look to use more free-form interactions with all partners, which may not only replicate the categories for constructs found in this research, but add to their richness.

It is also important to note that our study gives a snapshot of user perceptions on these dimensions, yet does not provide information as to how these may be fluid over an interaction, change with different experiences, or how these may change from initial perceptions before interaction. Mapping this dynamism has been noted as a consistent challenge in partner model and perspective taking research, in both human \cite{branigan_role_2011} and machine \cite{cowan_voice_2015} dialogue. Future approaches to this topic should focus on novel methods to gain a view on the dynamics of user perceptions. The themes identified may also act as a starting point to develop a measure to allow researchers and designers to quantify the impact of design choices on the dimensions of humanness. Validated metrics to quantify user perceptions are needed in the field \cite{clark2018state}. We plan to use this work as a starting point for developing such a metric.

\section{Conclusion}
Concepts of humanness are core to the design of speech interfaces like IPAs, yet the specific dimensions of humanness that people use to define these interactions are not fully understood. Our study has clearly outlined key themes related to how users view humanness in dialogue interaction and how this varies in speech-based IPA dialogue. It highlights that whilst humanness may be integral to speech interface design, significant thought needs to be placed into how humanness may be achieved and implemented with sensitivity to the specific dimensions identified.

\begin{acks}
This research was supported by an employment based PhD scholarship funded by the Irish Research Council and Voysis Ltd (R17830).
\end{acks}

\balance
\bibliographystyle{ACM-Reference-Format}


\section{Introduction}

The existence of dark matter provides unambiguous evidence of new physics
beyond the Standard Model (SM) of particle physics. Indeed, given that there is
far more dark matter in the Universe than there is baryonic matter, determining
its precise nature is one of the most exciting challenges in particle physics
today.  One promising group of candidates for dark matter is the class of stable
particles known as weakly interacting massive particles (WIMPs). These
particles arise naturally in many extensions of the SM that are motivated by
the need to stabilize the electroweak scale. One of the most heavily studied extensions
is the Minimal Supersymmetric Standard Model (MSSM), which predicts a
`supersymmetric partner', or \emph{superpartner}, for each SM
particle\footnote{For a detailed review of the MSSM, see Ref.
\cite{Martin:1997ns}.}. In the MSSM with \emph{R}-parity conservation, the
lightest supersymmetric partner (LSP) is predicted to be absolutely stable,
making it a good candidate for dark matter. The identity of the LSP is
determined by the mass hierarchy of the superpartners, which in turn depends on
the mechanism of supersymmetry breaking.  However, taking into account experimental
constraints and phenomenological considerations, the lightest of the MSSM
particles known as the neutralinos emerges as the most attractive candidate
for the LSP \cite{Bertone:2004pz}.

Though a natural MSSM spectrum has the potential to tame the hierarchy problem,
it is under siege from recent data from the Large Hadron Collider (LHC)
\cite{Aaboud:2018ujj, Sirunyan:2018vjp}.  A compelling alternative scenario
comes in the form of split supersymmetry (split SUSY) \cite{Wells:2003tf,
ArkaniHamed:2004yi, Giudice:2004tc}. In this scenario, the lightest
superpartners are the fermionic ones, on the scale of 1-10 TeV, while the
scalar superpartners can be much heavier, on the scale of 100 - 1000 TeV. In
exchange for accepting some level of fine-tuning, we obtain numerous benefits,
including the suppression of flavor-changing neutral currents and greater
compatibility with data from CP-violation experiments.  The low lying fermionic
superpartners consist of neutralinos and charginos (collectively known as
electroweakinos), and gluinos. The current LHC search limits on gluinos are
already around 2 TeV for prompt decay and 2.5 TeV for disappearing track
searches~\cite{CMS-PAS-SUS-19-005}.   The limits on electroweakinos with heavy
scalar superparticles, however, are still relatively weak: around 650 $-$ 750
GeV or less~\cite{ATL-PHYS-PUB-2019-022, Sirunyan:2017lae, Aaboud:2018htj,
Sirunyan:2018ubx, Aaboud:2018zeb}, depending on the decay modes of the parent
electroweakinos and their species.  In this paper, we will focus on the
discovery potential of the neutralinos and charginos at a 100 TeV $pp$
collider.

The decay patterns of the electroweakinos (Winos, Bino, and Higgsinos) depend
on the hierarchy of the MSSM parameters $M_1, M_2$, and $\mu$~\footnote{For our
study, we assume all the mass parameters are positive.  For the current
analyses, the results do not depend much on the sign of those parameters.},
which roughly determine the masses of the Bino, Winos, and Higgsinos,
respectively~\cite{Han:2013kza}.  In this paper, we consider the Higgsino NLSPs
and Bino LSP scenario with $M_1 < \mu \ll M_2$.  One of the motivations for
studying Higgsinos is that the parameter $\mu$ that governs their masses plays an
important role in electroweak symmetry breaking~\cite{Acharya:2014pua}, and
is driven to be relatively small by naturalness considerations.
Finding Higgsinos has traditionally been more challenging than finding Winos,
due to their smaller production rates.
This, however, can be remedied by the increased production cross sections and
luminosities at a 100 TeV $pp$ collider.

Since the Bino is the only supersymmetric particle lighter than the Higgsinos in
the mass hierarchy we consider, the neutral Higgsinos will decay to a Bino and a
neutral SM boson (\emph{Z/h}) with a branching fraction of 100\%.  Searches for
Higgsino NLSPs with Bino LSPs via  multi-lepton final states have been studied
in Ref.~\cite{Gori:2014oua} for a 100 TeV $pp$ collider, for pair production of
$\widetilde{\chi}^0_{2,3}\widetilde{\chi}^\pm_1$ and
$\widetilde\chi_{1}^+\widetilde\chi_1^-$. In this paper, we attempt to exploit
the relatively large branching fraction of the SM Higgs boson to \emph{b} quarks,
and study the potential of the $\widetilde{\chi}^0_2\widetilde{\chi}^0_3\rightarrow Zh
\widetilde{\chi}^0_1\widetilde{\chi}^0_1\rightarrow bb\ell\ell {\slashed{E}_T}$ channel.

A 100 TeV $pp$ collider would open up an immense number of physics
opportunities not afforded to the 14 TeV LHC, including exploring the most
interesting regions of the split SUSY parameter space
\cite{Arkani-Hamed:2015vfh}.
Such a collider would be a natural next step after the LHC, and is being
actively discussed in the particle physics community, with two major proposals
being the Future Circular Collider (FCC)-hh by CERN \cite{FCC-hh}, and the
Super Proton Proton Collider (SppC)  in China \cite{CEPC}.  As the next energy
frontier machine, it is crucial to fully explore the physics
potential of a 100 TeV $pp$ collider, especially for new particles that are
either too heavy or too rare to be produced at the
LHC~\cite{Arkani-Hamed:2015vfh,Contino:2016spe,Golling:2016gvc,Mangano:2016jyj}.
In particular, studies have already been performed on the prospects of
discovering neutralino dark matter for compressed \cite{Low:2014cba,
diCortona:2014yua,Cirelli:2014dsa,Mahbubani:2018tin,Han:2018wus} and
well-separated \cite{Gori:2014oua,Acharya:2014pua} neutralino spectra.

In our study, we use razor variables~\cite{Rogan:2010kb}, which were originally designed
for searches involving two heavy, mass-degenerate pair-produced
particles, each of which decays into a visible and invisible set of
particles. This topology matches that of our search channel, making this
set of variables a natural choice for our analysis.

At a 100 TeV collider, the  SM backgrounds are going to be even larger than
at the LHC.  An ancillary goal of this paper is to investigate the
potential for machine learning (ML) techniques to augment our analysis.
With the advent of more powerful computers and simultaneous advances in
the field of statistical learning in recent years, the usage of ML is
rising in experimental particle physics -- in fact, the discovery of the
SM Higgs boson in 2012 was done with the help of neural networks
\cite{Aad:2012tfa} and boosted decision trees (BDTs)
\cite{Chatrchyan:2012xdj}.

The rest of the paper is structured as follows. In Sec.~\ref{sec:model}, we
describe our model and search channel in more detail, and review the existing
experimental search bounds. In Sec.~\ref{sec:analysis}, we describe our
analysis strategies for both traditional cut-and-count analysis and analysis
performed using BDTs. In Sec.~\ref{sec:results}, we present the results of our
analyses, i.e. the expected 5$\sigma$ discovery and 95\% C.L. exclusion
reaches. Finally, we conclude with the implications in
Sec.~\ref{sec:conclusion}.

\section{The neutralino sector of the MSSM}
\label{sec:model}

The neutralino sector of MSSM consists of four mass eigenstates
($\widetilde{\chi}^0_1,\widetilde{\chi}^0_2,\widetilde{\chi}^0_3,\widetilde{\chi}^0_4$), which are mixtures of the Bino, Wino, and two neutral
Higgsinos: ($\widetilde{B}^0,\widetilde{W}^0,\H_d,\H_u$). In the basis of these gauge
eigenstates, the mass matrix of the neutralinos can be written as
\begin{align}
  \mathbf{M}_{\widetilde{N}}=
  \begin{pmatrix}
    M_1         & 0            & - c_\beta s_W m_Z &  s_\beta s_W m_Z \\
    0           & M_2          &  c_\beta c_{\theta_W} m_Z  & - s_\beta c_{\theta_W} m_Z \\
    - c_\beta s_W m_Z  &  c_\beta c_{\theta_W} m_Z    & 0          & -\mu \\
     s_\beta s_W m_Z & - s_\beta c_{\theta_W} m_Z & -\mu       & 0
  \end{pmatrix},
\end{align}
where we employ the notation $s_\theta, c_\theta = \sin\theta, \cos\theta$. Here,  $\theta_W$ is the Weinberg mixing angle,
and $\tan\beta= v_u/v_d$, where $v_u$ and $v_d$ are the vacuum
expectation values of $H_u^0$ and $H_d^0$, with $v_u^2+v_d^2=v^2=(246\ {\rm
GeV})^2$. The parameters $M_1$, $M_2$, and $\mu$ are the mass parameters of
the Bino, Wino, and the Higgsinos, respectively.
The mass eigenstates are labeled as $\widetilde{\chi}^0_i$ for $i=1 \ldots 4$ with increasing mass eigenvalues.
In the limit where $ m_Z \ll |\mu\pm M_1|, |\mu\pm M_2|,$ the mass eigenstates
are, to good approximation, a nearly pure Bino, $\widetilde{B}^0$, with mass $M_1$,
a nearly pure Wino, $\widetilde{W}^0$, with mass $M_2$, and nearly pure Higgsinos
$\H_{1,2} = (\H_u \pm \H_d)/\sqrt{2}$, with mass
$|\mu|$.

The optimal search strategy for finding electroweakinos   highly depends on
their mass spectra \cite{Han:2013kza}.  Phenomenological collider studies on
finding electroweakinos with a small mass splitting of about 0.1 $-$ 50 GeV at a
100 TeV $pp$  collider can be found in \cite{Low:2014cba, Bramante:2014tba,
Berlin:2015aba, Cirelli:2014dsa}.
For $3\ \text{ab}^{-1}$ of data, the projected exclusion reaches for pure Wino
LSPs using the monojet channel is about $1.4$~TeV.  The reach
can be increased to $\sim3$ TeV with a disappearing tracks search.
Neutralinos up to 1 TeV can be excluded for mass splittings of about 20-30 GeV,
using a soft lepton search \cite{Low:2014cba}. Additionally, Bino-Wino LSPs
with masses up to 1.5 TeV for an inter-state mass splitting of around 1 GeV  can be
discovered at a 100 TeV $pp$ collider with $\sim7\ \text{ab}^{-1}$
of data \cite{Bramante:2014tba}.

Well-separated spectra have been studied in \cite{Gori:2014oua,
Acharya:2014pua} using multi-lepton channels, of the form

\begin{align}
  pp\rightarrow \widetilde{\chi}_i\widetilde{\chi}_j\rightarrow VV +
  \widetilde{\chi}^0_1\widetilde{\chi}^0_1\rightarrow 2\ell/3\ell + {\slashed{E}_T},
\end{align}

\noindent where $\widetilde{\chi}_{i,j}$ can be a neutralino or chargino, $\widetilde{\chi}^0_1$
is the lightest neutralino, and $V$ is any of the SM gauge bosons $W$, $Z$
or scalar $h$.  These searches work well since the large mass difference
between the electroweakinos can lead to energetic leptons that can be easily
identified. Among the multi-lepton searches for Higgsinos, the trilepton
searches: $\widetilde{\chi}^0_{2,3}\widetilde{\chi}^\pm_1\rightarrow WZ\widetilde{\chi}^0_1\widetilde{\chi}^0_1\rightarrow \ell\ell\ell +
{\slashed{E}_T}$, with $\ell=e, \ \mu$,  have the best reach, due to the relatively high
production cross-section of chargino-neutralino pairs combined with the large
reduction in $t\overline{t}$ and QCD backgrounds from requiring three
leptons.  Higgsino NLSPs can be excluded at 95\% C.L. with mass up to 2.8 TeV
or be discovered with  mass up to 1.8 TeV, for a massless Bino LSP.   The reach
reduces for larger $M_1$, and vanishes for $M_1\gtrsim 600$ GeV (1000 GeV)
for discovery (exclusion).  Multilepton searches with \emph{Zh} and \emph{ZZ}
as the intermediate dibosons are unlikely to be as powerful, due to the smaller
pair-production cross-section of neutral Higgsinos, combined with the low
branching fraction of \emph{Z/h} to leptons. However, if we move beyond
multilepton searches, the channel with \emph{Zh} (originally suggested in
Ref.~\cite{Han:2013kza}) as the intermediate dibosons emerges as a possible
competitor to the \emph{WZ} channel.

\begin{figure}[h]
  \centering
  \includegraphics[width=0.49\textwidth]{images/xsection_plot.pdf}
  \includegraphics[width=0.49\textwidth]{images/br_plot.pdf}
  \caption{Higgsino pair production $\widetilde{\chi}^0_2\widetilde{\chi}^0_3$
    cross section (left panel), calculated using
    Prospino2~\cite{Beenakker:1999xh},  and decay branching fractions
    (right panel), calculated in SUSY-HIT~\cite{Djouadi:2006bz},  as a function
    of $\mu$ for $M_1=25$ GeV.
}
  \label{fig:xsection_plot}
\end{figure}

In the left panel of Fig.~\ref{fig:xsection_plot}, we show the pair-production
cross section $\widetilde{\chi}^0_2\widetilde{\chi}^0_3$ for the Higgsino-like NLSPs.  The cross section varies
between 200 fb to 0.5 fb for Higgsino masses between 500 GeV and 2500 GeV. The
right panel of Fig.~\ref{fig:xsection_plot} shows the decay branching fractions
of $\chi_{2,3}^0\rightarrow Z/h \chi_1^0$, as calculated by SUSY-HIT
\cite{Djouadi:2006bz}.  While one of the Higgsinos mostly decays to $Z$, the
other Higgsino mostly decays to $h$.    Therefore,  for pair produced Higgsinos
$\widetilde{\chi}^0_2\widetilde{\chi}^0_3$, the \emph{Zh} mode has the greatest branching fraction.  After
multiplying all the branching fractions, the signal cross sections of

\begin{align}
  pp\rightarrow \widetilde{\chi}^0_2\widetilde{\chi}^0_3\rightarrow (Z\widetilde{\chi}^0_1)(h\widetilde{\chi}^0_1)\rightarrow bb\ell\ell \widetilde{\chi}^0_1\widetilde{\chi}^0_1.
\end{align}

\noindent become comparable to the trilepton processes, as shown in Table
\ref{tab:xsections}. Thus, the $Zh+{\slashed{E}_T}$ signal provides an alternative discovery
channel for Higgsinos. The signal contains two \emph{b}-jets, two same flavor,
opposite sign leptons, and missing transverse energy. The main backgrounds for
this process are: $t\overline{t}$, \emph{tbW} with the $b$ and $W$ not coming
from a \emph{t}, and \emph{bbWW} with no intermediate $s$-channel top quarks.

\begin{table}
  \centering
  \begin{tabular}{l|rr}
    \toprule
    Stage & $\widetilde{\chi}^0_{2,3}\widetilde{\chi}^\pm_1$ & $\widetilde{\chi}^0_2\widetilde{\chi}^0_3$\\
    \midrule
    Pair production cross section & 60 fb & 17.2 fb\\
    Intermediate diboson contribution & ($WZ$) 29.7 fb  & ($Zh$) 9.15 fb \\
    Applying \emph{BR}$(W\rightarrow \ell\nu)$, \emph{BR}$(Z\rightarrow \ell\ell)$ \& \emph{BR}$(h\rightarrow bb)$ & 0.42 fb & 0.37 fb\\
    \bottomrule
  \end{tabular}
  \caption{Comparison of Higgsino pair production cross-sections for
    $(\widetilde{\chi}^0_{2,3}\widetilde{\chi}^\pm_1)$ and
    $(\widetilde{\chi}^0_{2}\widetilde{\chi}^0_{3})$, for $\mu\approx 1$ TeV, at a 100
    TeV \emph{pp} collider.
    The pair-production cross section for $\widetilde{\chi}^0_{2,3}\widetilde{\chi}^\pm_1$ is taken from
    \cite{Gori:2014oua}, while the pair-production cross-section for $\widetilde{\chi}^0_2\widetilde{\chi}^0_3$
    is calculated at Next-to-Leading Order (NLO) using Prospino2. The branching
    ratios to the diboson intermediate states are calculated using SUSY-HIT,
    and the branching fractions to the SM final states are taken from
    \cite{Olive:2016xmw}.}
  \label{tab:xsections}
\end{table}

Searches for pair produced nearly mass-degenerate Higgsinos  have been
performed at ATLAS and CMS for various decay topologies
\cite{Aaboud:2018htj,Sirunyan:2018ubx,Aaboud:2018zeb}.
For neutralino-chargino pair production with \emph{WZ} and \emph{Wh} topologies, the 95\%
C.L. exclusion limits are about 650 GeV, 490 GeV, and 535 GeV for \emph{WZ}, \emph{Wh} and an equal mixture of \emph{WZ}+\emph{Wh} topology, with the strongest limits coming from
the multi-lepton searches.  For neutralino pair production with \emph{ZZ, Zh}, and \emph{hh} topologies\footnote{While such searches assume a gravitino LSP with Higgsino NLSPs, the limits are still relevant for our Higgsino NLSPs - Bino
LSP scenario.}, the limits are about 650 $-$ 750 GeV via multi-lepton or
multi-$b$ jets searches, depending on the decay branching fractions of the Higgsino NLSPs.

\section{Analysis Details}\label{sec:analysis}

In this section, we  describe our strategies for both the traditional
cut-and-count analysis, as well as the analysis carried out with boosted
decision trees.

\subsection{Simulation}\label{simulation}

We simulated parton-level events using \texttt{MadGraph5 v2.3.2.2} and
\texttt{MadEvent} \cite{Alwall:2014hca}, then passed those events to Pythia 6
\cite{Sjostrand:2006za} for showering and hadronization. Finally, we used
\texttt{Delphes 3} \cite{deFavereau:2013fsa} to perform a fast, parametrized
detector simulation, with the detector card devised by the FCC-hh working
group\footnote{\url{https://github.com/HEP-FCC/FCCSW/blob/master/Sim/SimDelphesInterface/data/FCChh_DelphesCard_Baseline_v01.tcl}}.
For the backgrounds, we allowed up to one additional jet in the final state, to
approximate NLO QCD effects, and performed MLM matching with the \texttt{xqcut}
parameter set to 40 GeV. The Higgsino pair production cross sections were
calculated using \texttt{Prospino2}~\cite{Beenakker:1999xh} at NLO.
To decouple the Wino, we set its mass parameter $M_2$ to 3 TeV.

Since we expect our signal process to have a dilepton resonance from an on-shell
$Z$ boson, we restricted the phase space for event generation for backgrounds to
the region where the invariant mass of dilepton pairs lies between 80 and 100
GeV. Additionally, the Bino dark matter that escapes the detector would result
in a large amount of missing transverse energy (${\slashed{E}_T}$), so we required
a minimum ${\slashed{E}_T}$ of 100 GeV at the parton level for the backgrounds as
well.

At the detector simulation level, we relaxed the lepton isolation criterion in the
\texttt{Delphes} detector card from $\Delta R_{min}$ from 0.4 to 0.05. This is
motivated by the fact that due to the large mass difference between the Higgsino
NLSPs and the Bino LSP in our search channel, the intermediate $Z$ bosons will be
highly boosted, and the leptons to which they decay will be highly collimated.
The value of 0.05 is consistent with what is suggested in previous 100 TeV
studies~\cite{Acharya:2014pua,Gori:2014oua,Bramante:2014tba} and will allow
for easier comparison between different search strategies.

\subsection{Analysis using cut-and-count}
\label{event-selection}

For the cut-and-count analysis, we implemented successive one-dimensional cuts
on the variables listed below, using the MadAnalysis 5 package
\cite{Conte:2012fm}.

\begin{enumerate}
  \tightlist
  \item \emph{Trigger}: Events were selected if they had at least one lepton
    with $p_{T}^\ell >$ 100 GeV.

  \item \emph{Identification:}

    \begin{itemize}
      \item Events contain exactly two leptons of the same flavor
        and with opposite signs (\textsc{sfos}), with $p_{T}^\ell >$ 15 GeV and
        $\vert\eta^\ell\vert <$ 2.5.
      \item Events contain at least two \emph{b}-tagged jets with
        $p_{T}^b > 30$ GeV and $\vert\eta^b\vert <$ 2.5.
    \end{itemize}

   \item \emph{Missing Tranverse Energy:}  ${\slashed{E}_T}> 400$ GeV.

 \item \emph{Invariant mass of Z-candidate:}  $85\ {\rm GeV}<m_{\ell^{+}\ell^{-}}<95\ {\rm GeV}$.

  \item \emph{Invariant mass of h-candidate:} $75\ {\rm GeV}<m_{bb}<150\ {\rm GeV}$ for two leading $b$-jets.

  \item \emph{Razor variables}:  Two razor variables are used in our analyses:
\begin{align}
M_R &= \sqrt{(E_Z+E_h)^2 - (p_Z^z + p_h^z)^2},\\
M_T^R &= \sqrt{\frac{1}{2}\left[\slashed{E}_T(|\vec{p}_{ZT}|+|\vec{p}_{hT}|)
- \vec{\slashed{E}}_T\cdot(\vec{p}_{ZT}+\vec{p}_{hT})\right]}
\end{align}
 for $E_{Z,h}$ and $\vec{p}_{Z,h}$ being the reconstructed energy and momentum for $Z$ and $h$, respectively.
      Representative kinematic
  distributions of these variables for a signal benchmark point with $\mu=1$ TeV and $M_1=25$ GeV, as well as $tt$ and $tbW$ backgrounds are
  shown in Fig.~\ref{fig:razor_histos}, after the trigger and identification cuts.
  The distribution for the \emph{bbWW} background is not shown, since it is negligible in
  comparison to the others.   We can see that the distribution of $M_R$ is peaked
  around 1 TeV for the signal, which is consistent with what we would expect as
  it corresponds to the mass difference between the NLSPs and the LSP.  The $tt$
  background peaks around 500 GeV, while the distribution of $tbW$ background
  almost overlaps with that of the signal process.
  Similar behaviour is also seen in the $M_T^R$ distributions.
  For this mass combination, requiring $M_R >$ 800
  GeV and $M_T^R >$ 400 GeV yields the greatest significance.

\end{enumerate}

\begin{figure}[h]
\centering
\includegraphics[trim = {0.3cm 0.6cm 0.1cm 0}, clip, width=0.496\textwidth]{images/mR_copy.png}
\includegraphics[trim = {0.3cm 0.6cm 0.1cm 0}, clip, width=0.496\textwidth]{images/mTR_copy.png}
\caption{Normalized distributions of the razor kinematic variables $M_R$
  (left) and $M_T^R$ (right) for  a signal benchmark point with $\mu=1$ TeV and $M_1=25$ GeV,
  as well as the dominant $tt$ and $tbW$ backgrounds
  after the trigger and identification cuts.}
\label{fig:razor_histos}
\end{figure}

\begin{table}[h]
  \centering
  \input{tables/cutflowtable.tex}
  \caption{Representative cut flow table for a signal benchmark point with $\mu=1$ TeV,
    $M_1 = 25$ GeV at 100  TeV $pp$ collider, for a traditional cut-and-count analysis. All cross sections
    are given in unit of fb, and the units for the missing energy, invariant
    mass, and razor variable cuts are GeV. The significance, $S/\sqrt{B}$, is
    calculated for an integrated luminosity of 3 ab$^{-1}$.  }
\label{tab:cc_cutflowtable}
\end{table}

Table \ref{tab:cc_cutflowtable} shows the cut efficiencies for a representative
signal benchmark point with $\mu =$ 1 TeV and $M_1$ = 25 GeV.   The
cross-sections of the backgrounds are obtained from MadEvent after performing the MLM matching procedure. Their small size reflects
the $m_{\ell\ell}$ and $\slashed{E}_T$ cuts we imposed at the parton level. For all
the benchmark points, we require a minimum of 3 signal events left over after
cuts. We can see that, after applying our cuts, $tbW$ remains as the
dominant backgrounds, followed by $tt$.   The values of the razor variable cuts were chosen to
maximize the significance, $S/\sqrt{B}$ (calculated for an integrated luminosity
of 3000 fb$^{-1}$), shown in the last column.

\subsection{Analysis using gradient boosted decision trees}\label{subsec:bdt}

For each signal mass combination and each background process, we preselected
events that passed the lepton trigger, contained two SFOS leptons and two
$b$-tagged jets, and calculated a number of kinematic variables for each event.   We then
placed these kinematic variables in an array, with each row corresponding to an
event, and the columns corresponding to the kinematic variables. A mixture of
low-level and high-level kinematic variables was shown to have the greatest
effectiveness. The kinematic variables chosen were $m_{\ell\ell}$, $m_{bb}$,
$M_R$, $M_T^R$, $\slashed{E}_T,$ the total hadronic transverse energy $H_T$,
and the transverse momenta of individual final state leptons and $b$-quarks:
$p_T^{\ell_1}$, $p_T^{\ell_2}$, $p_T^{b_1}$, and $p_T^{b_2}$. We then divided
the generated events into training and test sets, with the training set
comprising 75\% of signal events and 30\% of background events.  We used the
\texttt{scikit-learn} package  \cite{Pedregosa2011} to implement our analysis.
A boosted decision tree classifier was trained with 1000 weak learners and a
learning rate of 0.025.

After the classifier is trained, we use it to assign scores to individual
events. A more negative score indicates a more background-like event, and a
more positive score denotes a more signal-like event. After the scores have
been assigned to the events in the test sets, we can use this score just as we
would a regular kinematic variable, that is, we apply a cut that selects
events with a minimum of this score. The value of the cut is chosen to maximize
the significance for each signal benchmark point. The distribution of scores
for the representative benchmark point ($\mu$ = 1 TeV, $M_1$ = 25 GeV) is shown in Fig.~\ref{fig:bdt_response}. We observe that there is an appreciable
separation between the signal and background distributions.

\begin{figure}[h]
\centering
\includegraphics[trim = {0 0.5cm 0 0},clip]{images/bdt_response_png}
\caption{Distribution of the decision function of the gradient boosted decision
tree classifier for   the representative signal benchmark point ($\mu = 1$ TeV, $M_1 = 25$ GeV) and backgrounds.}
\label{fig:bdt_response}
\end{figure}

\begin{table}[h]
  \centering
  \input{tables/paper_bdt_cutflowtable.tex}
  \caption{Representative cut flow table for the same benchmark point and
    integrated luminosity as in Table \ref{tab:cc_cutflowtable}, but using a BDT  analysis instead. The preselection is equivalent
    to the trigger and identification cuts listed in Table
    \ref{tab:cc_cutflowtable}. As before, all the cross sections are in
  fb.  }
\label{tab:bdt_cutflowtable}
\end{table}

Table \ref{tab:bdt_cutflowtable} shows the cut efficiencies for the same
representative benchmark point as in Table \ref{tab:cc_cutflowtable}, but this
time for a boosted decision tree analysis. We observe that the statistical
significance we can achieve goes up from 2.4 to 8.4, a roughly four-fold
increase.

For certain values of the
cuts on kinematic variables or the score of the classifier, no events survived
for one or more of the background components. To guard against overly optimistic
estimates of the significance $S/\sqrt{B}$, we set a lower bound of 3 for the
number of Monte Carlo (MC) events after cuts for each background component, corresponding to
the 95\% confidence interval for a Poisson distribution.

\begin{figure}[h]
\centering
\input{images/dm_100_TeV_contours.pgf}

\caption{ Discovery (red) and exclusion (blue) contours for the traditional
  cut-and-count analysis (solid) and boosted decision tree analysis (dashed),
  for 100 TeV $pp$ collider with an integrated luminosity of 3000 fb$^{-1}$.
}

\label{fig:contours}
\end{figure}

\section{Discovery and exclusion limits}
\label{sec:results}

Fig.~\ref{fig:contours} shows the expected reach  for 95\% C.L. exclusion and
5$\sigma$ discovery in the parameter space of $\mu$ vs.  $M_1$ for 100 TeV $pp$
collider with an integrated luminosity of 3000 fb$^{-1}$. The cut-and-count
strategy is able to discover Higgsinos up to 850 GeV  and exclude them up to
1.35 TeV.  The boosted decision
tree analysis improves this reach -- it is able to discover Higgsinos up to
1.4 TeV, and exclude them up to 1.8 TeV in favorable points in parameter space.
Similarly, Binos can be discovered up to 350 GeV and excluded up to 700 GeV
with the rectangular cut analysis, and the BDT analysis can discover them up to
about 900 GeV and exclude them up to 1.4 TeV. Since the background left after
cuts is relatively low, we do not include systematic errors in these estimates.

Comparing the BDT results with that of the cut-and-count bases analysis, the improvement is substantial, but perhaps not spectacular. However, it should
be kept in mind that this is probably a conservative estimate. Since we reserve
30\% of the background data for training purposes, and impose the condition that the
minimum number of MC events after cuts for the backgrounds must be 3, the true rate of background
rejection by the classifier is likely underestimated, and the expected
significance from the BDT analysis will improve. This extrapolation is
qualitatively supported by the high degree of separation between signal and
backgrounds seen in Fig.~\ref{fig:bdt_response}, and the fact that a larger
fraction of the signal events are preserved with the BDT approach compared to
the rectangular cut approach, as can be seen in Tables \ref{tab:cc_cutflowtable}
and \ref{tab:bdt_cutflowtable}.

In regions with large mass differences between the LSP and NLSP, the reach in $\mu$ is
reduced.  This is because the intermediate SM Higgs boson will be highly boosted, and
thus decay into a highly collimated pair of $b$-jets. The extent to which this
affects our results can be estimated from Table \ref{tab:cc_cutflowtable}. We
see that requiring two $b$-tagged jets reduces our signal cross-section by
roughly 87\%. This is far more than the 28\% reduction that one might naively
expect as a result of applying the $b$-tagging efficiency of about 85\%
specified in our \texttt{Delphes} card.

For massless Binos, the reach of the $Zh+{\slashed{E}_T}$ channel in our study is less than the reach of the multi-lepton channel studied in Ref.~\cite{Gori:2014oua},
which can exclude Higgsinos with masses up to 2.8 TeV and discover them for masses
up to 1.8 TeV.
However, for heavier Binos (larger $M_1$), the reach of the multi-lepton channel
is reduced, and vanishes completely for $M_1\gtrsim 600$ GeV (1000 GeV)
for discovery (exclusion). In this region, the reach of the $Zh+{\slashed{E}_T}$ channel in our study exceeds that of the multi-lepton search for larger LSP masses: an increasement of about 300 GeV with BDT analysis.

\section{Conclusion}\label{sec:conclusion}

In this paper, we examined the discovery/exclusion potential of a 100 TeV \emph{pp} collider
for pair-produced heavy Higgsino NLSPs with a Bino LSP, via intermediate \emph{Z} and
\emph{h} bosons that decay to a pair of leptons and \emph{b} quarks respectively:
$\widetilde{\chi}^0_2\widetilde{\chi}^0_3\rightarrow Zh \widetilde{\chi}^0_1\widetilde{\chi}^0_1\rightarrow bb\ell\ell {\slashed{E}_T}$.

We pursued two analysis strategies. One was the traditional cut-and-count methods, including razor variables
that are sensitive to the mass difference between the LSP and the NLSPs. The
other strategy was to use a boosted decision tree classifier trained with a
number of low- and high-level kinematic variables, including razor variables. We expected
that the machine learning approach would be able to more efficiently determine
the optimal decision boundary between signal and background events in feature
space than the traditional   cut-and-count method.

Overall, we find that the reach of our analysis strategy is a significant
improvement over that of the LHC. We found that the rectangular cut strategy has
the potential to discover Higgsino NLSPs up to a mass of 850 GeV, and exclude
them up to a mass of 1.35 TeV, for a Bino LSP mass around 350 GeV and 700 GeV
respectively.
As expected, the boosted decision tree classifier
performs better, with the ability to discover Higgsino NLSPs up to 1.4 TeV and
exclude them up to a mass of 1.8 TeV, for a Bino mass of about 900 GeV and 1.4
TeV respectively.   In the
process, we highlighted the importance of generating enough MC  events
to estimate the huge backgrounds at a 100 TeV $pp$ collider.

Additionally, we found that the reach for both strategies is considerably
reduced when the difference $ \mu-M_1$ is high, since it results in a highly
boosted $h$ that decays to a collimated pair of $b$-jets that will most likely be
identified as a single jet. This is not an insurmountable difficulty - there are
ways to deal with collimated jets, although they are beyond the scope of this
work. This issue does, however, highlight the necessity of improving isolation
performance at a 100 TeV $pp$ machine, where large mass hierarchies can result in
highly boosted/collimated decay products. For a review of currently used methods
to determine jet substructure, see \cite{Shelton:2013an}. In the future,
machine learning techniques might be profitably applied to this area as well -
for a review of developments along this line, see \cite{Schwartzman:2016jqu}.

The collimation of the $b$ jets, along with the fact that we simulate
detector effects using Delphes, result in a relatively lower reach in the high
$\mu$, low $M_1$ region than the multilepton analysis in~\cite{Gori:2014oua},
which is able to exclude Higgsinos up to a mass of 2.8 TeV for massless Binos,
or discover them up to a mass of 1.8 TeV. However, the reach of our analysis
is higher in the region where the difference between $\mu$ and $M_1$
is smaller. This is consistent with our usage of the razor variable $M_R$,
which is sensitive to the mass difference between the parent Higgsinos and the
daughter Bino.

A 100 TeV \emph{pp} collider represents an excellent opportunity to discover
physics beyond the Standard Model. The extremely high energies and luminosities
involved will present new challenges for particle physicists, and it is likely
that machine learning will play an important part in facing them.

\acknowledgments

We would like to thank Matt Leone and Ken Johns for helpful discussions. The
research activities of AP and SS were supported in part by the Department of
Energy under Grant DE-FG02-13ER41976/de-sc0009913. An allocation of computer
time from the UA Research Computing High Performance Computing (HPC) and High
Throughput Computing (HTC) at the University of Arizona is gratefully
acknowledged. We also thank KITP for its hospitality when this draft is completed.
This research was supported in part by the National Science Foundation under Grant No. NSF PHY-1748958.