You will be shown sections from a scientific research paper, together with a question about the paper. Your task is to answer the question based on the contents of the paper. Your response should be a `json` object with two fields:

"answer": An array of strings extracted directly from the paper which, collectively, answer the question.
"evidence": An array of strings. Each should be an excerpt from the paper, in which one or more of the extracted answers can be found.

For instance, for the question "What baselines did the authors compare against?", a sample response might be:

{
  "answer": ["BERT", "RoBERTa"],
  "evidence": ["In our experiments, we compare the performance of our model against BERT and RoBERTa."]
}

Do not include any text in your response other than the json.

If the question is unanswerable given the provided excerpts, respond with the single word "null".

Paper: Structured Embedding Models for Grouped Data

Word embeddings are a powerful approach for analyzing language, and exponential family embeddings (EFE) extend them to other types of data. Here we develop structured exponential family embeddings (S-EFE), a method for discovering embeddings that vary across related groups of data. We study how the word usage of U.S. Congressional speeches varies across states and party affiliation, how words are used differently across sections of the ArXiv, and how the co-purchase patterns of groceries can vary across seasons. Key to the success of our method is that the groups share statistical information. We develop two sharing strategies: hierarchical modeling and amortization. We demonstrate the benefits of this approach in empirical studies of speeches, abstracts, and shopping baskets. We show how S-EFE enables group-specific interpretation of word usage, and outperforms EFE in predicting held-out data.

Introduction
Word embeddings BIBREF0 , BIBREF1 , BIBREF2 , BIBREF3 , BIBREF4 , BIBREF5 , BIBREF6 are unsupervised learning methods for capturing latent semantic structure in language. Word embedding methods analyze text data to learn distributed representations of the vocabulary that capture its co-occurrence statistics. These representations are useful for reasoning about word usage and meaning BIBREF7 , BIBREF8 . Word embeddings have also been extended to data beyond text BIBREF9 , BIBREF10 , such as items in a grocery store or neurons in the brain. efe is a probabilistic perspective on embeddings that encompasses many existing methods and opens the door to bringing expressive probabilistic modeling BIBREF11 , BIBREF12 to the problem of learning distributed representations.
We develop sefe, an extension of efe for studying how embeddings can vary across groups of related data. We will study several examples: in U.S. Congressional speeches, word usage can vary across states or party affiliations; in scientific literature, the usage patterns of technical terms can vary across fields; in supermarket shopping data, co-purchase patterns of items can vary across seasons of the year. We will see that sefe discovers a per-group embedding representation of objects. While the naïve approach of fitting an individual embedding model for each group would typically suffer from lack of data—especially in groups for which fewer observations are available—we develop two methods that can share information across groups.
Figure FIGREF1 illustrates the kind of variation that we can capture. We fit an sefe to ArXiv abstracts grouped into different sections, such as computer science (cs), quantitative finance (q-fin), and nonlinear sciences (nlin). sefe results in a per-section embedding of each term in the vocabulary. Using the fitted embeddings, we illustrate similar words to the word 1.10intelligence. We can see that how 1.10intelligence is used varies by field: in computer science the most similar words include 1.10artificial and 1.10ai; in finance, similar words include 1.10abilities and 1.10consciousness.
In more detail, embedding methods posit two representation vectors for each term in the vocabulary; an embedding vector and a context vector. (We use the language of text for concreteness; as we mentioned, efe extend to other types of data.) The idea is that the conditional probability of each observed word depends on the interaction between the embedding vector and the context vectors of the surrounding words. In sefe, we posit a separate set of embedding vectors for each group but a shared set of context vectors; this ensures that the embedding vectors are in the same space.
We propose two methods to share statistical strength among the embedding vectors. The first approach is based on hierarchical modeling BIBREF13 , which assumes that the group-specific embedding representations are tied through a global embedding. The second approach is based on amortization BIBREF14 , BIBREF15 , which considers that the individual embeddings are the output of a deterministic function of a global embedding representation. We use stochastic optimization to fit large data sets.
Our work relates closely to two threads of research in the embedding literature. One is embedding methods that study how language evolves over time BIBREF16 , BIBREF17 , BIBREF18 , BIBREF19 , BIBREF20 , BIBREF21 . Time can be thought of as a type of “group”, though with evolutionary structure that we do not consider. The second thread is multilingual embeddings BIBREF22 , BIBREF23 , BIBREF24 , BIBREF25 ; our approach is different in that most words appear in all groups and we are interested in the variations of the embeddings across those groups.
Our contributions are thus as follows. We introduce the sefe model, extending efe to grouped data. We present two techniques to share statistical strength among the embedding vectors, one based on hierarchical modeling and one based on amortization. We carry out a thorough experimental study on two text databases, ArXiv papers by section and U.S. Congressional speeches by home state and political party. Using Poisson embeddings, we study market basket data from a large grocery store, grouped by season. On all three data sets, sefe outperforms efe in terms of held-out log-likelihood. Qualitatively, we demonstrate how sefe discovers which words are used most differently across U.S. states and political parties, and show how word usage changes in different ArXiv disciplines.

Background: Exponential Family Embeddings
In exponential family embeddings, we have a collection of objects, and our goal is to learn a vector representation of these objects based on their co-occurrence patterns.
Let us consider a dataset represented as a (typically sparse) matrix INLINEFORM0 , where columns are datapoints and rows are objects. For example, in text, each column corresponds to a location in the text, and each entry INLINEFORM1 is a binary variable that indicates whether word INLINEFORM2 appears at location INLINEFORM3 .
In efe, we represent each object INLINEFORM0 with two sets of vectors, embeddings vectors INLINEFORM1 and context vectors INLINEFORM2 , and we posit a probability distribution of data entries INLINEFORM3 in which these vectors interact. The definition of the efe model requires three ingredients: a context, a conditional exponential family, and a parameter sharing structure. We next describe these three components.
Exponential family embeddings learn the vector representation of objects based on the conditional probability of each observation, conditioned on the observations in its context. The context INLINEFORM0 gives the indices of the observations that appear in the conditional probability distribution of INLINEFORM1 . The definition of the context varies across applications. In text, it corresponds to the set of words in a fixed-size window centered at location INLINEFORM2 .
Given the context INLINEFORM0 and the corresponding observations INLINEFORM1 indexed by INLINEFORM2 , the distribution for INLINEFORM3 is in the exponential family, DISPLAYFORM0 
 with sufficient statistics INLINEFORM0 and natural parameter INLINEFORM1 . The parameter vectors interact in the conditional probability distributions of each observation INLINEFORM2 as follows. The embedding vectors INLINEFORM3 and the context vectors INLINEFORM4 are combined to form the natural parameter, DISPLAYFORM0 
 where INLINEFORM0 is the link function. Exponential family embeddings can be understood as a bank of glm. The context vectors are combined to give the covariates, and the “regression coefficients” are the embedding vectors. In Eq. EQREF6 , the link function INLINEFORM1 plays the same role as in glm and is a modeling choice. We use the identity link function.
The third ingredient of the efe model is the parameter sharing structure, which indicates how the embedding vectors are shared across observations. In the standard efe model, we use INLINEFORM0 and INLINEFORM1 for all columns of INLINEFORM2 . That is, each unique object INLINEFORM3 has a shared representation across all instances.
The objective function. In efe, we maximize the objective function, which is given by the sum of the log-conditional likelihoods in Eq. EQREF5 . In addition, we add an INLINEFORM0 -regularization term (we use the notation of the log Gaussian pdf) over the embedding and context vectors, yielding DISPLAYFORM0 
Note that maximizing the regularized conditional likelihood is not equivalent to maximum a posteriori. Rather, it is similar to maximization of the pseudo-likelihood in conditionally specified models BIBREF26 , BIBREF10 .

Table 2: Test log-likelihood on the three considered datasets. S-EFE consistently achieves the highest held-out likelihood. The competing methods are the global EFE, which can not capture group variations, and the separate EFE, which cannot share information across groups.

Question: What hierarchical modelling approach is used?
{"answer": ["the group-specific embedding representations are tied through a global embedding"], "evidence": ["We propose two methods to share statistical strength among the embedding vectors. The first approach is based on hierarchical modeling BIBREF13 , which assumes that the group-specific embedding representations are tied through a global embedding. The second approach is based on amortization BIBREF14 , BIBREF15 , which considers that the individual embeddings are the output of a deterministic function of a global embedding representation. We use stochastic optimization to fit large data sets."]}