You will be shown sections from a scientific research paper, together with a question about the paper. Your task is to answer the question based on the contents of the paper. Your response should be a `json` object with two fields:

"answer": An array of strings extracted directly from the paper which, collectively, answer the question.
"evidence": An array of strings. Each should be an excerpt from the paper, in which one or more of the extracted answers can be found.

For instance, for the question "What baselines did the authors compare against?", a sample response might be:

{
  "answer": ["BERT", "RoBERTa"],
  "evidence": ["In our experiments, we compare the performance of our model against BERT and RoBERTa."]
}

Do not include any text in your response other than the json.

If the question is unanswerable given the provided excerpts, respond with the single word "null".

Paper: Embedding Geographic Locations for Modelling the Natural Environment using Flickr Tags and Structured Data

Meta-data from photo-sharing websites such as Flickr can be used to obtain rich bag-of-words descriptions of geographic locations, which have proven valuable, among others, for modelling and predicting ecological features. One important insight from previous work is that the descriptions obtained from Flickr tend to be complementary to the structured information that is available from traditional scientific resources. To better integrate these two diverse sources of information, in this paper we consider a method for learning vector space embeddings of geographic locations. We show experimentally that this method improves on existing approaches, especially in cases where structured information is available.

Vector space embeddings
The use of low-dimensional vector space embeddings for representing objects has already proven effective in a large number of applications, including natural language processing (NLP), image processing, and pattern recognition. In the context of NLP, the most prominent example is that of word embeddings, which represent word meaning using vectors of typically around 300 dimensions. A large number of different methods for learning such word embeddings have already been proposed, including Skip-gram and the Continuous Bag-of-Words (CBOW) model BIBREF8 , GloVe BIBREF9 , and fastText BIBREF14 . They have been applied effectively in many downstream NLP tasks such as sentiment analysis BIBREF15 , part of speech tagging BIBREF16 , BIBREF17 , and text classification BIBREF18 , BIBREF19 . The model we consider in this paper builds on GloVe, which was designed to capture linear regularities of word-word co-occurrence. In GloVe, there are two word vectors INLINEFORM0 and INLINEFORM1 for each word in the vocabulary, which are learned by minimizing the following objective: DISPLAYFORM0 
where INLINEFORM0 is the number of times that word INLINEFORM1 appears in the context of word INLINEFORM2 , INLINEFORM3 is the vocabulary size, INLINEFORM4 is the target word bias, INLINEFORM5 is the context word bias. The weighting function INLINEFORM6 is used to limit the impact of rare terms. It is defined as 1 if INLINEFORM7 and as INLINEFORM8 otherwise, where INLINEFORM9 is usually fixed to 100 and INLINEFORM10 to 0.75. Intuitively, the target word vectors INLINEFORM11 correspond to the actual word representations which we would like to find, while the context word vectors INLINEFORM12 model how occurrences of INLINEFORM13 in the context of a given word INLINEFORM14 affect the representation of this latter word. In this paper we will use a similar model, which will however be aimed at learning location vectors instead of the target word vectors.
Beyond word embeddings, various methods have been proposed for learning vector space representations from structured data such as knowledge graphs BIBREF20 , BIBREF21 , BIBREF22 , social networks BIBREF23 , BIBREF24 and taxonomies BIBREF25 , BIBREF26 . The idea of combining a word embedding model with structured information has also been explored by several authors, for example to improve the word embeddings based on information coming from knowledge graphs BIBREF27 , BIBREF28 . Along similar lines, various lexicons have been used to obtain word embeddings that are better suited at modelling sentiment BIBREF15 and antonymy BIBREF29 , among others. The method proposed by BIBREF30 imposes the condition that words that belong to the same semantic category are closer together than words from different categories, which is somewhat similar in spirit to how we will model categorical datasets in our model.

Model Description
In this section, we introduce our embedding model, which combines Flickr tags and structured scientific information to represent a set of locations INLINEFORM0 . The proposed model has the following form: DISPLAYFORM0 
where INLINEFORM0 and INLINEFORM1 are parameters to control the importance of each component in the model. Component INLINEFORM2 will be used to constrain the representation of the locations based on their textual description (i.e. Flickr tags), INLINEFORM3 will be used to constrain the representation of the locations based on their numerical features, and INLINEFORM4 will impose the constraint that locations belonging to the same category should be close together in the space. We will discuss each of these components in more detail in the following sections.

Tag Based Location Embedding
Many of the tags associated with Flickr photos describe characteristics of the places where these photos were taken BIBREF45 , BIBREF46 , BIBREF47 . For example, tags may correspond to place names (e.g. Brussels, England, Scandinavia), landmarks (e.g. Eiffel Tower, Empire State Building) or land cover types (e.g. mountain, forest, beach). To allow us to build location models using such tags, we collected the tags and meta-data of 70 million Flickr photos with coordinates in Europe (which is the region our experiments will focus on), all of which were uploaded to Flickr before the end of September 2015. In this section we first explain how tags can be weighted to obtain bag-of-words representations of locations from Flickr. Subsequently we describe a tag selection method, which will allow us to specialize the embedding depending on which aspects of the considered locations are of interest, after which we discuss the actual embedding model.
Tag weighting. Let INLINEFORM0 be a set of geographic locations, each characterized by latitude and longitude coordinates. To generate a bag-of-words representation of a given location, we have to weight the relevance of each tag to that location. To this end, we have followed the weighting scheme from BIBREF7 , which combines a Gaussian kernel (to model spatial proximity) with Positive Pointwise Mutual Information (PPMI) BIBREF48 , BIBREF49 .
Let us write INLINEFORM0 for the set of users who have assigned tag INLINEFORM1 to a photo with coordinates near INLINEFORM2 . To assess how relevant INLINEFORM3 is to the location INLINEFORM4 , the number of times INLINEFORM5 occurs in photos near INLINEFORM6 is clearly an important criterion. However, rather than simply counting the number of occurrences within some fixed radius, we use a Gaussian kernel to weight the tag occurrences according to their distance from that location: INLINEFORM7 
where the threshold INLINEFORM0 is assumed to be fixed, INLINEFORM1 is the location of a Flickr photo, INLINEFORM2 is the Haversine distance, and we will assume that the bandwidth parameter INLINEFORM3 is set to INLINEFORM4 . A tag occurrence is counted only once for all photos by the same user at the same location, which is important to reduce the impact of bulk uploading. The value INLINEFORM5 reflects how frequent tag INLINEFORM6 is near location INLINEFORM7 , but it does not yet take into account the total number of tag occurrences near INLINEFORM8 , nor how popular the tag INLINEFORM9 is overall. To measure how strongly tag INLINEFORM10 is associated with location INLINEFORM11 , we use PPMI, which is a commonly used measure of association in natural language processing. However, rather than estimating PPMI scores from term frequencies, we will use the INLINEFORM12 values instead: INLINEFORM13 
where: INLINEFORM0 
 with INLINEFORM0 the set of all tags, and INLINEFORM1 the set of locations.
Tag selection. Inspired by BIBREF50 , we use a term selection method in order to focus on the tags that are most important for the tasks that we want to consider and reduce the impact of tags that might relate only to a given individual or a group of users. In particular, we obtained good results with a method based on Kullback-Leibler (KL) divergence, which is based on BIBREF51 . Let INLINEFORM0 be a set of (mutually exclusive) properties of locations in which we are interested (e.g. land cover categories). For the ease of presentation, we will identify INLINEFORM1 with the set of locations that have the corresponding property. Then, we select tags from INLINEFORM2 that maximize the following score: INLINEFORM3 
where INLINEFORM0 is the probability that a photo with tag INLINEFORM1 has a location near INLINEFORM2 and INLINEFORM3 is the probability that an arbitrary tag occurrence is assigned to a photo near a location in INLINEFORM4 . Since INLINEFORM5 often has to be estimated from a small number of tag occurrences, it is estimated using Bayesian smoothing: INLINEFORM6 
where INLINEFORM0 is a parameter controlling the amount of smoothing, which will be tuned in the experiments. On the other hand, for INLINEFORM1 we can simply use a maximum likelihood estimation: INLINEFORM2 
Location embedding. We now want to find a vector INLINEFORM0 for each location INLINEFORM1 such that similar locations are represented using similar vectors. To achieve this, we use a close variant of the GloVe model, where tag occurrences are treated as context words of geographic locations. In particular, with each location INLINEFORM2 we associate a vector INLINEFORM3 and with each tag INLINEFORM4 we associate a vector INLINEFORM5 and a bias term INLINEFORM6 , and consider the following objective (which in our full model ( EQREF7 ) will be combined with components that are derived from the structured information): INLINEFORM7 
 Note how tags play the role of the context words in the GloVe model, while instead of learning target word vectors we now learn location vectors. In contrast to GloVe, our objective does not directly refer to co-occurrence statistics, but instead uses the INLINEFORM0 scores. One important consequence of this is that we can also consider pairs INLINEFORM1 for which INLINEFORM2 does not occur in INLINEFORM3 at all; such pairs are usually called negative examples. While they cannot be used in the standard GloVe model, some authors have already reported that introducing negative examples in variants of GloVe can lead to an improvement BIBREF52 . In practice, evaluating the full objective above would not be computationally feasible, as we may need to consider millions of locations and millions of tags. Therefore, rather than considering all tags in INLINEFORM4 for the inner summation, we only consider those tags that appear at least once near location INLINEFORM5 together with a sample of negative examples.

Structured Environmental Data
There is a wide variety of structured data that can be used to describe locations. In this work, we have restricted ourselves to the same datasets as BIBREF7 . These include nine (real-valued) numerical features, which are latitude, longitude, elevation, population, and five climate related features (avg. temperature, avg. precipitation, avg. solar radiation, avg. wind speed, and avg. water vapor pressure). In addition, 180 categorical features were used, which are CORINE land cover classes at level 1 (5 classes), level 2 (15 classes) and level 3 (44 classes) and 116 soil types (SoilGrids). Note that each location should belong to exactly 4 categories: one CORINE class at each of the three levels and a soil type.
Numerical features. Numerical features can be treated similarly to the tag occurrences, i.e. we will assume that the value of a given numerical feature can be predicted from the location vectors using a linear mapping. In particular, for each numerical feature INLINEFORM0 we consider a vector INLINEFORM1 and a bias term INLINEFORM2 , and the following objective: INLINEFORM3 
 where we write INLINEFORM0 for set of all numerical features and INLINEFORM1 is the value of feature INLINEFORM2 for location INLINEFORM3 , after z-score normalization.
Categorical features. To take into account the categorical features, we impose the constraint that locations belonging to the same category should be close together in the space. To formalize this, we represent each category type INLINEFORM0 as a vector INLINEFORM1 , and consider the following objective: INLINEFORM2 

Evaluation Tasks
We will use the method from BIBREF7 as our main baseline. This will allow us to directly evaluate the effectiveness of embeddings for the considered problem, since we have used the same structured datasets and same tag weighting scheme. For this reason, we will also follow their evaluation methodology. In particular, we will consider three evaluation tasks:
Predicting the distribution of 100 species across Europe, using the European network of nature protected sites Natura 2000 dataset as ground truth. For each of these species, a binary classification problem is considered. The set of locations INLINEFORM0 is defined as the 26,425 distinct sites occurring in the dataset.
Predicting soil type, again each time treating the task as a binary classification problem, using the same set of locations INLINEFORM0 as in the species distribution experiments. For these experiments, none of the soil type features are used for generating the embeddings.
Predicting CORINE land cover classes at levels 1, 2 and level 3, each time treating the task as a binary classification problem, using the same set of locations INLINEFORM0 as in the species distribution experiments. For these experiments, none of the CORINE features are used for generating the embeddings.
In addition, we will also consider the following regression tasks:
Predicting 5 climate related features: the average precipitation, temperature, solar radiation, water vapor pressure, and wind speed. We again use the same set of locations INLINEFORM0 as for species distribution in this experiment. None of the climate features is used for constructing the embeddings for this experiment.
Predicting people's subjective opinions of landscape beauty in Britain, using the crowdsourced dataset from the ScenicOrNot website as ground truth. The set INLINEFORM0 is chosen as the set of locations of 191 605 rated locations from the ScenicOrNot dataset for which at least one georeferenced Flickr photo exists within a 1 km radius.

Variants and Baseline Methods
We will refer to our model as EGEL (Embedding GEographic Locations), and will consider the following variants. EGEL-Tags only uses the information from the Flickr tags (i.e. component INLINEFORM0 ), without using any negative examples and without feature selection. EGEL-Tags+NS is similar to EGEL-Tags but with the addition of negative examples. EGEL-KL(Tags+NS) additionally considers term selection. EGEL-All is our full method, i.e. it additionally uses the structured information. We also consider the following baselines. BOW-Tags represents locations using a bag-of-words representation, using the same tag weighting as the embedding model. BOW-KL(Tags) uses the same representation but after term selection, using the same KL-based method as the embedding model. BOW-All combines the bag-of-words representation with the structured information, encoded as proposed in BIBREF7 . GloVe uses the objective from the original GloVe model for learning location vectors, i.e. this variant differs from EGEL-Tags in that instead of INLINEFORM1 we use the number of co-occurrences of tag INLINEFORM2 near location INLINEFORM3 , measured as INLINEFORM4 .

Acknowledgments
Shelan Jeawak has been sponsored by HCED Iraq. Steven Schockaert has been supported by ERC Starting Grant 637277.

Fig. 1: Comparison between the performance of the GloVe and bag-of-words models for predicting scenicness, as a function of the number of tag occurrences at the considered locations.

Question: what dataset is used in this paper?
{"answer": [" the same datasets as BIBREF7"], "evidence": ["There is a wide variety of structured data that can be used to describe locations. In this work, we have restricted ourselves to the same datasets as BIBREF7 . These include nine (real-valued) numerical features, which are latitude, longitude, elevation, population, and five climate related features (avg. temperature, avg. precipitation, avg. solar radiation, avg. wind speed, and avg. water vapor pressure). In addition, 180 categorical features were used, which are CORINE land cover classes at level 1 (5 classes), level 2 (15 classes) and level 3 (44 classes) and 116 soil types (SoilGrids). Note that each location should belong to exactly 4 categories: one CORINE class at each of the three levels and a soil type."]}