DataFinder: Scientific Dataset Recommendation from Natural Language Descriptions
Modern machine learning relies on datasets to develop and validate research
ideas. Given the growth of publicly available data, finding the right dataset
to use is increasingly difficult. Any research question imposes explicit and
implicit constraints on how well a given dataset will enable researchers to
answer this question, such as dataset size, modality, and domain. We
operationalize the task of recommending datasets given a short natural language
description of a research idea, to help people find relevant datasets for their
needs. Dataset recommendation poses unique challenges as an information
retrieval problem; datasets are hard to directly index for search and there are
no corpora readily available for this task. To facilitate this task, we build
the DataFinder Dataset which consists of a larger automatically-constructed
training set (17.5K queries) and a smaller expert-annotated evaluation set (392
queries). Using this data, we compare various information retrieval algorithms
on our test set and present a superior bi-encoder retriever for text-based
dataset recommendation. This system, trained on the DataFinder Dataset, finds
more relevant search results than existing third-party dataset search engines.
To encourage progress on dataset recommendation, we release our dataset and
models to the public.