IntelliGraphs: Datasets for Benchmarking Knowledge Graph Generation
Knowledge Graph Embedding (KGE) models are used to learn continuous
representations of entities and relations. A key task in the literature is
predicting missing links between entities. However, Knowledge Graphs are not
just sets of links but also have semantics underlying their structure.
Semantics is crucial in several downstream tasks, such as query answering or
reasoning. We introduce the subgraph inference task, where a model has to
generate likely and semantically valid subgraphs. We propose IntelliGraphs, a
set of five new Knowledge Graph datasets. The IntelliGraphs datasets contain
subgraphs with semantics expressed in logical rules for evaluating subgraph
inference. We also present the dataset generator that produced the synthetic
datasets. We designed four novel baseline models, which include three models
based on traditional KGEs. We evaluate their expressiveness and show that these
models cannot capture the semantics. We believe this benchmark will encourage
the development of machine learning models that emphasize semantic
understanding.