RelBench: A Benchmark for Deep Learning on Relational Databases
We present RelBench, a public benchmark for solving predictive tasks over
relational databases with graph neural networks. RelBench provides databases
and tasks spanning diverse domains and scales, and is intended to be a
foundational infrastructure for future research. We use RelBench to conduct the
first comprehensive study of Relational Deep Learning (RDL) (Fey et al., 2024),
which combines graph neural network predictive models with (deep) tabular
models that extract initial entity-level representations from raw tables.
End-to-end learned RDL models fully exploit the predictive signal encoded in
primary-foreign key links, marking a significant shift away from the dominant
paradigm of manual feature engineering combined with tabular models. To
thoroughly evaluate RDL against this prior gold-standard, we conduct an
in-depth user study where an experienced data scientist manually engineers
features for each task. In this study, RDL learns better models whilst reducing
human work needed by more than an order of magnitude. This demonstrates the
power of deep learning for solving predictive tasks over relational databases,
opening up many new research opportunities enabled by RelBench.
66a9e7ed85f81589aaee8291
Joshua Robinson
66a9e7ed85f81589aaee8292
Rishabh Ranjan
66a9e7ed85f81589aaee8293
Weihua Hu
66a9e7ed85f81589aaee8294
Kexin Huang
admin_assigned
2024-08-05T09:35:40.752Z
631fc126788f34547a9d7ee1
https://cdn-avatars.huggingface.co/v1/production/uploads/1674202999503-631fc126788f34547a9d7ee1.png
Kexin Huang
user
kexinhuang
66a9e7ed85f81589aaee8295
Jiaqi Han
claimed_verified
2024-11-21T08:11:12.695Z
66ce1661d620749ad81dda84
/avatars/e8a454c3d45b2e878388b1a6e466a5a1.svg
Jiaqi Han
user
jiaqihan99
66a9e7ed85f81589aaee8296
Alejandro Dobles
admin_assigned
2024-08-05T09:35:15.329Z
65d23685f94e504965d92721
/avatars/4c76c11ed975e0e432e7f6ab24ee99a8.svg
Alejandro Dobles
user
adobles
66a9e7ed85f81589aaee8297
Matthias Fey
66a9e7ed85f81589aaee8298
Jan E. Lenssen
66a9e7ed85f81589aaee8299
Yiwen Yuan
admin_assigned
2024-08-05T09:34:58.016Z
64e43a6382fb6ae7a08afdbb
/avatars/4876d26cfb4acbb31ad52b971b67e77a.svg
Yiwen Yuan
user
yiweny
66a9e7ed85f81589aaee829a
Zecheng Zhang
claimed_verified
2024-07-31T07:43:30.330Z
609a5543456ee4a3392e8427
https://cdn-avatars.huggingface.co/v1/production/uploads/1620727835835-609a5543456ee4a3392e8427.jpeg
Zecheng Zhang
user
zechengz
66a9e7ed85f81589aaee829b
Xinwei He
66a9e7ed85f81589aaee829c
Jure Leskovec
RelBench: A Benchmark for Deep Learning on Relational Databases
We present RelBench, a public benchmark for solving predictive tasks over
relational databases with graph neural networks. RelBench provides databases
and tasks spanning diverse domains and scales, and is intended to be a
foundational infrastructure for future research. We use RelBench to conduct the
first comprehensive study of Relational Deep Learning (RDL) (Fey et al., 2024),
which combines graph neural network predictive models with (deep) tabular
models that extract initial entity-level representations from raw tables.
End-to-end learned RDL models fully exploit the predictive signal encoded in
primary-foreign key links, marking a significant shift away from the dominant
paradigm of manual feature engineering combined with tabular models. To
thoroughly evaluate RDL against this prior gold-standard, we conduct an
in-depth user study where an experienced data scientist manually engineers
features for each task. In this study, RDL learns better models whilst reducing
human work needed by more than an order of magnitude. This demonstrates the
power of deep learning for solving predictive tasks over relational databases,
opening up many new research opportunities enabled by RelBench.
RelBench is a public benchmark for relational deep learning (RDL) tasks that combines graph neural networks with tabular models, demonstrating superior performance and reduced human effort compared to traditional feature engineering.
graph neural networks relational deep learning RelBench predictive tasks relational databases primary-foreign key links manual feature engineering