Transfer learning on large datasets for the accurate prediction of material properties
Graph neural networks trained on large crystal structure databases are
extremely effective in replacing ab initio calculations in the discovery and
characterization of materials. However, crystal structure datasets comprising
millions of materials exist only for the Perdew-Burke-Ernzerhof (PBE)
functional. In this work, we investigate the effectiveness of transfer learning
to extend these models to other density functionals. We show that pre-training
significantly reduces the size of the dataset required to achieve chemical
accuracy and beyond. We also analyze in detail the relationship between the
transfer-learning performance and the size of the datasets used for the initial
training of the model and transfer learning. We confirm a linear dependence of
the error on the size of the datasets on a log-log scale, with a similar slope
for both training and the pre-training datasets. This shows that further
increasing the size of the pre-training dataset, i.e. performing additional
calculations with a low-cost functional, is also effective, through transfer
learning, in improving machine-learning predictions with the quality of a more
accurate, and possibly computationally more involved functional. Lastly, we
compare the efficacy of interproperty and intraproperty transfer learning.