Efficacy of MRI data harmonization in the age of machine learning. A multicenter study across 36 datasets
Pooling publicly-available MRI data from multiple sites allows to assemble
extensive groups of subjects, increase statistical power, and promote data
reuse with machine learning techniques. The harmonization of multicenter data
is necessary to reduce the confounding effect associated with non-biological
sources of variability in the data. However, when applied to the entire dataset
before machine learning, the harmonization leads to data leakage, because
information outside the training set may affect model building, and potentially
falsely overestimate performance. We propose a 1) measurement of the efficacy
of data harmonization; 2) harmonizer transformer, i.e., an implementation of
the ComBat harmonization allowing its encapsulation among the preprocessing
steps of a machine learning pipeline, avoiding data leakage. We tested these
tools using brain T1-weighted MRI data from 1740 healthy subjects acquired at
36 sites. After harmonization, the site effect was removed or reduced, and we
showed the data leakage effect in predicting individual age from MRI data,
highlighting that introducing the harmonizer transformer into a machine
learning pipeline allows for avoiding data leakage.