Dataset Condensation via Efficient Synthetic-Data Parameterization
The great success of machine learning with massive amounts of data comes at a
price of huge computation costs and storage for training and tuning. Recent
studies on dataset condensation attempt to reduce the dependence on such
massive data by synthesizing a compact training dataset. However, the existing
approaches have fundamental limitations in optimization due to the limited
representability of synthetic datasets without considering any data regularity
characteristics. To this end, we propose a novel condensation framework that
generates multiple synthetic data with a limited storage budget via efficient
parameterization considering data regularity. We further analyze the
shortcomings of the existing gradient matching-based condensation methods and
develop an effective optimization technique for improving the condensation of
training data information. We propose a unified algorithm that drastically
improves the quality of condensed data against the current state-of-the-art on
CIFAR-10, ImageNet, and Speech Commands.