You will be shown sections from a scientific research paper, together with a question about the paper. Paragraphs in the paper are separated by newlines. Your task is to answer the question based on the contents of the paper.

Paper:
----------------------------------------
Learning Data Manipulation for Augmentation and Weighting

Manipulating data, such as weighting data examples or augmenting with new instances, has been increasingly used to improve model training. Previous work has studied various rule- or learning-based approaches designed for specific types of data manipulation. In this work, we propose a new method that supports learning different manipulation schemes with the same gradient-based algorithm. Our approach builds upon a recent connection of supervised learning and reinforcement learning (RL), and adapts an off-the-shelf reward learning algorithm from RL for joint data manipulation learning and model training. Different parameterization of the ``data reward'' function instantiates different manipulation schemes. We showcase data augmentation that learns a text transformation network, and data weighting that dynamically adapts the data sample importance. Experiments show the resulting algorithms significantly improve the image and text classification performance in low data regime and class-imbalance problems.

Learning Data Manipulation ::: Method ::: Learning Manipulation Parameters
To learn the parameters $\mathbf {\phi }$ in the manipulation reward $R_\phi (\mathbf {x}, y | \mathcal {D})$, we could in principle adopt any off-the-shelf reward learning algorithm in the literature. In this work, we draw inspiration from the above gradient-based reward learning (section SECREF3) due to its simplicity and efficiency. Briefly, the objective of $\mathbf {\phi }$ is to maximize the ultimate measure of the performance of model $p_\theta (\mathbf {y}|\mathbf {x})$, which, in the context of supervised learning, is the model performance on a held-out validation set.
The algorithm optimizes $\mathbf {\theta }$ and $\mathbf {\phi }$ alternatingly, corresponding to Eq.(DISPLAY_FORM7) and Eq.(DISPLAY_FORM8), respectively. More concretely, in each iteration, we first update the model parameters $\mathbf {\theta }$ in analogue to Eq.(DISPLAY_FORM7) which optimizes intrinsic reward-enriched objective. Here, we optimize the log-likelihood of the training set enriched with data manipulation. That is, we replace $R_\delta $ with $R_\phi $ in Eq.(DISPLAY_FORM5), and obtain the augmented M-step:
By noticing that the new $\mathbf {\theta }^{\prime }$ depends on $\mathbf {\phi }$, we can write $\mathbf {\theta }^{\prime }$ as a function of $\mathbf {\phi }$, namely, $\mathbf {\theta }^{\prime }=\mathbf {\theta }^{\prime }(\mathbf {\phi })$. The practical implementation of the above update depends on the actual parameterization of manipulation $R_\phi $, which we discuss in more details in the next section.
The next step is to optimize $\mathbf {\phi }$ in terms of the model validation performance, in analogue to Eq.(DISPLAY_FORM8). Formally, let $\mathcal {D}^v$ be the validation set of data examples. The update is then:
where, since $\mathbf {\theta }^{\prime }$ is a function of $\mathbf {\phi }$, the gradient is backpropagated to $\mathbf {\phi }$ through $\mathbf {\theta }^{\prime }(\mathbf {\phi })$. Taking data weighting for example where $\mathbf {\phi }$ is the training sample weights (more details in section SECREF15), the update is to optimize the weights of training samples so that the model performs best on the validation set.
The resulting algorithm is summarized in Algorithm FIGREF11. Figure FIGREF11 illustrates the computation flow. Learning the manipulation parameters effectively uses a held-out validation set. We show in our experiments that a very small set of validation examples (e.g., 2 labels per class) is enough to significantly improve the model performance in low data regime.
It is worth noting that some previous work has also leveraged validation examples, such as learning data augmentation with policy gradient BIBREF3 or inducing data weights with meta-learning BIBREF4. Our approach is inspired from a distinct paradigm of (intrinsic) reward learning. In contrast to BIBREF3 that treats data augmentation as a policy, we instead formulate manipulation as a reward function and enable efficient stochastic gradient updates. Our approach is also more broadly applicable to diverse data manipulation types than BIBREF4, BIBREF3.

Learning Data Manipulation ::: Instantiations: Augmentation & Weighting ::: Fine-tuning Text Augmentation
The recent work BIBREF18, BIBREF19 developed a novel contextual augmentation approach for text data, in which a powerful pretrained language model (LM), such as BERT BIBREF8, is used to generate substitutions of words in a sentence. Specifically, given an observed sentence $\mathbf {x}^*$, the method first randomly masks out a few words. The masked sentence is then fed to BERT which fills the masked positions with new words. To preserve the original sentence class, the BERT LM is retrofitted as a label-conditional model, and trained on the task training examples. The resulting model is then fixed and used to augment data during the training of target model. We denote the augmentation distribution as $g_{\phi _0}(\mathbf {x}|\mathbf {x}^*, \mathbf {y}^*)$, where $\mathbf {\phi }_0$ is the fixed BERT LM parameters.
The above process has two drawbacks. First, the LM is fixed after fitting to the task data. In the subsequent phase of training the target model, the LM augments data without knowing the state of the target model, which can lead to sub-optimal results. Second, in the cases where the task dataset is small, the LM can be insufficiently trained for preserving the labels faithfully, resulting in noisy augmented samples.
To address the difficulties, it is beneficial to apply the proposed learning data manipulation algorithm to additionally fine-tune the LM jointly with target model training. As discussed in section SECREF4, this reduces to properly parameterizing the data reward function:
That is, a sample $(\mathbf {x}, y)$ receives a unit reward when $y$ is the true label and $\mathbf {x}$ is the augmented sample by the LM (instead of the exact original data $\mathbf {x}^*$). Plugging the reward into Eq.(DISPLAY_FORM13), we obtain the data-augmented update for the model parameters:
That is, we pick an example from the training set, and use the LM to create augmented samples, which are then used to update the target model. Regarding the update of augmentation parameters $\mathbf {\phi }$ (Eq.DISPLAY_FORM14), since text samples are discrete, to enable efficient gradient propagation through $\mathbf {\theta }^{\prime }$ to $\mathbf {\phi }$, we use a gumbel-softmax approximation BIBREF39 to $\mathbf {x}$ when sampling substitution words from the LM.

Experiments ::: Experimental Setup
Base Models. We choose strong pretrained networks as our base models for both text and image classification. Specifically, on text data, we use the BERT (base, uncased) model BIBREF8; while on image data, we use ResNet-34 BIBREF9 pretrained on ImageNet. We show that, even with the large-scale pretraining, data manipulation can still be very helpful to boost the model performance on downstream tasks. Since our approach uses validation sets for manipulation parameter learning, for a fair comparison with the base model, we train the base model in two ways. The first is to train the model on the training sets as usual and select the best step using the validation sets; the second is to train on the merged training and validation sets for a fixed number of steps. The step number is set to the average number of steps selected in the first method. We report the results of both methods.
Comparison Methods. We compare our approach with a variety of previous methods that were designed for specific manipulation schemes: (1) For text data augmentation, we compare with the latest model-based augmentation BIBREF19 which uses a fixed conditional BERT language model for word substitution (section SECREF15). As with base models, we also tried fitting the augmentatin model to both the training data and the joint training-validation data, and did not observe significant difference. Following BIBREF19, we also study a conventional approach that replaces words with their synonyms using WordNet BIBREF40. (2) For data weighting, we compare with the state-of-the-art approach BIBREF4 that dynamically re-estimates sample weights in each iteration based on the validation set gradient directions. We follow BIBREF4 and also evaluate the commonly-used proportion method that weights data by inverse class frequency.
Training. For both the BERT classifier and the augmentation model (which is also based on BERT), we use Adam optimization with an initial learning rate of 4e-5. For ResNets, we use SGD optimization with a learning rate of 1e-3. For text data augmentation, we augment each minibatch by generating two or three samples for each data points (each with 1, 2 or 3 substitutions), and use both the samples and the original data to train the model. For data weighting, to avoid exploding value, we update the weight of each data point in a minibatch by decaying the previous weight value with a factor of 0.1 and then adding the gradient. All experiments were implemented with PyTorch (pytorch.org) and were performed on a Linux machine with 4 GTX 1080Ti GPUs and 64GB RAM. All reported results are averaged over 15 runs $\pm $ one standard deviation.

Experiments ::: Low Data Regime ::: Results
Table TABREF26 shows the manipulation results on text classification. For data augmentation, our approach significantly improves over the base model on all the three datasets. Besides, compared to both the conventional synonym substitution and the approach that keeps the augmentation network fixed, our adaptive method that fine-tunes the augmentation network jointly with model training achieves superior results. Indeed, the heuristic-based synonym approach can sometimes harm the model performance (e.g., SST-5 and IMDB), as also observed in previous work BIBREF19, BIBREF18. This can be because the heuristic rules do not fit the task or datasets well. In contrast, learning-based augmentation has the advantage of adaptively generating useful samples to improve model training.
Table TABREF26 also shows the data weighting results. Our weight learning consistently improves over the base model and the latest weighting method BIBREF4. In particular, instead of re-estimating sample weights from scratch in each iteration BIBREF4, our approach treats the weights as manipulation parameters maintained throughout the training. We speculate that the parametric treatment can adapt weights more smoothly and provide historical information, which is beneficial in the small-data context.
It is interesting to see from Table TABREF26 that our augmentation method consistently outperforms the weighting method, showing that data augmentation can be a more suitable technique than data weighting for manipulating small-size data. Our approach provides the generality to instantiate diverse manipulation types and learn with the same single procedure.
To investigate the augmentation model and how the fine-tuning affects the augmentation results, we show in Figure TABREF27 the top-5 most probable word substitutions predicted by the augmentation model for two masked tokens, respectively. Comparing the results of epoch 1 and epoch 3, we can see the augmentation model evolves and dynamically adjusts the augmentation behavior as the training proceeds. Through fine-tuning, the model seems to make substitutions that are more coherent with the conditioning label and relevant to the original words (e.g., replacing the word “striking” with “bland” in epoch 1 v.s. “charming” in epoch 3).
Table TABREF27 shows the data weighting results on image classification. We evaluate two settings with the ResNet-34 base model being initialized randomly or with pretrained weights, respectively. Our data weighting consistently improves over the base model and BIBREF4 regardless of the initialization.

Experiments ::: Imbalanced Labels ::: Results
Table TABREF29 shows the classification results on SST-2 with varying imbalance ratios. We can see our data weighting performs best across all settings. In particular, the improvement over the base model increases as the data gets more imbalanced, ranging from around 6 accuracy points on 100:1000 to over 20 accuracy points on 20:1000. Our method is again consistently better than BIBREF4, validating that the parametric treatment is beneficial. The proportion-based data weighting provides only limited improvement, showing the advantage of adaptive data weighting. The base model trained on the joint training-validation data for fixed steps fails to perform well, partly due to the lack of a proper mechanism for selecting steps.
Table TABREF30 shows the results on imbalanced CIFAR10 classification. Similarly, our method outperforms other comparison approaches. In contrast, the fixed proportion-based method sometimes harms the performance as in the 50:1000 and 100:1000 settings.
We also tested the text augmentation LM on the SST-2 imbalanced data. Interestingly, the augmentation tends to hinder model training and yields accuracy of around 50% (random guess). This is because the augmentation LM is first fit to the imbalanced data, which makes label preservation inaccurate and introduces lots of noise during augmentation. Though a more carefully designed augmentation mechanism can potentially help with imbalanced classification (e.g., augmenting only the rare classes), the above observation further shows that the varying data manipulation schemes have different applicable scopes. Our approach is thus favorable as the single algorithm can be instantiated to learn different schemes.
----------------------------------------

Question: How much is classification performance improved in experiments for low data regime and class-imbalance problems?

To answer the question, format your response as a `json` object with two fields:

"answer": A string providing a succinct answer to the question, in your own words.
"evidence": An array of strings. Each entry should be a full paragraph from the paper. Together, the evidence should serve as a justification for the answer.

For instance, for the question "What baselines did the authors compare against?", a sample response might be:

{
  "answer": "BERT and RoBERTa."
  "evidence": ["We compare our approach against two baselines. In Table 1, we compare against BERT. In Table 2, we compare against RoBERTa. Our findings indicate that our approach improves over both baeslines..."]
}

The "answer" field should be roughly 262 characters in length.

Do not include any text in your response other than the json. If the question is unanswerable given the provided excerpts, respond with the single word "null".

To repeat, the question is: How much is classification performance improved in experiments for low data regime and class-imbalance problems?

Answer JSON object:
{"answer": "Low data: SST-5, TREC, IMDB around 1-2 accuracy points better than baseline\nImbalanced labels: the improvement over the base model increases as the data gets more imbalanced, ranging from around 6 accuracy points on 100:1000 to over 20 accuracy points on 20:1000", "evidence": ["Table TABREF26 shows the manipulation results on text classification. For data augmentation, our approach significantly improves over the base model on all the three datasets. Besides, compared to both the conventional synonym substitution and the approach that keeps the augmentation network fixed, our adaptive method that fine-tunes the augmentation network jointly with model training achieves superior results. Indeed, the heuristic-based synonym approach can sometimes harm the model performance (e.g., SST-5 and IMDB), as also observed in previous work BIBREF19, BIBREF18. This can be because the heuristic rules do not fit the task or datasets well. In contrast, learning-based augmentation has the advantage of adaptively generating useful samples to improve model training.", "Table TABREF29 shows the classification results on SST-2 with varying imbalance ratios. We can see our data weighting performs best across all settings. In particular, the improvement over the base model increases as the data gets more imbalanced, ranging from around 6 accuracy points on 100:1000 to over 20 accuracy points on 20:1000. Our method is again consistently better than BIBREF4, validating that the parametric treatment is beneficial. The proportion-based data weighting provides only limited improvement, showing the advantage of adaptive data weighting. The base model trained on the joint training-validation data for fixed steps fails to perform well, partly due to the lack of a proper mechanism for selecting steps."]}