text:
\section{Experiments}

\subsection{High priority}
\begin{enumerate}
    \item zoom in and out

    done. result depends on the used interpolation method. worse accuracy and robustness for BILINEAR; shear/rotate competitive result for NEAREST. zoom in better than zoom out
    \item if weak croppad+shear = stronger croppad
    \item adaptively weak the following augmentation if the previous ones are stronger
    \item rebalance rcrop and others
\end{enumerate}

\subsection{low priority}
\begin{enumerate}
    \item croppad and move as segmented pieces
    \item cutout, erase with origin + controllable noises
    \item accuracy and robustness evaluation for various strength of individual transformations on alternative architectures

    partially done.
    \item test if weak rotate or shear vulnerable to rcrop augmented models

\end{enumerate}

\section{observations}
\begin{itemize}
    \item clean examples are not truely more vulnerable for complex augmented model, it is simply caused by the dramatically different batch statistics. So the robustness can be recovered when the model is switched to eval mode i.e. using the fixed batch statistics

    the sweet point we are looking for should be also vulnerable when using the fixed training batch statistics

    \item augment should be optimized per model capacity. for small capacity models, too far augment damages performance

    \item croppad: allow many free diversity not raising hardness

    rotate: gain diversity by adding hardness

    \item fill destroy the established batch statistics

	1. randomly sampled from the rest image to preserve the image statistics

	2. randomly sampled from another distribution to alter the statistics

	\item data augmentation also depends on the weight decay, larger decay with weaker augmentation
\end{itemize}

\section{Next Paper}
\begin{itemize}
    \item optimize augmentations towards a diverse set of robust models
    \begin{itemize}
        \item high clean accuracy on clean trained models
        \item low robust accuracy on robust trained models
    \end{itemize}
    \item adapt the difficulty according to the training stages: harder at the later stage when overfitting severer
\end{itemize}

\section{Contributions}
\begin{enumerate}
    \item the effect of diversity and hardness of data augmentation in adversarial training
    \begin{itemize}
        \item diversity: always beneficial. benign diversity and hard diversity. benign diversity doesn't increase the hardness like moving cropped image around the image space, while hard diversity increases diversity by hardening examples.
        \item hardness: serve as regularization to prevent from overfitting. Training on harder examples alleviates overfitting, sacrificing clean accuracy (if no bonus of alleviating robust overfitting) and may or may not increase robustness
    \end{itemize}
    \item train and visualize various strength of individual augmentations to adjust the range of strength to explore.
    \item augmentation for standard training doesn't work well for adversarial training
\end{enumerate}

\subsubsection*{Author Contributions}
If you'd like to, you may include  a section for author contributions as is done
in many journals. This is optional and at the discretion of the authors.

\subsubsection*{Acknowledgments}
Use unnumbered third level headings for the acknowledgments. All
acknowledgments, including those to funding agencies, go at the end of the paper.

\section{Introduction} \label{sec: introduction}
Adversarial training, despite its effectiveness in defending against adversarial attack, is prone to overfitting. Specifically, while performance on classifying training adversarial examples improves during the later stages of training, test adversarial robustness degenerates.
This phenomenon is called \textit{robust overfitting} \citep{rice_overfitting_2020}.
To alleviate overfitting, \citet{rice_overfitting_2020} propose to track the model's robustness on a reserved validation data and select the checkpoint with the best validation robustness instead of the one at the end of training.
This simple technique, named early-stopping (ES), matches the performance of contemporary state-of-the-art methods, suggesting that overfitting in adversarial training impairs its performance significantly.
Preventing robust overfitting is, therefore, important for improving adversarial training.

Data augmentation is an effective technique to alleviate overfitting in standard training, but it seems to not work well in adversarial training.
Almost all previous attempts \citep{rice_overfitting_2020, wu_adversarial_2020, gowal_uncovering_2021, rebuffi_data_2021, carmon_unlabeled_2019} to prevent robust overfitting by data augmentation have failed.
Specifically, this previous work found that several advanced data augmentation methods like Cutout \citep{devries_improved_2017}, Mixup \citep{zhang_mixup_2018} and Cutmix \citep{yun_cutmix_2019} failed to improve the robustness of adversarially-trained models to match that of the simple augmentation Flip-Padcrop with ES, as shown in~\cref{fig: overview da comparison}. Thus the method of using ES with Flip-Padcrop has been widely accepted as the "baseline" for combating robust overfitting.
Even with ES, Cutout still fails to improve the robustness over the baseline, while Mixup boosts the robustness marginally ($< 0.4\%$) \citep{rice_overfitting_2020, wu_adversarial_2020}.
This contrasts with their excellent performance in standard training.
Recently, \citet{tack_consistency_2022} observed that AutoAugment \citep{cubuk_autoaugment_2019} can eliminate robust overfitting and boost robustness greatly.
This, however, contradicts the result of \citet{gowal_uncovering_2021, carmon_unlabeled_2019} where the baseline was found to outperform AutoAugment in terms of robustness.
Overall, to date, there has been no uncontroversial evidence showing that robust generalization can be further improved over the baseline by data augmentation alone, and no convincing explanation about this ineffectiveness.

This work focuses on improving the robust generalization ability of adversarial training by data augmentation.
We first demonstrate that the superior robustness of AutoAugment claimed by \citet{tack_consistency_2022} is actually a false security since its robustness against the more reliable  AutoAttack (AA) \citep{croce_reliable_2020} (48.71\%) is just slightly higher than the baseline's (48.21\%) as shown in~\cref{fig: overview da comparison} (see \cref{sec: contradict autoaugment} for a detailed discussion).
We then investigate the impact of the hardness and diversity of data augmentation on the performance of adversarial training. It is found that, in general, hard augmentation can alleviate robust overfitting and improve the robustness but at the expense of clean accuracy within a certain limit of hardness.
Over that limit, both robustness and accuracy decline, even though robust overfitting is mitigated more with the increase in hardness.
On the other hand, diverse augmentation generally can alleviate robust overfitting and boost both accuracy and robustness.
These results give us the insight that the optimal data augmentation for adversarial training should have as much diversity as possible and well-balanced hardness.

To improve robust generalization, we propose a new image transformation, Cropshift, a more diverse  replacement for the conventional crop operation, Padcrop. Cropshift is used as a component in a new data augmentation scheme that we call Improved Diversity and Balanced Hardness (IDBH).
Empirically, IDBH achieves the state-of-the-art robustness and accuracy among data augmentation methods in adversarial training. It improves the end robustness to be significantly higher than the robustness of the baseline augmentation with early-stopping (\cref{fig: overview da comparison}), which all previous attempts failed to achieve. Furthermore, it matches the performance of the state-of-the-art regularization methods for improving adversarial training and, when combined with weight averaging, considerably outperforms almost all of them in terms of robustness.

\begin{SCfigure}
\centering
\includegraphics[width=.48\textwidth, trim=10 40 5 15,clip]{figures/da-robustness.png}
\caption{Our method is the only one that significantly improves both accuracy and robustness over the baseline (Flip-Padcrop with early-stopping).
Cutout and Cutmix fail to beat the baseline regarding robustness.
AutoAugment achieves only a small improvement on robustness over the baseline.
Robustness is evaluated against AutoAttack. See~\cref{sec: result} for details of training and evaluation settings.}
\label{fig: overview da comparison}
\end{SCfigure}

\section{Related Works} \label{sec: related works}

Robust overfitting can be successfully mitigated by smoothing labels, using Knowledge Distillation (KD) \citep{chen_robust_2021} and Temporal Ensembling (TE) \citep{dong_exploring_2022}, and/or smoothing weights using Stochastic Weight Averaging (SWA) \citep{chen_robust_2021} and Adversarial Weight Perturbation (AWP) \citep{wu_adversarial_2020}.
Moreover, \citet{singla_low_2021} found that using activation functions with low curvature improved the generalization of both accuracy and robustness.
Alternatively, \citet{yu_understanding_2022} attributed robust overfitting to the training examples with small loss value, and showed that enlarging the loss of those examples during training, called Minimum Loss Constrained Adversarial Training (MLCAT), can alleviate robust overfitting.
Our work prevents robust overfitting by data augmentation, and hence complements the above methods.

To date, it is still unclear if more training data benefits generalization in adversarial training.
\citet{schmidt_adversarially_2018} showed that adversarial training requires more data, compared to its standard training counterpart, to achieve the same level of generalization.
In contrast, \citet{min_curious_2021, chen_more_2020} proved that more training data can hurt the generalization in some particular adversarial training regimes on some simplified models and tasks.
Empirically, a considerable improvement has been observed in both clean and robust accuracy when the training set is dramatically expanded, in a semi-supervised way, with unlabeled data \citep{carmon_unlabeled_2019,alayrac_are_2019}, e.g., using Robust Self-Training (RST) \citep{carmon_unlabeled_2019} or with synthetic data generated by a generative model \citep{gowal_improving_2021}. Although data augmentation alone doesn't work well, it was observed to improve robustness to a large degree when combined with SWA \citep{rebuffi_data_2021} or Consistency (CONS) regularization \citep{tack_consistency_2022}. In contrast, our work doesn't require any additional data or regularization: it improves robust generalization by data augmentation alone.

Common augmentations \citep{he_deep_2016} used in image classification tasks include Padcrop (padding the image at each edge and then cropping back to the original size) and Horizontal Flip. Many more complicated augmentations have been proposed to further boost generalization. Cutout \citep{devries_improved_2017} and Random Erasing \citep{zhong_random_2020} randomly drop a region in the input space. Mixup \citep{zhang_mixup_2018} and Cutmix \citep{yun_cutmix_2019} randomly interpolate two images, as well as their labels, into a new one. AutoAugment \citep{cubuk_autoaugment_2019} employs a combination of multiple basic image transformations like Color and Rotation and automatically searches for the optimal composition of them. TrivialAugment \citep{muller_trivialaugment_2021} matches the performance of AutoAugment with a similar schedule yet without any explicit search, suggesting that this computationally expensive process may be unnecessary. The method proposed here improves on the above methods by specifically considering the diversity and hardness of the augmentations. The difference between data augmentation in standard and adversarial training is discussed in \cref{sec: compare da in st}.

\section{How Data Augmentation Alleviates Robust Overfitting} \label{sec: analysis}

This section describes an investigation into how the hardness and the diversity of data augmentation effects overfitting in adversarial training. During training, the model's robustness was tracked at each epoch using PGD10 applied to the test set. The checkpoint with the highest robustness was selected as the "best" checkpoint.
Best (end) robustness/accuracy refers to the robustness/accuracy of the best (last) checkpoint. In this section, the terms accuracy and robustness refer to the end accuracy and robustness unless specified otherwise. The severity of robust overfitting was measured using the best robustness minus the end robustness. Hence, the more positive this gap in robustness the more severe the robust overfitting.
The training setup is described in \cref{sec: experiment setting}.

\subsection{Hardness} \label{sec: hardness}

Hardness was measured by the Affinity metric \citep{gontijo-lopes_tradeoffs_2021} adapted from standard training:
\begin{equation}
    hardness = \frac{Robustness(M, D_{test})}{Robustness(M, D'_{test})}
    \label{equ: hardness}
\end{equation}
where $M$ is an arbitrary model adversarially trained on the unaugmented training data. $D_{test}$ refers to the original test data set and $D'_{test}$ is $D_{test}$ with the augmentation (to be evaluated) applied.
$Robustness(M, D)$ is the robust accuracy of $M$ evaluated using PGD50 on $D$. Hardness is a model-specific measure. It increases as the augmentation causes the data to become easier to attack, i.e., as the perturbed, augmented, data becomes more difficult to be correctly classified.

\input{figures/hardness}

We found that moderate levels hardness can alleviate robust overfitting and improve the robustness but at the price of accuracy. Further increasing hardness causes both accuracy and robustness to decline, even though robust overfitting is alleviated further. The value of hardness where this occurs is very sensitive to the capacity of the model. Therefore, to maximize robustness, hardness should be carefully balanced, for each model, between alleviating robust overfitting and impairing accuracy.

\paragraph{Experiments design.}
We investigated the effects of hardness in adversarial training for individual and composed augmentations. For the individual augmentations the following 12 image transformations were choosen: ShearX, ShearY, TranslateX, TranslateY, Rotate, Color, Sharpness, Brightness, Contrast, Solarize, Cutout and Cropshift (a variant of Padcrop introduced in~\cref{sec: method}).
For each \cref{equ: hardness} was used to calibrate the strength of the augmentation (e.g. angle for Rotation) onto one of 7 levels of hardness (see \cref{sec: hardness strength} for specific values), except for Color and Sharpness which were applied at 3 strengths.
For simplicity, the integers 1 to 7 are used to represent these 7 degrees of hardness.
Standard Cutout is allowed to cut partially outside the image, and thus the hardness is not always directly related to the nominal size of the cut-out region (strength).
To ensure alignment between strength and hardness, we force Cutout to cut only inside the image and refer this variant as Cutout-i.
To control for diversity, each augmentation was applied with one degree of diversity. As a result the effects of applying Cutout and Cropshift with a certain hardness is deterministic throughout training. Specifically, Cutout always cuts at the fixed location (sampled once at the beginning of training). Similarly, CropShift crops a fixed region and shifts it to the fixed location (both sampled once at the beginning of training). We name these two variants as Cutout-i-1 and CropShift-1 respectively. Models were trained with each transformation at each hardness.

To investigate composed augmentations, models were trained with various multi-layered data augmentations: Flip-Padcrop (FP), FP-Cutout[Weak] (CW), FP-Cutout[Strong] (CS), FP-AutoAugment-Cutout (AuA) and FP-TrivialAugment-Cutout (TA).
All of them shared the same parameters for Flip and Padcrop. CW and CS used 8x8 and 10x10 Cutout respectively. AuA and TA used 16x16 Cutout as in their original settings \citep{cubuk_autoaugment_2019,muller_trivialaugment_2021}. Different from the default experimental setting, augmentations here were always applied during training, and robustness was evaluated against AA since AuA was observed to fool the PGD attack (\cref{sec: contradict autoaugment}).

Hardness increases from FP to TA as more layers stack up and/or the strength of individual components increases. Hence, this experiment can, as for the experiments with individual augmentations, be seen as an investigation into the effects of increasing hardness. Here, we are not controlling for diversity, which also roughly increases from FP to TA. However, this does not affect our conclusions as diversity boosts accuracy and robustness (see \cref{sec: diversity}), and hence, the decrease in these values that we observe with increased hardness cannot be explained by the effects of increased diversity.

\paragraph{Observations.}
It can be seen that the gap between best and end robustness drops, i.e., robust overfitting turns milder with the increase in hardness in \cref{fig: hardness individual ro,,fig: hardness composed ro}.
The gap of robustness for AuA is negative in \cref{fig: hardness composed ro} because the PGD10 attack was fooled to select a vulnerable checkpoint as the best: see \cref{sec: contradict autoaugment} for more discussion.
For accuracy and robustness, there are roughly three phases. First, both accuracy (\cref{fig: hardness individual accuracy}) and robustness (\cref{fig: hardness individual robustness}) increase with hardness. This is only observed for some transformations like Cropshift at hardness 1 and 2.
In this stage, the underlying model has sufficient capacity to fit the augmented data so it benefits from the growth of data complexity.

Second, accuracy (\cref{fig: hardness individual accuracy}) starts to drop while robustness (\cref{fig: hardness individual robustness}) continuously increases.
As the intensity of the transformation increases, the distribution of the transformed data generally deviates more from the distribution of the original data causing the mixture of them to be harder to fit for standard training.
Adversarial training can be considered as standard training plus gradient regularization \citep{li_understanding_2022}.
Roughly speaking, accuracy drops in this stage because the model's capacity is insufficient to fit the increasingly hard examples for the optimization of the clean loss (standard training) under the constraint of gradient regularization.
Nevertheless, robustness could still increase due to the benefit of increasing robust generalization, i.e., smaller adversarial vulnerability.
Third, accuracy (\cref{fig: hardness composed accuracy}) and robustness (\cref{fig: hardness composed robustness}) drop together.
Accuracy continues to decline due to the severer (standard) underfitting.
Meanwhile, the harm of decreasing accuracy now outweighs the benefit of reduced robust overfitting, which results in the degeneration of robustness.

The graphs of Color, TranslateX and TranslateY are omitted from~\cref{fig: hardness individual ro,,fig: hardness individual accuracy,,fig: hardness individual robustness} because they exhibit exceptional behavior at some values of hardness. Nevertheless, these results generally show that
robust overfitting is reduced and robustness is improved.
These results are presented and discussed in~\cref{sec: exception}.
\cref{sec: hardness best accuracy} provides a figure showing best accuracy as a function of hardness for individual augmentations. A more obvious downward trend with increasing hardness can be seen in this graph compared to the graph for end accuracy shown in \cref{fig: hardness individual accuracy}

\input{figures/diversity}
\subsection{Diversity} \label{sec: diversity}

To investigate the effects of augmentation diversity, variation in the augmentations was produced in three ways: (1) using
varied types of transformations ("type diversity"); (2) varying the spatial area to augment ("spatial diversity"); and (3) increasing the variance of the strength while keeping the mean strength constant ("strength diversity").

\paragraph{Type diversity.}
We uniformly drew $M$ transformations, with fixed and roughly equivalent hardness, from the same pool of transformations as the individual experiment in \cref{sec: hardness} but excluding Cutout and Cropshift. During training, one of these $M$ transformations was randomly sampled, with uniform probability, to be applied to each sample separately in a batch. Diversity increases as $M$ increases from 0 (no augmentation applied) to 10 (one augmentation from a pool of 10 applied).
The experiments were repeated for three levels of hardness $\{1, 2, 3\}$.
For all levels of hardness, the gap of robustness (\cref{fig: type diversity ro}) reduces, and the end robustness (\cref{fig: type diversity rob}) and accuracy (\cref{fig: type diversity acc}) increases, as $M$ increases. These trends are more pronounced for higher hardness levels.

\paragraph{Spatial diversity.}
Transformations like Cutout and Cropshift (described  in~\cref{sec: method}) have large inherent diversity due to the large number of possible crop and/or shift operations that can be performed on the same image. For example, there are 28x28 possible 4x4 pixel cutouts that could be taken from a 32x32 pixel image, all with the same strength of 4x4. In contrast, transformations like Shear and Rotation have only one, or if sign is counted, two variations at the same strength, and hence, have a much lower inherent diversity. To investigate the impact of this rich spatial diversity, we run the experiments to compare the performance of Cutout-i-1 with Cutout-i, and to compare Cropshift-1 and Cropshift, at various levels of hardness. In both cases the former variant of the augmentation method is less diverse than the latter.
We observed that the rich spatial diversity in Cutout-i and Cropshift helps dramatically shrink the gap between the best and end robustness (\cref{fig: spatial diversity ro}), and boost the end robustness (\cref{fig: spatial diversity rob}) and accuracy (\cref{fig: spatial diversity acc}) at virtually all hardness.

\paragraph{Strength diversity.}
Diversity in the strength was generated by defining four ranges of hardness: $\{4\}$, $\{3,4,5\}$, $\{2,3,4,5,6\}$ and $\{1,2,3,4,5,6,7\}$.
During training each image was augmented using a strength uniformly sampled at random from the given range. Hence, for each range the hardness of the augmentation, on average, was the same, but the diversity of the augmentations increased with
increasing length of the hardness range.
Models trained with differing diversity were trained with each of the individual transformations defined in~\cref{sec: hardness} excluding Color and Sharpness.
Strength diversity for Cutout-1, Cropshift-1 and Rotate can be seen to
significantly mitigate robust overfitting (\cref{fig: strength diversity ro}) and boost robustness (\cref{fig: strength diversity rob}), whereas for the other transformations it seems have no significant impact on these two metrics.
Nevertheless, a clear increase in accuracy (\cref{fig: strength diversity acc}) is observed when increasing strength diversity for almost all transformations.

\section{Diverse and Hardness-Balanced Data Augmentation} \label{sec: method}
\input{figures/cropshift}

This section first describes Cropshift, our proposed version of  Padcrop with enhanced diversity and disentangled hardness.
Cropshift (\cref{fig: cropshift illustration}; \cref{algo: cropshift}) first randomly crops a region in the image and then shifts it around to a random location in the input space.
The cropped region can be either square or rectangular.
The strength of Cropshift is parameterized by the total number, $N$, of cropped rows and columns.
For example, with strength $8$, Cropshift removes $l, r, t, b$ lines from the left, right, top and bottom borders respectively, such that $l+r+t+b = 8$.
Cropshift significantly diversifies the augmented data in terms of  both the content being cropped and the localization of the cropped content in the final input space.
Furthermore, Cropshift offers a more fine-grained control on the hardness. In contrast, for Padcrop hardness is not directly related to the size of the padding, as for example, using 4 pixel padding can results in cropped images with a variety of total image content that is trimmed (from 4 rows and 4 columns trimmed, to no rows and no columns trimmed).

To mitigate robust overfitting, we propose a new data augmentation scheme with Improved Diversity and Balanced Hardness (IDBH).
Inspired by \citet{muller_trivialaugment_2021}, we design the high-level framework of our augmentation as a 4-layer sequence: flip, crop, color/shape and dropout.
Each has distinct semantic meaning and is applied with its own probability.
Specifically, we implement flip using Horizontal Flip, crop using Cropshift, dropout using Random Erasing, and color/shape using a set of Color, Sharpness, Brightness, Contrast, Autocontrast, Equalize, Shear (X and Y) and Rotate.
The color/shape layer, when applied to augment an image, first samples a transformation according to a probability distribution and then samples a strength from the transformation's strength range.
This distribution and the strength range of each component transformation are all theoretically available to optimize.
Pseudo-code for the proposed augmentation procedure can be found in~\cref{sec: optimization}.

The probability and the strength of each layer was jointly optimized by a heuristic search to maximize the robustness.
It is important to optimize all layers together, rather than individually.
First, this enables a more extensive and more fine-grained search for hardness so that a potentially better hardness balance can be attained.
Moreover, it allows a certain hardness to be achieved with greater diversity.
For example, raising hardness through Cropshift also improves diversity, while doing so through the color/shape layer hardly increases diversity.
However, optimizing the parameters of all layers jointly adds significantly to the computational burden.
To tackle this issue, the search space was reduced based on insights gained from the preliminary experiments and other work, and grid search was performed only over this smaller search space.
Full details are given in~\cref{sec: optimization}.
A better augmentation schedule might be possible if, like AuA, a more advanced automatic search was applied.
However, automatically searching data augmentation in adversarial training is extremely expensive, and was beyond the resources available to us.

IDBH improves diversity through the probabilistic multi-layered structure which results in a very diverse mixture of augmentations including individual transformations and their compositions.
We further diversify our augmentation by replacing the conventional crop and dropout methods, Padcrop and Cutout, in AuA and TA with their diversity-enhanced variants Cropshift and Random Erasing respectively.
IDBH enables balanced hardness, as the structure design and optimization strategy produce a much larger search space of hardness, so that we are able to find an augmentation that achieves a better trade-off between accuracy and robustness.

\section{Results} \label{sec: result}

We adopt the following setup for training and evaluation (fuller details in \cref{sec: experiment setting}). The model architectures used were Wide ResNet34 with widening factor of 1 (WRN34-1), its 10x widened version WRN34-10 \citep{zagoruyko_wide_2016} and PreAct ResNet18 (PRN18) \citep{he_identity_2016}.
The training method was PGD10 adversarial training \citep{madry_towards_2018}.
SWA was implemented as in \citet{rebuffi_data_2021}.
We re-optimized the strength of Cutout and Cutmix per model architecture.
AuA was parameterized as in \citet{cubuk_autoaugment_2019} since we didn't have sufficient resource to optimize.
TA is parameter-free so no tuning was needed.
For PreAct ResNet18, we report the result of two variants, weak and strong, of our method with slightly different parameters (hardness), because we observed a considerable degradation of best robustness on the strong variant when combined with SWA.
The reported results are averages over three runs, and the standard deviation is reported in \cref{sec: standard deviation data}.

\subsection{State-of-the-art Data Augmentation for Adversarial Training} \label{sec: data augmentation result}
\input{tables/da-robustness}
From~\cref{tab: data augmentation robustness} it can be seen that our method, IDBH, achieves the state-of-the-art best and end robustness for data augmentations, and its improvement over the previous best method is significant.
The robust performance is further boosted when combined with SWA.
Moreover, our method is the only one on WRN34-1 that successfully improves the end robustness to be higher than the best robustness achieved by the baseline.
On PRN18, IDBH[strong] improves the end robustness over the baseline's best robustness by $+1.78\%$, which is much larger than the existing best record ($+0.5\%$) achieved by AuA.
This suggests that data augmentation alone, contrary to the previous failed attempts, can significantly beat the baseline augmentation with ES. More importantly, our method also presents the highest best and end accuracy on these architectures, both  w. and w.o. SWA, except for WRN34-1 with SWA where our method is  very close to the best. Overall, our method improves both accuracy and robustness achieving a much better trade-off between them.

Data augmentation was found to be sensitive to the capacity of the underlying model. As shown in~\cref{tab: data augmentation robustness}, augmentations such as baseline, AuA and TA perform dramatically different on two architectures because they use the same configuration across the architectures. Meanwhile, augmentations like Cutout and ours achieve relatively better performance on both architectures but with  different settings for hardness. For example, the optimal strength of Cutout is 8x8 on WRN34-1, but 20x20 on PRN18. Therefore, it is vital for augmentation design to allow optimization with a wide enough range of hardness in order to generalize across models with different capacity.

\subsection{Benchmarking state-of-the-art robustness without extra data}

\cref{tab: regularization methods robustness} shows the robustness of recent robust training and regularization approaches. It can be seen that IDBH matches the best robustness achieved by these methods on PRN18, and outperforms them considerably in terms of best robustness on WRN34-10. This is despite IDBH not being optimised for WRN34-10.
In addition, our method also produces an end accuracy that is comparable to the best achieved by other methods suggesting a better trade-off between accuracy and robustness.
More importantly, the robustness can be further improved by combining SWA and/or AWP with IDBH. This suggests that IDBH improves adversarial robustness in a way complementary to other regularization techniques. We highlight that our method when combined with both AWP and SWA achieves state-of-the-art robustness without extra data. We compare our method with those relying on extra data in \cref{sec: extra data}.

\subsection{Generalization to Other Datasets} \label{sec: svhn tin}

Our augmentation generalizes well to other datasets like SVHN and TIN (\cref{tab: svhn tin}).
It greatly reduces the severity of robust overfitting and improves both accuracy and robustness over the baseline augmentation on both datasets.
The robustness on SVHN has been dramatically improved by $+7.12\%$ for best and $+13.23\%$ for end.
The robustness improvement on TIN is less significant than that on the other datasets because we simply use the augmentation schedule of CIFAR10 without further optimization. A detailed comparison with those regularization methods on these two datasets can be found in \cref{sec: reg comparison svhn tin}.
Please refer to \cref{sec: experiment setting} for the training and evaluation settings.

\subsection{Ablation Test}
We find that Cropshift outperforms Padcrop in our augmentation framework.
To compare them, we replaced Cropshift with Padcrop in IDBH and kept the remaining layers unchanged.
The strength of Padcrop was then optimized for the best robustness separately for w. and w.o. SWA.
As shown in~\cref{tab: padcrop cropshift}, changing Cropshift to Padcrop in our augmentation observably degrades both accuracy and robustness both w. and w.o. SWA.

\input{tables/reg-robustness}
\input{tables/dataset-ablation}

\section{Conclusion}
This work has investigated data augmentation as a solution to  robust overfitting.
We found that improving robust generalization for adversarial training requires data augmentation to be as diverse as possible while having appropriate hardness for the task and network architecture.
The optimal hardness of data augmentation is very sensitive to the capacity of the model.
To mitigate robust overfitting, we propose a new image transformation Cropshift and a new data augmentation scheme IDBH incorporating Cropshift.
Cropshift significantly boosts the diversity and improves both accuracy and robustness compared to the conventional crop transformation.
IDBH improves the diversity and allows the hardness to be better balanced compared to alternative augmentation methods.
Empirically, IDBH achieves the state-of-the-art accuracy and robustness for data augmentations in adversarial training.
This proves that, contrary to previous findings, data augmentation alone can significantly improve robustness and beat the robustness achieved with baseline augmentation and early-stopping.
The limit of our work is that we did not have sufficient computational resources to perform more advanced, and more expensive, automatic augmentation search like AutoAugment, which implies that the final augmentation schedule we have described may be suboptimal.
Nevertheless, the proposed augmentation method still significantly improves both accuracy and robustness compared to the previous best practice.

\section*{Acknowledgments}
The authors acknowledge the use of the research computing facility at King's College London, King's Computational Research, Engineering and Technology Environment (CREATE), and the Joint Academic Data science Endeavour (JADE) facility. This research was funded by the King's - China Scholarship Council (K-CSC).

\paragraph{Reproducibility Statement.}
Our methods including both Cropshift and IDBH can be easily implemented using the popular machine learning development frameworks like PyTorch \citep{paszke_pytorch_2019}. These two algorithms Cropshift and IDBH are illustrated in pseudo codes in \cref{algo: cropshift} and \cref{algo: IDBH} respectively.
The procedure of optimizing IDBH is described in detail in \cref{sec: optimization}. The full parameters of the optimal augmentation schedules we found are described in \cref{tab: augmentation schedule} and \cref{tab: color shape layer}. The training and evaluation settings are described in \cref{sec: result} and \cref{sec: experiment setting}. To further facilitate the reproducibility, we are going to share our code and the pre-trained models with the reviewers and area chairs once the discussion forum is open, and will publish them alongside the paper if accepted.

redpajama_set_name: RedPajamaArXiv


The Vietnam Veterans Memorial in Washington, D.C. is described on the National Park Service website as "a place where everyone, regardless of opinion, can come together and remember and honor those who served." Its purpose, we are told, is "to separate the issues of the sacrifices of the veterans from the U.S. policy in the war, thereby creating a venue for reconciliation." To facilitate reconciliation, separating "the issue of the sacrifice" from any controversy that may arise over foreign policy, the website itself conveniently omits any mention of the 58,191 names of the dead and missing inscribed in the Wall.
Elise Engler is not one to let us off so easily. Her Tax-Onomies record, among other eye-opening subjects, a total of nearly 15,000 dead, so far, in the war in Iraq: 12,000 faceless Arab civilians, the "collateral damage" of the war, as well as 2,500 ill-fated members of the "coalition of the willing," shrouded in their respective national flags.
The origins and development of writing and picture-making are closely tied to taking stock. Inventories of cattle, foodstuffs, and other property subject to commercial transactions are among the earliest evidence we have of efforts to record, in pictures and words, material goods. The Sumerians, in Iraq, and the Elamites, in Iran, nearly five millennia ago, made simple pictographs in clay tablets to record such things; these images evolved, over time, into a written language of cuneiform signs. Engler's long, narrow drawing panels, containing row after row of images, many of which are nearly identical, have strong resemblances to these ancient bookkeeping tablets. Her device of creating narrative through juxtaposition and scale recalls the picto-hieroglyphic language of ancient shallow relief sculpture.
More contemporary sources also appear to have left a mark on Engler's work. The whimsy and playfulness of Oyvind Fahlstrom's remarkable drawings surely come to mind as one rich source of creative nourishment. Magritte is another artist who is recalled here, both for his use of repetition, and his knack for convincing us of the physical ability objects and people to be suspended in mid-air: floating men and clouds and crescent moons and baguettes. The scale of people and things in Saul Steinberg's world, as well as the rubber stamps he employed, are surely strong influences for Engler, as is the populous, color-bathed universe of Florine Stettheimer.
Much of the history of art is comprised of the efforts of artists to capture a world teeming with people and things. This is as evident in the dry goods stacked in the windows of Walker Evans' storefronts, as it is in the ancient stele of Mesopotamia. Abundance is celebrated; on occasions it is the subject of derision. Material riches are simultaneously festive and foreboding. I'm thinking here, for example, of the lush fruits and vegetables used by Carlo Crivelli in several of his paintings of the Virgin and Child. On one level the pickle/zucchini forms and bulbous fruit are metaphors for the concealed body parts of both the Virgin and Child. These foodstuffs are also evidence of the redemption, renewal, and vitality ushered in by the Holy Family. Fresh fruit—reminders of the forbidden nibbling in the Garden of Eden. However, there is also an undercurrent here, in the midst of all of the splendor and fecundity: our knowledge of what lies ahead.
The dark side of plenitude is one of the more pervasive and striking aspects of Elise Engler's drawings. As ombudsman, we expect her to knock some sense into us, and make us face the facts of how our tax dollars are squandered on killing, over-outfitting virology labs, or supplying the Defense Department with daisy cutters. Her outpouring of images--the sheer wastefulness of resources, human and material--is nauseating. However, the effects of her investigations are no less disturbing in the face of far more innocent inventories, be they collections of chairs, the contents of refrigerators, women's handbags, or all the stuff kept in the family car. Individual responses to excess obviously vary.
On a visit, in 1950, to the Metropolitan Museum of Art, Hemingway identified a Breughel as one of his all-time favorite paintings. He remarked to Lillian Ross that "It's the great one, the harvesters. It is a lot of people cutting grain, but he uses the grain geometrically, to make an emotion that is so strong for me that I can hardly take it." Faced with the richness of the harvest itself, the weary harvesters, and the luxurious mood of restfulness and abandon, setting aside their labors, the painting apparently triggered something in Hemingway that could not be anticipated.
In his memoir of life on a New Hampshire farm, String Too Short to be Saved, the poet, Donald Hall, describes the relationship between repeated motions (those associated with harvesting) and one's personal knowledge of death. The perpetual motion associated with the harvesting suggests endlessness, perpetuity, at odds with one's personal elegiac sense of doom and end. As Hall says, these two feelings "contradicted each other, but lived together like old brothers who had not spoken for forty years." Perhaps it's something along these lines that Hemingway was reacting to in the Breughel painting. It is this aspect of bountifulness in Engler's work, coupled with the deceptively straightforward manner of its execution, that I find unsettling and disquieting.


text:
The Blackburn Skua; Fighting Dive-Bomber or Dive-Bombing Fighter?
TRANSCRIPT FOR VIDEO:
The Blackburn Skua is, like many other contemporary aircraft designs of the mid-1930s, an aeroplane that had good performance when it was first proposed but which was outclassed when it entered service.
Despite this the aircraft gave solid service at the start of the Second World War, serving through some of the hardest battles and managed to chalk up a good record in the limited span that it was in combat.
Designed to a 1934 requirement for a two-seat carrier-based dive bomber, the Skua would have its specifications changed to also act as a fighter.
This is not a great idea, as the requirements for these two types are very different. Any fighter that has to carry a second crewman is going to be at a great disadvantage to a single seat fighter.
But there were good reasons for the decision.
Firstly, the Fleet Air Arm (the organisation that flies the Royal Navy's aircraft) understood the difficulties at that time of trying to fly and navigate alone across open ocean. A second crewman, who could assist in this by using specialist equipment, was a massive help to a pilot operating at sea.
It's notable that this idea was so entrenched that it was even stipulated in later carrier fighters like the Fairy Fulmar.
The FAA also didn't plan for its carrier aircraft to be tangling with land-based fighters.
British thinking before the Second World War was that the Royal Navy's carriers and fleets would primarily defend themselves with guns from air attack.
Their aircraft carriers featured heavily armoured decks that could shrug off attacks from bombers and protect aircraft within the ship when under attack.
This heavy armour, combined with the doctrine of not using the deck to store aircraft as was the norm within the US and Japanese navies at the time, meant that Royal Navy carriers operated much fewer numbers of aircraft than their contemporaries.
As a result it made sense to have those aircraft act as "jack-of-all trades" because they were available in such limited numbers.
As an aside, If you are interested in understanding more about the Royal Navy's carrier philosophy, their carriers, aircraft and the battles they fought, you should check out armouredcarriers.com.
They're an excellent source on this subject and I'll put a link to their website in the description and to their Youtube channel at the end of the video.
The Skua was designed to act as a naval dive-bomber with the added ability to shoot down reconnaissance aircraft that were shadowing the fleet. It could also help protect against enemy bombers.
To meet these requirements the Skua had a pretty respectable armament for the time. Four .303 Browning machine guns were mounted in the wings, added to by a single Lewis or Vickers K machine gun operated by the navigator to protect the rear of the aircraft.
It carried a 500-lb bomb under the fuselage on a bomb crutch as its main striking weapon, and could also carry small 20 and 40-lb bombs under the wing.
As pointed out, the Skua was a compromise design and its performance showed that. Fitted with a Bristol Perseus radial engine, which produced just over 900hp with emergency boost, the aircraft's maximum speed on paper was only 225mph.
But in many ways the Skua was a thoroughly modern aircraft, especially in contrast to the planes it replaced. It was the FAA's first monoplane aircraft, as well as it's first with retractable landing gear and variable pitch propellers.
Entering service aboard HMS Ark Royal in November 1938, the Skua was the FAAs' main aircraft at the outbreak of World War Two. And it went into action almost immediately.
On 14 September, 1939, Skua's attacked U-30 as it shelled a merchant ship. Unfortunately, the strike didn't go well as two of the aircraft were damaged by the blast from their own bombs, causing them to ditch.
The U-30 was able to return to Germany with the downed crews aboard, the first naval airmen to be captured in the war,
Better success followed on the 26th when three Skua's managed to shoot down a German Dornier Do18 flying boat over the North Sea.
Things began to pick up for the Skua's when the German's invaded Norway in April, 1940.
On the 16th of that month Skua's attacked the light cruiser Konigsberg in Bergen Harbour, sinking her in what was both the first major warship sunk in war by air attack and the first major warship ever to be sunk by dive bombing in combat.
Skua's would also enjoy a fair amount of success against German bombers over Norway in their fighter guise, being credited with the shooting down of forty aircraft and even one pilot, Lieutenant Commander William Lucy, making ace in the aircraft.
The Skua's fought hard through the Norwegian Campaign, hitting ground targets and shipping, but suffered badly against fighters like Bf 109s – which was exactly the sort of aircraft they were supposed to NOT tangle with.
A raid by Skua's on the battle cruiser Scharnhorst on the 13th June, 1940 showed up the limitations of the type. Of fifteen aircraft launched, eight were shot down battling through the defending fighters and heavy flak clouds. Only one bomb hit the vessel, inflicting no damage.
Though it was appreciated that the Skua was in need of replacement, the aircraft would see use for a few more years. They served covering the British withdrawal from Dunkirk, losing some of their numbers to RAF Spitfires who weren't familiar with the Skua nor the FAAs camouflage patterns.
They would also serve in the Mediterranean, even having to fight French fighters after that nation's capitulation and helping defend the critical convoys that resupplied Malta.
The types final battle came on 6th February 1941 when, during Force H's operation to bombard Genoa, an Italian Cant 506 float plane was shot down.
Soon after this the Skua was taken out of front line service and relegated to target towing and advanced trainers. The last one was struck from service in March 1945.
Other recommended links:
https://dingeraviation.net//SkuaRoc/blackburn_skua.htm
https://dingeraviation.net//SkuaRoc/norway.htm#victories

redpajama_set_name: RedPajamaCommonCrawl


*Attention Investors* This is a great opportunity and the sellers are eager to sell. This is a fixer upper and the home is being sold œAS IS. Home does have foundation damage.
Blue Ridge Boulevard, East on 90th Terr., North on Manchester to address.
Kansas City is located in Missouri. Kansas City, Missouri 64138 has a population of 475,378.
The median household income in Kansas City, Missouri 64138 is $45,376. The median household income for the surrounding county is $46,917 compared to the national median of $53,482. The median age of people living in Kansas City 64138 is 35 years.
This content last refreshed on 4/19/2019 3:29 PM. Some properties which appear for sale on this web site may subsequently have sold or may no longer be available.
I am interested in 8914 Manchester Avenue, Kansas City, MO 64138.


At Eerde we love all the seasons Mother Nature gives us! During the winter we can not wait until snow kicks in and we can have snowball fights! We took an hour off and the students plus staff had so much fun, sliding and gliding through the snow, and of course snowball fighting! The hour ended with hot chocolate. We can't wait for the next snow to come!


text:
Islamic Tolerance Alert
Despit Hamas and Fatah's trying to strike a moderate tone, they remain as committed as ever to genocide against the Jewish People. Hamas Spokesman, Dr. Ismail Radwan reiterated:
1. The Hour – the Islamic Resurrection and End of Time– is literally dependent on the killing of Jews by Muslims.
2. The remaining Jews will unsuccessfully attempt to hide, as the rocks and trees will expose them, calling out "there is a Jew behind me, kill him!"
3. "Palestine… will be liberated through the rifle," a euphemism meaning that Israel will be destroyed through violence.
The Hamas spokesman ended with prayers to Allah to "take" Israel and the USA.
The following is the text of the Hamas spokesman's call for genocide of Jews:
The Hamas spokesman, Dr. Ismail Radwan, PA TV, March 30, 2007 -
The Hour [Resurrection] will not take place until the Muslims fight the Jews and the Muslims kill them, and the rock and the tree will say: "Oh, Muslim, servant of Allah, there is a Jew behind me, kill him!"
We must remind our Arab and Muslim nation, its leaders and people, its scholars and students, remind them that Palestine and the Al Aqsa mosque will not be liberated through summits nor by international resolutions, but it will be liberated through the rifle. It will not be liberated through negotiations, but through the rifle, since this occupation knows no language but the language of force… O Allah, strengthen Islam and Muslims, and bring victory to your Jihad-fighting worshipers, in Palestine and everywhere… Allah take the oppressor Jews and Americans and their supporters!
Here, also, is a video of a Palestinian girl singing a song of praise to her mother Reem Riyashi, a suicide murderer bomber. Don't worry, as long as Israel retreats to the pre-67 border and allows for the Right of Return, that should solve everything...
Brooke said...
I am floored that Olmert (moron) would even entertain the notion of the pre-'67 border. The mere suggestion should have been met with a carpet bombing of the moron who voiced it.
Sulayman said...
Way to totally misrepresent.
The "Muslims fighting Jews" hadith you and he are alluding to is talking about the apocalyptric end-times, after Jesus (peace be upon him) returns and there's a war against the Dajjal (antichrist). Nice try, but even the Christian zionists have a bleak ending for the Jewish people after that fact.
As for the Palestinian girl, it's contemptible and Muslims have condemned the far right for things like that. That said, you should see some Jewish newspaper raving about the "ha'avrim" (Arabs) in similarly unflattering terms.
Bar Kochba: this is hardly a euphemism. These terrorists are never shy about saying exactly what they mean. It's the rest of the world that is shy about understanding them.
Anti-Zionism = Anti-Semitism
Updates from Wackistan
Annex and Transfer
Return to Homesh
These Dry Bones Shall Live Again
Happy Birthday Israel!
Moral Equivalency?
O that Damn Cycle of Violence!
Yom Hazikaron- Remeberance Day for the Fallen Hero...
Thirty Three Murders
Have the Germans done Tshuva?
3 killed in Severe Inner Struggle' (Jihad)
Never Again?
Olmert meets with Holocaust Denier
Prisonner Exchange
Some Post-Passover Clarity...
Passover Gift


text:
Shahrabad Rural District () may refer to:

Shahrabad Rural District (Razavi Khorasan Province)
Shahrabad Rural District (Firuzkuh County), Tehran province

redpajama_set_name: RedPajamaWikipedia


text:
Q: How to use purrr tilde function with multi-line formulas I would like to use purrr's short-hand formula specification with multi-line formulas.
For example, I would like to repurpose the following formula:
purrr::imap_chr(sample(10), ~ paste0(.y, ": ", .x))

And do the following instead:
purrr::imap_chr(sample(10), ~
     b <- 10
     paste0(.y, ": ", .x + b))

However, I receive an error when I attempt to do this:

Error: unexpected symbol in:
  "           b <- 10
             paste0"

A: We can use braces { to create multi-line formulas
purrr::imap_chr(sample(10), ~ {
 b <- 10
 paste0(.y, ": ", .x + b)
 })
#[1] "1: 18"  "2: 16"  "3: 19"  "4: 11"  "5: 13"  "6: 12"  "7: 15"  "8: 14"  "9: 17"  "10: 20"

redpajama_set_name: RedPajamaStackExchange


Print beautifully coloured documents with the HP Q7581A LaserJet 503A Toner Cartridge, compatible with HP LaserJet CP3505, 3800 printers. This toner allows you to print large quantities while not having to sacrifice on quality. This reliable ink cartridge yields about 6,000 standard pages.


text:
Q: No Suitable Constructor Found Incredibly new to Java so forgive the simplicity in this issue. I've looked through posts on this issue, but none seem to help.
This is the error I'm getting; 'no suitable contructor found for AQAWriteTextFile2016(String)' for the code below.
String filename = "src/warships/resources/PreviousGame.txt";
AQAWriteTextFile2016 write = new AQAWriteTextFile2016(filename);

I apologise if the solution is something simple, I've checked through the AQAWriteTextFile2016 class and all it needs to operate is a String parameter.
AQAWriteTextFile2016(String filename) {

openFile(filename, false);

}

  public void openFile(String filename, boolean append) {
try {

  outputFile = new BufferedWriter(new FileWriter(filename, append));

} catch (IOException e) {
  e.printStackTrace();
} // end try/catch open file
} // end openFileString

A: From the snippet in your code, we see that the constructor has no access specifier. So it has the default visibility, which is package-private. It can only be seen from within the same package.
If the call to the constructor is made from a different package, then it cannot be seen from there.
Hence, you should make the constructor public, or make the call from within the same package.

A: It might be that what you think as the constructor with String argument is actually a method having name AQAWriteTextFile2016. Please look closely at the return type.  A constructor should not have any return type. Hope it helps.


If you by any chance happen to be in Hollywood, you should take Warner Bros. Studio Tour (costs around $60). Not only will you walk on the backlot where the streets of Los Angeles, 2019 was filmed, You also get to see some Hollywood memorabilia and among those items are the Blimp (whom told everyone to go to the colonies, a message paid for by The Shimago/Dominguez Corporation). Brian Boskind was kind enough to thru the magic of Facebook let me know that the Blimp was there on display and he took some pictures of it that I got his approval to publish.
The blimp is very detailed but is also very hard to get a full and detailed view of in the movie. On these pictures You can see details that I guess not may people has seen before.
A user called "minicity" has uploaded 142 stunning photos to imgur from the studio in which they built some of the miniatures used in Blade Runner back when the movie was in production. These are truly a set of spectacular images and contains more than just miniature builds. I took a quick decision and decided to upload all of the images here too (since there was an option to download a .zipped version with all the images) AND to be sure that they will be available for everyone to enjoy.


WESTVILLE – When Allan Stewart was asked why the MacDonald Cup for palliative care is such an important charity to him, his words came easily.
The 15th annual MacDonald Cup in memory of Walter MacDonald will be played at Glen Lovat Golf Club this Sunday starting at 1 p.m. As of a week ago, there were three spots still available for this year's event.
There will also a MacDonald Cup fund-raiser on Saturday at the White Tail pub featuring comedian Jimmy Flynn.
Opening for Jimmy Flynn will be Little Smoke with special guest Paul Tupper.
Stewart also donated two VIP tickets for the 2017 Cavendish Beach Music Festival.
Tickets for these passes will be available at the Jimmy Flynn event.
Those wishing to golf or for tickets to the Jimmy Flynn event may call Marlee MacNaughton at 396-6394 or Marlene MacNaughton at 396-1483. Marlee MacNaughton can be contacted for more information.


Gnarly-looking Exige Sport 380 sheds 25kg and sprouts race aero. GT4 trade-in time?
In the last couple of years, Lotus has steadily worked through its range of light, fast cars, making them lighter and faster.
We've had the Exige Sport 350, the Elise Sport 250, and the Evora Sport 410. Less weight, more power. Heartland Lotus stuff. So much fun, in fact, they're going around again.
Meet the brand new Lotus Exige Sport 380. Somehow, Norfolk's techies have shaved 25kg out of the Exige – which was hardly plush with equipment, dropping it down to 1066kg without fuel or washer fluid, or 1100kg ready to go (if you've added the titanium exhaust). That's damn light for a car with a big ol' 3.5-litre supercharged Toyota V6 bolted between the axles, wearing wider Michelin Pilot Cup tyres.
Lashings of carbonfibre add the lightness: the Elise Cup rear wing and diffuser, plus new splitter and front panels, save 2.7kg. Slimmer seats save 6kg.
Ultra light wheels and brakes save a further 10kg, with a standard lithium-ion battery cutting out more weight. Short of filling the cabin with helium, it's difficult to know how much lighter the Exige can get.
To this flyweight recipe, Lotus has added power. A revised supercharger pulley (for higher charge pressure), an uprated fuel pump, a recalibrated ECU and shoutier exhaust team up to add 30bhp and 7lb ft to the Exige Sport 350's heady totals.
The Sport 380, as a result, chucks out 375bhp, 302lb ft and will hit 62mph in 3.5 seconds. Despite generating 140kg of claimed downforce (check out those 997 RS-style front aero flics), the top speed is 178mph. Adjustable suspension is standard. Thought the Exige was a dainty little performer, did you? Forget it.
So, it's a lot faster and lighter than a Porsche Cayman GT4 (which is sold out), and would stay with a Porsche 911 GT3 (which is sold out). Maybe the BMW M4 GTS would…wait, no that's sold out too.
If you want your track day kicks, the Exige (not sold out, and yours for £69,700) looks like a mighty good option. It's £11,000 more than the current Exige Sport 350, but the rest of its numbers are positively supercar-spec.


Ruth M. Carl, age 71, of Poplar Bluff, Mo., passed away Sunday, April 28, 2013 in the Cedargate Nursing Home.
Mrs. Carl was born June 27, 1941, in Poplar Bluff, to the late Blake R. and Mary (Timmons) Birdsong. She was a beautician and enjoyed needle point, crochet, and reading.
She is survived by her children, Debra Abu-Raqabeh of St. Charles, Mo., Robert Anthony Carl of Sealy, Tx., George A. Carl Jr. ,of Lake Providence, Louisiana, Thomas Carl and Chris Carl, both of Poplar Bluff; three sisters, Anita Birdsong, Alene Sweeza and Janis Birdsong, all of Poplar Bluff; three brothers, Russell Birdsong, Harvielle, Mo., Teddy G. Birdsong and Blake T. Birdsong, both of Poplar Bluff; her grandchildren, Christina Carl, of Poplar Bluff, Anthony Wayne Bales, Ellsinore, Mo., Susan Dorris, St. Charles, Mo., Amber Carl, Poplar Bluff, Christy Francis, Dallas, Tx., and Farrah Carl, of Poplar Bluff. Six great-grandchildren also survive.
Also preceding her in death are her sister, Mary Stewart; her brother, Bill Birdsong and one grandson, Samuel Smith.
Visitation for Mrs. Carl will begin at 10 a.m. Saturday, May 4, 2013, followed by funeral services at 11 a.m., at Fowler-Sullivan Funeral Home, 844 Cedar, Poplar Bluff, Mo., with the Rev. Keith Luke officiating. Burial in the Black Creek Cemetery.
James "Norman" Bailey, age 84, passed away at his home in Colorado Springs, Colorado, on Tuesday April 24, 2013.
Norman was born on September 24, 1928 in Clay County, Ark.
He moved with his family to Poplar Bluff, Mo. as a child, where he later met and married Sue Alexander McKinney in February, 1967. He retired from Kelly Electric as a foreman in 1993. He moved with his wife to Colorado Springs in May of 2009, to be closer to his son and grandchildren. Norman enjoyed woodworking, fishing, hunting, and antiquing. He spent many happy hours at the Sale Barn.
He is preceded in death by his parents, James E. and Nota (Stafford) Bailey; five brothers and two sisters.
He is survived by his wife, Sue; his son, Richard McKinney and wife, Denise; one grandson Steven McKinney and wife, Bjana; one granddaughter, Sarah McKinney and husband, Hector; and five grandchildren, Taylor, Caitlyn, Inesia, Isaac, and Eloise, all of Colorado Springs. He is also survived by one brother, Olden Bailey; numerous nieces, nephews, and friends; and a son, Jim Bailey, by a previous marriage.
Norman was a kind and gentle man who was beloved by his family and will be greatly missed.
A graveside service will be held at 10 a.m. Friday, May 3, 2013 at Mt. Zion Cemetery, on W Highway in Williamsville, Mo., with Rev. Ralph Mayes officiating. Arrangements made by Fowler-Sullivan Funeral Home, 844 Cedar Street, Poplar Bluff, Mo.
Vicki Reynolds,age 50, of Williamsville, Mo., passed away Friday, April 26, 2013, at her residence.
Mrs. Reynolds was born November 12, 1962, in Phoenix, AZ, to her parents, William Eugene and Tresa Jane (Price) Abney, who survive. Also surviving are her husband, Elmo Reynolds; children, Donald Gilliam, Shawn Gilliam, Jason Reynolds, Roy Reynolds, Matilda Reynolds, and Kenneth Reynolds; one sister, Christy Gibson, and one brother, Billy Abney and 15 grandchildren.
Visitation will begin at noon Monday April 29, 2013, at Fowler-Sullivan Funeral Home, 844 Cedar Street, Poplar Bluff, Mo. followed by the funeral service at 1p.m., in the chapel, with Ron Fowler officiating. Burial will be in Black River Cemetery, JJ Hwy, near Williamsville, Mo.
James Albert May, age 66, of Grandin, Mo., passed away Sunday, April 21, 2013, in the Poplar Bluff Regional Medical Center.
Mr. May was born July 15, 1946, in Newton, Kansas, to the late Virgil Thomas and Lillian L. (Jantz) May. He served in the U.S. Air Force and later retired as a cabinet maker. He enjoyed fishing and was a collector.
He was married to Wilma (Sapp) May in Newton, Kansas. She survives. He is also survived by two sisters, Carol and Peggy and two brothers, Tommy and Melvin.
Emily Marie Knuckles, age 74, lifetime resident of Poplar Bluff, Mo., passed away April 20, 2013 at her home.
Mrs. Knuckles was born May 16, 1938 in Parma, Mo. She was a cook for the Missouri Youth Services and enjoyed cooking and being with family and bowling. She was of the Baptist faith.
She is survived by her children, Kathy Wooten and husband, Keith of Grandin, Mo., Thomas Knuckles Jr. and wife, Robin of Ellsinore, Mo., Billy Knuckles and Carla Knuckles both of Poplar Bluff, Joyce Green (honorary daughter) of Poplar Bluff; three sisters, Wilma Teachener of Poplar Bluff, Mary Riley and husband, Mickey of Malden, Mo., Carol Todd and husband, Jerry of Tiptonville, Tn.; three brothers, Chester Daves , Freddie Daves and wife, Sharon, and Marvin Berry and wife, Robin all of Poplar Bluff; grandchildren, Kasie Knuckles, Thomas A. Knuckles, Emily Knuckles all of Poplar Bluff, Kristi Law of Neelyville; God grandsons, Jason Holder of Poplar Bluff and Kyle Holder of Afghanistan; great- grandchildren, Katlyn Burris of Williamsville, Brooklynn Thayer of Grandin, Aubree Knuckles of Poplar Bluff, Ashley Law and Jacen Litterst of Neelyville; and one great-grandchild, Brendon Wooten (who is on the way); special friends, Donnie Vickery, Betty Richardson, Opal Mowery and many nieces and nephews.
Mrs. Knuckles was preceded in death by a daughter, Anita K. Knuckles; her parents, Jesse Daves (December 31, 1992) and Alpha (Bridges) Daves Berry (January 12, 1984); one sister, Beulah Reese, and one great- grandchild, Maria Fay Knuckles.
Visitation for Mrs. Knuckles will be held at 7 p.m. Tuesday April 23, 2013, at Fowler-Sullivan Funeral Home, 844 Cedar Street, Poplar Bluff, Mo. Funeral services at 1p.m. Wednesday April 24, 2013 in the chapel, with Rev. Shelly Elkins officiating. Burial in Memorial Gardens Cemetery, Poplar Bluff, Mo.
In lieu of flowers, the family request memorial be made to UCAN.
Ethel Marie Allbritton, age 78, of Poplar Bluff, Mo., passed away Saturday, April 20, 2013 in the Poplar Bluff Regional Medical Center.
Mrs. Allbritton was born October 20, 1934 in Neelyville, Mo. She was a homemaker and enjoyed gardening and flowers. She was a member of White Hollow General Baptist Church in Clubb, Mo.
On January 8, 1951, she married Rev. Harold Leroy Allbritton, who preceded her in death January 16, 2013.
Mrs. Allbritton is survived by nine children, Darwin Allbritton and wife, Kathy of Jonesboro, Ar, Delores Hill and husband, Larry of Poplar Bluff, Linda Hall and husband, Gary of Poplar Bluff, Dewayne Allbritton of Piedmont, Lisa Young and husband, Jeff, Valley Park, Mo., Steven Allbritton and wife, Karen, Lowndes, Mo., William Allbritton and wife, Michelle of Jonesboro, Ar., John Allbritton, Poplar Bluff, and Debra Hendrix and husband, Kevin, of Doniphan; two brothers, Bill Henthorn of Mt. Vernon, Ia. and Loren Hentorn of Laughlin, Nevada; twenty-three grandchildren, and seventeen great-grandchildren.
She is also preceded in death by her parents, Walter Henthorn and Flora Bell (Cook) Cope; two sons, Michael "Mike" Allbritton and Teddy Allbritton; one sister, Melba Myers and one brother, Don Henthorn.
Visitation for Mrs. Allbritton will begin at 1 p.m. Tuesday, April 23, 2013 at Fowler-Sullivan Funeral Home, 844 Cedar Street, Poplar Bluff, Mo. , followed by funeral services at 2 p.m. in the chapel with Rev. Allen Leutert officiating. Burial in the Poplar Bluff City Cemetery.
Henrietta Whitehorn, age 63, of Poplar Bluff, Mo., passed away Wednesday, April 17, 2013 at her home.
Mrs. Whitehorn was born July 9, 1949, at Caruthersville, Mo., to the late Henry and Mattie (Harris) Whitehorn. She was a former resident of St. Louis, Mo., but lived in Butler County for the past 27 years and worked at Gates Rubber Co. in Poplar Bluff. She enjoyed her family, watching T.V. and going to church. She was a member of Lively Stone Church of God.
She is survived by three children, Rhonda Higgs, Tiffany Harris, and Lance Whitehorn; eight sisters, Mattie Griffin, Juanita Maxwell, Onietta Whitehorn, Gladys Hayes, Betty Sue Whitehorn, Cathaleen Jackson,Stella Hunt, and Florine Parson; two brothers, Larry Whitehorn and Charles Whitehorn; seven grandchildren, Dominique Higgs, Candace Higgs, Darius Higgs, Darian Higgs, Daliecia Harris, Jasmine Harris, and Kaleb Whitehorn; one great-grandchild, ZaMiyah Higgs.
Mrs. Whitehorn is also preceded in death by one son, Bobby Whitehorn and one grandchild, Dio Janay Higgs.
Visitation will be from 6-8 p.m. Friday, April 26, 2013, at Fowler-Sullivan Funeral Home, 844 Cedar Street, Poplar Bluff, Mo. Funeral services will be held at 12 noon, Saturday, April 27, 2013 in the funeral home chapel, with Elder Jeffery Price officiating. Burial in the Poplar Bluff City Cemetery.
Lois Geneva Plunk,71, of Harviell, Mo., passed away Saturday April 13, 2013 at her home.
Mrs. Plunk was born December 15, 1941 in Harrisburg, Arkansas, to the late William Jackson and Virginia (Eaton) Jackson. She was married to Harlen Plunk on May 2, 1981, in St Louis, Missouri. He survives.
She was a homemaker and enjoyed fishing, being with family, watching old movies, playing cards, cooking, and she loved having company. She was a member of the Harviell First Baptist Church.
She is also survived by her children, Robert Dunscombe and wife, Lisa and Sonja Ryan and husband, Kevin; two sisters, Flora Rice and Alice Winters; two brothers, Jerry Jackson and Dean Jackson; four grandchildren, Jamie Kleyn, Danielle Williams, Matthew Dunscombe and Lindsay Ryan; four great- grandchildren, Ryan, Nathan, Logan, and Mia.
Visitation for Mrs. Plunk beginning at 10 a.m. Wednesday, April 17, 2013, at Fowler-Sullivan Funeral Home, 844 Cedar Street, Poplar Bluff, followed by funeral service at 12 noon in the chapel, with Rev. Tommy Knodell officiating. Burial will be in Black Creek Cemetery in Poplar Bluff.
Family requests, in lieu of flowers that memorials be made to American Lung Association, 1118 Hampton Ave, St Louis, Missouri 63139.
Deloras A. Payne of Poplar Bluff, Mo. passed away Monday, April 8, 2013, at the Poplar Bluff Regional Medical Center.
Mrs. Payne was a former resident of St. Louis, Mo. and retired from the U.S. Federal Government Dept. of Justice. She enjoyed her family, southern gospel music, reading, decorating, flowers, figure skating, travel, piano, and St. Louis Cardinal Baseball. She presently attended Westside Church of God, but was a member of her home church, First Church of God.
She is survived by one daughter, Lori Dawn Ethridge; two brothers, Larry Dare and wife, Judy and Terry Dare and wife, Jackie; her grandchildren, Scotty Engle II and Regina Ethridge; great-grandchildren, Ariana Engle and Kaylee Ethridge as well as several nieces and nephews.
She is preceded in death by her parents, Marvin Eugene and Opal Belle (Hendrix) Dare.
A graveside service will be held at 10 a.m. Thursday, April 11, 2013 at Memorial Gardens Cemetery in Poplar Bluff with Pastor Bill Wiles officiating. Fowler-Sullivan Funeral Home, of Poplar Bluff, in charge of arrangements.
Edna Mae (Moore) Walker,age 95, of Poplar Bluff, Mo., passed away Friday April 5, 2013, at the Oak Dale Care Center.
Mrs. Walker was born October 24, 1917 in Oxly, Missouri, to the late Floyd and Rosie (Sheridan) Moore. A former resident of Doniphan, Mo., she had lived in Poplar Bluff for 14 years. She was a homemaker, who enjoyed gardening, flowers and quilting.
On July 6, 1933, she married Fred Walker at Oxly, Mo. He preceded her in death on May 13, 1976.
Survivors include her children, Ethel Gillespie (Glen), of Cold Water, Mo., Bonnie Burns of Poplar Bluff, Bob Walker (Mayjean) of Belknap, Ill., Don Walker (Naomi) of Monet, Mo. Patricia Bickford (Archie) of Poplar Bluff, Bill Walker (Kathy) of St. James, Mo., Linda Snodgrass, Cape Girardeau, Fred Walker (Sabrina), Apache Junction, Az., Brenda Stenger of Poplar Bluff and John Walker (Beverly) of Poplar Bluff; two sisters, Anna Bell (Billy) Vick and Rose Mary Fisher of Trinity, Tx.; 27 grandchildren and several great-grandchildren.
She is also preceded in death by one son, Jim Walker; and several brothers and sisters.
Services for Mrs. Walker will be Sunday April 7, 2013, with visitation beginning at 12 noon, followed by funeral services at 2 p.m. at Fowler-Sullivan Funeral Home, 844 Cedar Street in Poplar Bluff, with Pastor Marvin Creelman officiating. Burial will be in Gum Cemetery, near Naylor.