Update README.md
Browse files
README.md
CHANGED
|
@@ -19,6 +19,11 @@ This is what the 'caution' addition looks like in an optimizer:
|
|
| 19 |
exp_avg = exp_avg * mask
|
| 20 |
```
|
| 21 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 22 |
|
| 23 |
# LaProp
|
| 24 |
|
|
@@ -74,4 +79,5 @@ This is what the 'caution' addition looks like in an optimizer:
|
|
| 74 |

|
| 75 |
|
| 76 |
## MARS Train Loss
|
| 77 |
-

|
|
|
|
|
|
| 19 |
exp_avg = exp_avg * mask
|
| 20 |
```
|
| 21 |
|
| 22 |
+
Train args:
|
| 23 |
+
|
| 24 |
+
```
|
| 25 |
+
./distributed_train.sh 2 --dataset hfds/timm/mini-imagenet --num-classes 100 --model vit_wee_patch16_reg1_gap_256 -j 8 --epochs 200 --warmup-prefix --sched-on-updates --warmup-lr 0 --mixup .2 --model-ema --model-ema-decay 0.999 --model-ema-warmup --aa rand-m9-mstd0.5-inc1 --remode pixel --reprob 0.25 --amp --weight-decay .05 --drop 0.1 --drop-path .1 -b 288 --opt cadamw --lr 1e-3
|
| 26 |
+
```
|
| 27 |
|
| 28 |
# LaProp
|
| 29 |
|
|
|
|
| 79 |

|
| 80 |
|
| 81 |
## MARS Train Loss
|
| 82 |
+

|
| 83 |
+
|