TR2-D2 / tr2d2-dna /configs_gosai /lr_scheduler /cosine_decay_warmup.yaml

zyc4975matholic

Include DNA training code

303c2e0 2 months ago

213 Bytes

	_target_: utils.CosineDecayWarmupLRScheduler
	t_in_epochs: False
	t_initial: ${eval:${trainer.max_steps}-${.warmup_t}}
	warmup_prefix: True
	warmup_lr_init: 1e-6
	warmup_t: ${eval:0.1*${trainer.max_steps}}
	lr_min: 1e-6