Unveiling the Role of Learning‑Rate Schedules via Functional Scaling Laws
Binghui Li, Zixun Huang, Fengling Chen, Lean Wang, Lei Wu
Scaling laws have played a cornerstone role in guiding the training of large language models (LLMs). However, existing theoretical analyses primarily focus on the final-step risk, overlooking the intermediate-step risk and, crucially, the impact of learning rate schedule (LRS). In this paper, we aim to bridge this gap by studying a teacher–student kernel regression task trained via online stochastic gradient descent (SGD). Leveraging a novel **intrinsic time** viewpoint, we derive a  **Functional Scaling Law  (FSL)** that applies to general learning-rate and batch-size schedules. Remarkably, the impact of the schedules is captured through anexplicit convolution-type functional term, making their effects fully tractable. To illustrate the utility of FSL,  we analyze three widely used LRSs -- constant, exponential decay, and warmup-stable-decay (WSD) -- under both data-limited and compute-limited regimes. Our analyses provide theoretical justification for  widely adopted empirical practices  in LLMs pre-training such as (i) higher-capacity models are more data- and compute-efficient; (ii) learning rate decay can improve training efficiency; (iii) WSD-like schedules can outperform direct-decay schedules.Lastly, we also explore the practical relevance of FSL as a surrogate model studying the  loss convergence of LLM pre-training.Overall,  FSL offers a principled approach for understanding the role of learning-rate and batch-size scheduling in scaling laws and, more broadly, in stochastic optimization.