Predictive Modeling in Sequential Recommendation: Performance Laws with Quality Measure Insights
Tingjia Shen, Hao Wang, Chuhan Wu, Jin Yao Chin, Wei Guo, Yong Liu, Huifeng Guo, Defu Lian, Ruiming Tang, Enhong Chen
With the growing size of data and models in Sequential Recommendation (SR), the time required for debugging has become increasingly prohibitive, underscoring the urgent need for effective guidance in parameter configuration. The Scaling Law (SL) offers analogous guidance in the language domain, having achieved significant success by predicting model loss when scaling model size. However, the existing guidance from SL for SR remains qualitative, which is because quantitative analysis of SL on SR encounters challenges with quality measurement on redundant sequence along with loss-performance discrepancy. In response, we introduce the \textbf{Performance Law} for SR models, which predicts model performance across various settings, intending to provide a quantitative framework for guiding the parameter optimization of future models. Initially, Performance Law utilizes Real Entropy to measure data quality, aiming to remove the low-quality influence of low-entropy redundant sequences. Subsequently, Performance Law investigates a fitting decay term, which facilitated the prediction of the major loss-performance discrepancy phenomena of overfitting, ultimately achieving quantitative performance prediction. Extensive experiment on various datasets demonstrates the effectiveness of Performance Law by displaying exceptional quantitative prediction ability against the original and modified qualitative SL. Additional application experiments on optimal parameter prediction and model expansion potential prediction also demonstrated the broad applicability of the Performance Law. Our anonymized code is available at https://anonymous.4open.science/r/Performance-Law-DDE3/