Nonparametric Quantile Regression with ReLU-Activated Recurrent Neural Networks
Hang Yu, Lyumin Wu, Wenxin Zhou, Zhao Ren
This paper studies nonparametric quantile regression using recurrent neural networks (RNNs) and sparse recurrent neural networks (SRNNs) to approximate the true conditional quantile function, assumed to exhibit a compositional structure referred to as the hierarchical interaction model. Specifically, we demonstrate that estimators based on RNNs and SRNNs with rectified linear unit (ReLU) activation function and properly chosen architectures achieve the optimal nonparametric convergence rate, up to a logarithmic factor, under stationary, exponentially $\boldsymbol{\beta}$-mixing observations. To establish this result, we derive approximation error bounds for functions within the hierarchical interaction model using RNNs and SRNNs. Our analysis leverages the close connections between sparse feedforward neural networks (SFNNs) and SRNNs. Numerical experiments and an empirical study on Gross Domestic Product (GDP) forecast further support the proposed theory.