You will be shown an excerpt from a computer science scientific research paper, followed by a question about the paper. Please answer the question. Do not include any text in your response other than the answer.

Context: Recall from equation 6 that in an LSTM stackwith residual connections there are two accumulators: \mathbf{c}_{t}^{i}along the time axis and \mathbf{x}_{t}^{i} along the depth axis. Intheory, both of the accumulators are unbounded, but in practice, wenoticed their values remain quite small. For quantized inference, weexplicitly constrain the values of these accumulators to be within[-\delta, \delta] to guarantee a certain range that can be used forquantization later. The forward computation of an LSTM stack withresidual connections is modified to the following: \begin{split}\mathbf{v_{t}}&=\mathbf{W_{s}}*\mathbf{y_{t}}\\\mathbf{v_{t}^{\prime}}&=\max(-\gamma,\min(\gamma,\mathbf{v_{t}}))\\\mathbf{p_{t}}&=softmax(\mathbf{v_{t}^{\prime}})\end{split}(13)In equation 13, \mathbf{W_{s}} is the weightmatrix for the linear layer, which has the same number of rows as thenumber of symbols in the target vocabulary with each row correspondingto one unique target symbol. \mathbf{v} represents the raw logits, which arefirst clipped to be between -\gamma and \gamma and then normalizedinto a probability vector \mathbf{p}. Input \mathbf{y_{t}} isguaranteed to be between -\delta and \delta due to thequantization scheme we applied to the decoder RNN. The clipping range\gamma for the logits \mathbf{v} is determined empirically, and inour case, it is set to 25. In quantized inference, the weight matrix\mathbf{W_{s}} is quantized into 8 bits as inequation 12, and the matrix multiplication is done using8 bit arithmetic. The calculations within the softmax function and theattention model are not quantized during inference. It is worth emphasizing that during training of the model we use full-precisionfloating point numbers. The only constraints we add to the modelduring training are the clipping of the RNN accumulator values into[-\delta,\delta] and softmax logits into[-\gamma,\gamma]. \gamma is fixed to be at 25.0, while thevalue for \delta is gradually annealed from a generous bound of\delta=8.0 at the beginning of training, to a rather stringent boundof \delta=1.0 towards the end of training. At inference time,\delta is fixed at 1.0. Those additional constraints do not degrademodel convergence nor the decoding quality of the model when it hasconverged. In Figure 4, we compare the lossvs. steps for an unconstrained model (the blue curve) and a constrainedmodel (the red curve) on WMT’14 English-to-French. We can see thatthe loss for the constrained model is slightly better, possibly due toregularization roles those constraints play.

Question: Is the \delta a hyper-parameter?
Yes it's hyper-parameter, as it's within fixed range during training noting that it's fixed within this range during inference.