You will be shown an excerpt from a computer science scientific research paper, followed by a question about the paper. Please answer the question. Do not include any text in your response other than the answer.

Context: There is an alternative interpretation of the ELBO: the VAE objective can be viewed as a regularized version of the autoencoder (AE) Goodfellow et al. (2016).It is thus natural to extend the negative of \mathcal{L}_{\text{ELBO}} in (3) by introducing a hyper-parameter \beta to control the strength of regularization:\displaystyle\mathcal{L}_{\beta}\displaystyle=\mathcal{L}_{E}+\beta\mathcal{L}_{R},~{}~{}\text{with}(4)\displaystyle\mathcal{L}_{E}\displaystyle=-\mathbb{E}_{q_{\boldsymbol{\phi}}(\boldsymbol{z}|\boldsymbol{x})}\big{[}\log p_{\boldsymbol{\theta}}(\boldsymbol{x}|\boldsymbol{z})\big{]}(5)\displaystyle\mathcal{L}_{R}=KL(qϕ(𝒛|𝒙)||p(𝒛))\displaystyle=\mbox{KL}(q_{\boldsymbol{\phi}}(\boldsymbol{z}|\boldsymbol{x})||p(\boldsymbol{z}))= KL ( italic_q start_POSTSUBSCRIPT bold_italic_ϕ end_POSTSUBSCRIPT ( bold_italic_z | bold_italic_x ) | | italic_p ( bold_italic_z ) )(6)where \mathcal{L}_{E} is the reconstruction error (or negative log-likelihood (NLL)), and \mathcal{L}_{R} is a KL regularizer.The cost function \mathcal{L}_{\beta} provides a unified perspective for understanding various autoencoder variants and training methods. We consider two types of latent space with the following objectives: We train the model parameters \{\boldsymbol{\phi},\boldsymbol{\theta}\} using two objectives: AE and VAE, discussed in Section 4.1. Pre-training AE using  (5) is straightforward. However, pre-training VAE can be challenging due to the notorious KL vanishing issue Bowman et al. (2016), where(\textup{\it i})an encoder that produces posteriors almost identical to the Gaussian prior for all sentences (rather than a more interesting posterior); and(\textup{\it ii})a decoder that completely ignores \boldsymbol{z} in (2), and a learned model that reduces to a simpler NLM. Along the way to build the first big VAE language model, there are several technical contributions/implications that are novel: (\textup{\it i}) Latent vector injection: this work demonstrates two schemes to discuss how to effectively inject conditioning vectors into GPT-2 without re-training it.(\textup{\it ii}) The design idea to combine BERT/GPT-2 serves as a practical recipe to inspire people to integrate and reuse existing PLMs for larger and complex models.(\textup{\it iii}) Pre-training on massive datasets itself is an effective approach to reduce KL vanishing, as demonstrated by the state of-the-art performance on four VAE language modeling datasets. (\textup{\it iv}) The proof of VAE objective from the lens of IB, showing that VAE is a principled approach to balance the compactness and usability of learned representations.(\textup{\it v}) Improved performance on several language tasks shows the importance and necessity of pre-training a latent space. Language VAEs have inspired new applications in NLP, via exploiting many interestingproperties of the model’s latent space Bowman et al. (2016); Kim et al. (2018b).Its modeling capacity and empirical performance is somewhat limited, partially due to the KL vanishing issue described in Section 4.3. Several attempts have been made to alleviate this issue, including different KL annealing/thresholding schemes Bowman et al. (2016); Fu et al. (2019); Higgins et al. (2017); Li et al. (2019), decoder architectures Yang et al. (2017); Dieng et al. (2018), auxiliary loss Zhao et al. (2017), semi-amortized inference Kim et al. (2018a), aggressive encoder training schedule He et al. (2019), batch normalized inference Zhu et al. (2020) and flexible posterior Fang et al. (2019). Subramanian et al. (2018) have shown some promise that general encoder can benefit language generation. Transformers Vaswani et al. (2017) are recently considered in VAEs for classification Gururangan et al. (2019) and storytelling Wang and Wan (2019). Pre-training VAEs has been recently considered in conditional text generation to amortize the training of decoders and to allow easy adaptation in new generation tasks Duan et al. (2019).

Question: What is "KL vanishing" with respect to VAEs?
The authors do not explicitly explain the KL vanishing problem in detail, but they do cite a recent work, Bowman et al. (2016), that probably contains more detailed information on this problem. Additionally, the authors explain that KL regularization is a problem that specifically happens with Variational Autoencoders only (i.e. regular AEs do not seem to have this problem). When explaining how VAEs can be considered to be equivalent to AEs with KL regularization, we see the main difference between AEs and VAEs is the extra KL term added to VAEs. Putting all this information together, one could conclude that "KL vanishing" refers to this KL term in a VAE becoming zero during training. The authors list multiple methods (annealing, specialized decoder architectures, auxiliary loss functions, etc) that people in the field have used to prevent this from happening.