Topic-VQ-VAE: Leveraging Latent Codebooks for Flexible Topic-Guided
  Document Generation
This paper introduces a novel approach for topic modeling utilizing latent
codebooks from Vector-Quantized Variational Auto-Encoder~(VQ-VAE), discretely
encapsulating the rich information of the pre-trained embeddings such as the
pre-trained language model. From the novel interpretation of the latent
codebooks and embeddings as conceptual bag-of-words, we propose a new
generative topic model called Topic-VQ-VAE~(TVQ-VAE) which inversely generates
the original documents related to the respective latent codebook. The TVQ-VAE
can visualize the topics with various generative distributions including the
traditional BoW distribution and the autoregressive image generation. Our
experimental results on document analysis and image generation demonstrate that
TVQ-VAE effectively captures the topic context which reveals the underlying
structures of the dataset and supports flexible forms of document generation.
Official implementation of the proposed TVQ-VAE is available at
https://github.com/clovaai/TVQ-VAE.
65825446cd17364539db4647
YoungJoon Yoo
admin_assigned
2023-12-20T11:56:57.928Z
6582d1b85def15befa412a88
/avatars/5104073d752a9ed1a57678dd70753790.svg
YoungJoon Yoo
user
yjyoo3312
65825446cd17364539db4648
Jongwon Choi
Topic-VQ-VAE: Leveraging Latent Codebooks for Flexible Topic-Guided
  Document Generation
This paper introduces a novel approach for topic modeling utilizing latent
codebooks from Vector-Quantized Variational Auto-Encoder~(VQ-VAE), discretely
encapsulating the rich information of the pre-trained embeddings such as the
pre-trained language model. From the novel interpretation of the latent
codebooks and embeddings as conceptual bag-of-words, we propose a new
generative topic model called Topic-VQ-VAE~(TVQ-VAE) which inversely generates
the original documents related to the respective latent codebook. The TVQ-VAE
can visualize the topics with various generative distributions including the
traditional BoW distribution and the autoregressive image generation. Our
experimental results on document analysis and image generation demonstrate that
TVQ-VAE effectively captures the topic context which reveals the underlying
structures of the dataset and supports flexible forms of document generation.
Official implementation of the proposed TVQ-VAE is available at
https://github.com/clovaai/TVQ-VAE.
A novel generative topic model, TVQ-VAE, uses latent codebooks from VQ-VAE to capture topic contexts and generate documents with flexible distributions.
Vector-Quantized Variational Auto-Encoder VQ-VAE latent codebooks conceptual bag-of-words generative topic model Topic-VQ-VAE TVQ-VAE inverse generation BoW distribution autoregressive image generation