Skip to main content

Reward-Modulated Adversarial Topic Modeling

  • Conference paper
  • First Online:
Database Systems for Advanced Applications (DASFAA 2020)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 12112))

Included in the following conference series:

  • 3016 Accesses

Abstract

Neural topic models have attracted much attention for their high efficiencies in training, in which, the methods based on variational auto-encoder capture approximative distributions of data, and those based on Generative Adversarial Net (GAN) are able to capture an accurate posterior distribution. However, the existing GAN-based neural topic model fails to model the document-topic distribution of input samples, making it difficult to get the representations of data in the latent topic space for downstream tasks. Moreover, to utilize the topics discovered by these topic models, it is time-consuming to manually interpret the meaning of topics, label the generated topics, and filter out interested topics. To address these limitations, we propose a Reward-Modulated Adversarial Topic Model (RMATM). By integrating a topic predictor and a reward function in GAN, our RMATM can capture the document-topic distribution and discover interested topics according to topic-related seed words. Furthermore, benefit from the reward function using topic-related seed words as weak supervision, RMATM is able to classify unlabeled documents. Extensive experiments on four benchmark corpora have well validated the effectiveness of RMATM.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://cs.nyu.edu/~roweis/data/.

  2. 2.

    http://archive.ics.uci.edu/ml/datasets/Bag+of+Words.

  3. 3.

    http://qwone.com/~jason/20Newsgroups/.

  4. 4.

    https://wiki.dbpedia.org/.

  5. 5.

    https://github.com/linkstrife/NVDM-GSM.

  6. 6.

    https://github.com/akashgit/autoencoding_vi_for_topic_models.

  7. 7.

    https://github.com/yumeng5/WeSHClass.

  8. 8.

    http://aksw.org/Projects/Palmetto.html.

  9. 9.

    https://www.usnews.com/topics/subjects.

  10. 10.

    https://en.wikipedia.org/wiki/Wikipedia:WikiProject_Lists_of_topics.

  11. 11.

    https://github.com/3Top/word2vec-api.

References

  1. Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent Dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003)

    MATH  Google Scholar 

  2. Fowlkes, E.B., Mallows, C.L.: A method for comparing two hierarchical clusterings. J. Am. Stat. Assoc. 78(383), 553–569 (1983)

    Article  Google Scholar 

  3. Goodfellow, I.J., et al.: Generative adversarial nets. In: NIPS, pp. 2672–2680 (2014)

    Google Scholar 

  4. Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., Courville, A.C.: Improved training of Wasserstein GANs. In: NIPS, pp. 5767–5777 (2017)

    Google Scholar 

  5. Hubert, L., Arabie, P.: Comparing partitions. J. Classif. 2(1), 193–218 (1985)

    Article  Google Scholar 

  6. Kingma, D.P., Welling, M.: Auto-encoding variational Bayes. In: ICLR (2014)

    Google Scholar 

  7. Meng, Y., Shen, J., Zhang, C., Han, J.: Weakly-supervised hierarchical text classification. In: AAAI, pp. 6826–6833 (2019)

    Google Scholar 

  8. Miao, Y., Grefenstette, E., Blunsom, P.: Discovering discrete latent topics with neural variational inference. In: ICML, pp. 2410–2419 (2017)

    Google Scholar 

  9. Miao, Y., Yu, L., Blunsom, P.: Neural variational inference for text processing. In: ICML, pp. 1727–1736 (2016)

    Google Scholar 

  10. Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. In: ICLR (2013)

    Google Scholar 

  11. Röder, M., Both, A., Hinneburg, A.: Exploring the space of topic coherence measures. In: WSDM, pp. 399–408 (2015)

    Google Scholar 

  12. Srivastava, A., Sutton, C.A.: Autoencoding variational inference for topic models. In: ICLR (2017)

    Google Scholar 

  13. Sutton, R.S., McAllester, D.A., Singh, S.P., Mansour, Y.: Policy gradient methods for reinforcement learning with function approximation. In: NIPS, pp. 1057–1063 (1999)

    Google Scholar 

  14. Vinh, N.X., Epps, J., Bailey, J.: Information theoretic measures for clusterings comparison: variants, properties, normalization and correction for chance. J. Mach. Learn. Res. 11, 2837–2854 (2010)

    MathSciNet  MATH  Google Scholar 

  15. Wang, R., Zhou, D., He, Y.: ATM: adversarial-neural topic model. Inf. Process. Manage. 56(6) (2019)

    Google Scholar 

Download references

Acknowledgment

The research described in this paper was supported by the National Natural Science Foundation of China (61972426).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Yanghui Rao .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Feng, Y., Feng, J., Rao, Y. (2020). Reward-Modulated Adversarial Topic Modeling. In: Nah, Y., Cui, B., Lee, SW., Yu, J.X., Moon, YS., Whang, S.E. (eds) Database Systems for Advanced Applications. DASFAA 2020. Lecture Notes in Computer Science(), vol 12112. Springer, Cham. https://doi.org/10.1007/978-3-030-59410-7_47

Download citation

Publish with us

Policies and ethics