Article

Dynamic topic models

Authors:

John D. LaffertyAuthors Info & Claims

ICML '06: Proceedings of the 23rd international conference on Machine learning

Pages 113 - 120

https://doi.org/10.1145/1143844.1143859

Published: 25 June 2006 Publication History

Abstract

A family of probabilistic time series models is developed to analyze the time evolution of topics in large document collections. The approach is to use state space models on the natural parameters of the multinomial distributions that represent the topics. Variational approximations based on Kalman filters and nonparametric wavelet regression are developed to carry out approximate posterior inference over the latent topics. In addition to giving quantitative, predictive models of a sequential corpus, dynamic topic models provide a qualitative window into the contents of a large document collection. The models are demonstrated by analyzing the OCR'ed archives of the journal Science from 1880 through 2000.

References

[1]

Aitchison, J. (1982). The statistical analysis of compositional data. Journal of the Royal Statistical Society, Series B, 44(2):139--177.]]

[2]

Blei, D., Ng, A., and Jordan, M. (2003). Latent Dirichlet allocation. Journal of Machine Learning Research, 3:993--1022.]]

Digital Library

[3]

Blei, D. M. and Lafferty, J. D. (2006). Correlated topic models. In Weiss, Y., Schölkopf, B., and Platt, J., editors, Advances in Neural Information Processing Systems 18. MIT Press, Cambridge, MA.]]

[4]

Buntine, W. and Jakulin, A. (2004). Applying discrete PCA in data analysis. In Proceedings of the 20th Conference on Uncertainty in Artificial Intelligence, pages 59--66. AUAI Press.]]

Digital Library

[5]

Erosheva, E. (2002). Grade of membership and latent structure models with application to disability survey data. PhD thesis, Carnegie Mellon University, Department of Statistics.]]

[6]

Fei-Fei, L. and Perona, P. (2005). A Bayesian hierarchical model for learning natural scene categories. IEEE Computer Vision and Pattern Recognition.]]

Digital Library

[7]

Griffiths, T. and Steyvers, M. (2004). Finding scientific topics. Proceedings of the National Academy of Science, 101:5228--5235.]]

[8]

Kalman, R. (1960). A new approach to linear filtering and prediction problems. Transaction of the AMSE: Journal of Basic Engineering, 82:35--45.]]

[9]

McCallum, A., Corrada-Emmanuel, A., and Wang, X. (2004). The author-recipient-topic model for topic and role discovery in social networks: Experiments with Enron and academic email. Technical report, University of Massachusetts, Amherst.]]

[10]

Pritchard, J., Stephens, M., and Donnelly, P. (2000). Inference of population structure using multilocus genotype data. Genetics, 155:945--959.]]

[11]

Rosen-Zvi, M., Griffiths, T., Steyvers, M., and Smith, P. (2004). The author-topic model for authors and documents. In Proceedings of the 20th Conference on Uncertainty in Artificial Intelligence, pages 487--494. AUAI Press.]]

Digital Library

[12]

Sivic, J., Rusell, B., Efros, A., Zisserman, A., and Freeman, W. (2005). Discovering objects and their location in images. In International Conference on Computer Vision (ICCV 2005).]]

Digital Library

[13]

Snelson, E. and Ghahramani, Z. (2006). Sparse Gaussian processes using pseudo-inputs. In Weiss, Y., Schölkopf, B., and Platt, J., editors, Advances in Neural Information Processing Systems 18, Cambridge, MA. MIT Press.]]

[14]

Wasserman, L. (2006). All of Nonparametric Statistics. Springer.]]

Digital Library

[15]

West, M. and Harrison, J. (1997). Bayesian Forecasting and Dynamic Models. Springer.]]

Digital Library

Cited By

Mozaidze E(2025)Topic modeling in the stream of short messages in RussianRussian Technological Journal10.32362/2500-316X-2025-13-1-38-4813:1(38-48)Online publication date: 4-Feb-2025
https://doi.org/10.32362/2500-316X-2025-13-1-38-48
Burns JKelsey TDonovan C(2025)Extracting emerging events from social media: X/Twitter and the multilingual analysis of emerging geopolitical topics in near real timeJournal of Social Media Research10.29329/jsomer.142:1(50-70)Online publication date: 2-Mar-2025
https://doi.org/10.29329/jsomer.14
高翔(2025)Research on Topic Recognition and Evolution Based on Improved LDA Model—Taking the Field of Software Open Source as an ExampleHans Journal of Data Mining10.12677/hjdm.2025.15100515:01(55-70)Online publication date: 2025
https://doi.org/10.12677/hjdm.2025.151005
Show More Cited By

Index Terms

Dynamic topic models
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches
      1. Factorization methods
        Canonical correlation analysis
2. Mathematics of computing
  1. Probability and statistics
    1. Statistical paradigms
      1. Regression analysis
      2. Statistical graphics

Recommendations

Scaling up Dynamic Topic Models
WWW '16: Proceedings of the 25th International Conference on World Wide Web

Dynamic topic models (DTMs) are very effective in discovering topics and capturing their evolution trends in time series data. To do posterior inference of DTMs, existing methods are all batch algorithms that scan the full dataset before each update of ...
Topic Models with Topic Ordering Regularities for Topic Segmentation
ICDM '14: Proceedings of the 2014 IEEE International Conference on Data Mining

Documents from the same domain usually discuss similar topics in a similar order. In this paper we present new ordering-based topic models that use generalised Mallows models to capture this regularity to constrain topic assignments. Specifically, these ...
Probabilistic topic models
KDD '11 Tutorials: Proceedings of the 17th ACM SIGKDD International Conference Tutorials

Probabilistic topic modeling provides a suite of tools for the unsupervised analysis of large collections of documents. Topic modeling algorithms can uncover the underlying themes of a collection and decompose its documents according to those themes. ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICML '06: Proceedings of the 23rd international conference on Machine learning

June 2006

1154 pages

ISBN:1595933832

DOI:10.1145/1143844

Program Chairs:
William Cohen,
Andrew Moore

Copyright © 2006 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 June 2006

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Acceptance Rates

ICML '06 Paper Acceptance Rate 140 of 548 submissions, 26%;

Overall Acceptance Rate 140 of 548 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1,506
Total Citations
View Citations
10,463
Total Downloads

Downloads (Last 12 months)790
Downloads (Last 6 weeks)86

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Mozaidze E(2025)Topic modeling in the stream of short messages in RussianRussian Technological Journal10.32362/2500-316X-2025-13-1-38-4813:1(38-48)Online publication date: 4-Feb-2025
https://doi.org/10.32362/2500-316X-2025-13-1-38-48
Burns JKelsey TDonovan C(2025)Extracting emerging events from social media: X/Twitter and the multilingual analysis of emerging geopolitical topics in near real timeJournal of Social Media Research10.29329/jsomer.142:1(50-70)Online publication date: 2-Mar-2025
https://doi.org/10.29329/jsomer.14
高翔(2025)Research on Topic Recognition and Evolution Based on Improved LDA Model—Taking the Field of Software Open Source as an ExampleHans Journal of Data Mining10.12677/hjdm.2025.15100515:01(55-70)Online publication date: 2025
https://doi.org/10.12677/hjdm.2025.151005
Mervaala E(2025)Climate Change Versus Economic Growth: Quantifying, Identifying and Comparing Articulations in News Media Using Dynamic Topic ModelingEnvironmental Communication10.1080/17524032.2025.2458222(1-23)Online publication date: 30-Jan-2025
https://doi.org/10.1080/17524032.2025.2458222
Lee MSpark A(2025)Unsupervised generation of tradable Topic Indices through textual analysisThe Journal of Finance and Data Science10.1016/j.jfds.2025.100149(100149)Online publication date: Jan-2025
https://doi.org/10.1016/j.jfds.2025.100149
Cao JWang XChen GTu WShen XZhao TChen JLi Q(2025)Disentangling the hourly dynamics of mixed urban function: A multimodal fusion perspective using dynamic graphsInformation Fusion10.1016/j.inffus.2024.102832117(102832)Online publication date: May-2025
https://doi.org/10.1016/j.inffus.2024.102832
Song JLu XHong JWang F(2025)External information enhancing topic model based on graph neural networkExpert Systems with Applications10.1016/j.eswa.2024.125709263(125709)Online publication date: Mar-2025
https://doi.org/10.1016/j.eswa.2024.125709
Peng RDong YLi GTian DShan G(2025)(Chinavis 2024) TextLens: large language models-powered visual analytics enhancing text clusteringJournal of Visualization10.1007/s12650-025-01043-yOnline publication date: 10-Feb-2025
https://doi.org/10.1007/s12650-025-01043-y
Zhang DYu ZZeng ZZhang BLin RHu H(2025)A text-based framework for carbon price forecasting via multivariate temporal graph neural networkThe Journal of Supercomputing10.1007/s11227-025-06974-981:3Online publication date: 13-Feb-2025
https://doi.org/10.1007/s11227-025-06974-9
Li YTerui N(2025)Social media effects on multi-generational diffusion of information technology productsElectronic Commerce Research10.1007/s10660-025-09950-wOnline publication date: 17-Feb-2025
https://doi.org/10.1007/s10660-025-09950-w
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten