Article

Dirichlet aggregation: unsupervised learning towards an optimal metric for proportional data

Authors:

Hong QinAuthors Info & Claims

ICML '07: Proceedings of the 24th international conference on Machine learning

Pages 959 - 966

https://doi.org/10.1145/1273496.1273617

Published: 20 June 2007 Publication History

Abstract

Proportional data (normalized histograms) have been frequently occurring in various areas, and they could be mathematically abstracted as points residing in a geometric simplex. A proper distance metric on this simplex is of importance in many applications including classification and information retrieval. In this paper, we develop a novel framework to learn an optimal metric on the simplex. Major features of our approach include: 1) its flexibility to handle correlations among bins/dimensions; 2) widespread applicability without being limited to ad hoc backgrounds; and 3) a "real" global solution in contrast to existing traditional local approaches. The technical essence of our approach is to fit a parametric distribution to the observed empirical data in the simplex. The distribution is parameterized by affinities between simplex vertices, which is learned via maximizing likelihood of observed data. Then, these affinities induce a metric on the simplex, defined as the earth mover's distance equipped with ground distances derived from simplex vertex affinities.

References

[1]

Blei, D. M., Ng, A. Y., Jordan, M. I. (2003) Latent Dirichlet Allocation. Journal of Machine Learning Research, 3(2003), 993--1022.

Digital Library

[2]

Blei, D. M. and Lafferty, J. D. (2006) Correlated Topic Models. In NIPS, volume 18.

[3]

Bouguila, N. and Ziou, D. (2006) Unsupervised Selection of a Finite Dirichlet Mixture Model: An MML-Based Approach. IEEE Transactions on Knowledge and Data Engineering. Vol.18, No.8.

Digital Library

[4]

Buntine, W. and Jakulin, A. (2004) Applying Discrete PCA in Data Analysis In Proceedings of the 20th UAI.

Digital Library

[5]

Csurka, G., Dance, C. R., Fan, L., Willamowski, J., Bray, C. (2004) Visual Categorization with Bags of Keypoints. In the 8th ECCV, Workshop on Statistical Learning in Computer Vision.

[6]

Deerwester, S., Dumais, S., Landauer, T., Furnas, G., Harshman, R. (1990) Indexing by Latent Semantic Analysis. Journal of the American Society of Information Science, 41(6), 391--407.

[7]

Gehler, P. V., Holub, A. D., Welling, M. (2006) The Rate Adapting Possion Model for Information Retrieval and Object Recognition. In Proceedings of the 23rd ICML.

Digital Library

[8]

Hofmann, T. (2001) Unsupervised Learning by Probabilistic Latent Semantic Analysis. Machine Learning, 42, 177--196.

Digital Library

[9]

Lebanon, G. (2003) Learning Riemannian Metrics. In Proceedings of the 19th UAI.

[10]

Li Fei-Fei, Fergus, R., Perona, P. (2004) Learning Generative Visual Models from Few Training Examples: an Incremental Bayesian Approach Tested on 101 Object Categories. IEEE. CVPR, Workshop on Generative-Model Based Vision.

Digital Library

[11]

Lowe, D. G. (2004) Distinctive Image Features from Scale-Invariant Keypoints. International Journal of Computer Vision, 60(2), 91--110.

Digital Library

[12]

Marlin, B. and Zemel, R. (2004) The Multiple Multiplicative Factor Model for Collaborative Filtering. In Proceedings of the 21st ICML.

Digital Library

[13]

Minka, T. P. (2003) Estimating a Dirichlet Distribution. Technical Report, Microsoft Research.

[14]

Omer, I. and Werman, M. (2006) Image Specific Feature Similarities. In Proceedings of the 9th ECCV.

Digital Library

[15]

Puzicha, J., Hofmann, T., Buhmann, J. (1997) Non-parametric Similarity Measures for Unsupervised Texture Segmentation and Image Retrieval. In Proceedings of the IEEE. CVPR, pp.267--272.

Digital Library

[16]

Roweis, S. and Saul, L. (2000) Nonlinear Dimensionality Reduction by Locally Linear Embedding. Science. 290: 2323--2326

[17]

Rubner, Y., Tomasi, C., Guibas, L. J. (2000) The Earth Mover's Distance as a Metric for Image Retrieval. International Journal of Computer Vision, 40(2), 99--121.

Digital Library

[18]

Salton, G. and McGill, M. editors (1983). Introduction to Modern Information Retrieval. McGraw-Hill.

Digital Library

[19]

Schütze, H. (1993) Word Space. In NIPS, volume 5.

[20]

Sjolander, K., Karplus, K., Brown, M., Hughey, R., Krogh, A., Mian, I. S., Haussler, D. (1996) Dirichlet Mixtures: A Method for Improving Detection of Weak but Significant Protein Sequence Homology. CABIOS, 12(4): 327--345

[21]

Swain, M. J. and Ballard, D. H. (1991) Color Indexing. International Journal of Computer Vision, 7(1), 11--32.

Digital Library

[22]

Tenenbaum, J., Silva, V. d., Langford, J. (2000) A Global Geometric Framework for Nonlinear Dimensionality Reduction. Science. 290: 2319--2323

[23]

Welling, M., Rosen-Zvi, M., Hinton, G. (2004) Exponential Family Harmoniums with an Application to Information Retrieval. In NIPS, volume 16.

[24]

Wyszecki, G. and Stiles, W. S. (1982) Color Science: Concepts and Methods, Quantitative Data and Formulae. John Wiley and Sons: New York, NY.

[25]

Yamamoto, M. and Sadamitsu, K. (2005) Dirichlet Mixtures in Text Modeling. Technical report. CS-TR-05-1. University of Tsukuba.

Cited By

Bdiri TBouguila NZiou D(2016)Variational Bayesian inference for infinite generalized inverted Dirichlet mixtures with feature selection and its application to clusteringApplied Intelligence10.1007/s10489-015-0714-644:3(507-525)Online publication date: 1-Apr-2016
https://dl.acm.org/doi/10.1007/s10489-015-0714-6
Wahabzada MMahlein ABauckhage CSteiner UOerke EKersting K(2015)Metro Maps of Plant Disease Dynamics—Automated Mining of Differences Using Hyperspectral ImagesPLOS ONE10.1371/journal.pone.011690210:1(e0116902)Online publication date: 26-Jan-2015
https://doi.org/10.1371/journal.pone.0116902
Bouguila N(2012)Hybrid Generative/Discriminative Approaches for Proportional Data Modeling and ClassificationIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2011.16224:12(2184-2202)Online publication date: 1-Dec-2012
https://dl.acm.org/doi/10.1109/TKDE.2011.162
Show More Cited By

Dirichlet aggregation: unsupervised learning towards an optimal metric for proportional data
1. Computing methodologies

Recommendations

On the hardness of maximum rank aggregation problems

The rank aggregation problem consists in finding a consensus ranking on a set of alternatives, based on the preferences of individual voters. The alternatives are expressed by permutations, whose pairwise distance can be measured in many ways.In this ...
Aggregation of asymmetric distances in Computer Science

In this paper we provide a general description of how to combine a collection (not necessarily finite) of asymmetric distances in order to obtain a single one as output. To this end we introduce the notion of asymmetric distance aggregation function ...
On the existence of aggregation functions with given super-additive and sub-additive transformations

In this note we study restrictions on the recently introduced super-additive and sub-additive transformations, AA and AA, of an aggregation function A. We prove that if A has a slightly stronger property of being strictly directionally convex, then A=A ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICML '07: Proceedings of the 24th international conference on Machine learning

June 2007

1233 pages

ISBN:9781595937933

DOI:10.1145/1273496

Editor:
Zoubin Ghahramani
University of Cambridge, United Kingdom

Copyright © 2007 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Machine Learning Journal

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 June 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Conference

ICML '07 & ILP '07

Sponsor:

ICML '07 & ILP '07: The 24th Annual International Conference on Machine Learning held in conjunction with the 2007 International Conference on Inductive Logic Programming

June 20 - 24, 2007

Oregon, Corvalis, USA

Acceptance Rates

Overall Acceptance Rate 140 of 548 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
267
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Bdiri TBouguila NZiou D(2016)Variational Bayesian inference for infinite generalized inverted Dirichlet mixtures with feature selection and its application to clusteringApplied Intelligence10.1007/s10489-015-0714-644:3(507-525)Online publication date: 1-Apr-2016
https://dl.acm.org/doi/10.1007/s10489-015-0714-6
Wahabzada MMahlein ABauckhage CSteiner UOerke EKersting K(2015)Metro Maps of Plant Disease Dynamics—Automated Mining of Differences Using Hyperspectral ImagesPLOS ONE10.1371/journal.pone.011690210:1(e0116902)Online publication date: 26-Jan-2015
https://doi.org/10.1371/journal.pone.0116902
Bouguila N(2012)Hybrid Generative/Discriminative Approaches for Proportional Data Modeling and ClassificationIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2011.16224:12(2184-2202)Online publication date: 1-Dec-2012
https://dl.acm.org/doi/10.1109/TKDE.2011.162
Ablavsky VSclaroff S(2011)Learning parameterized histogram kernels on the simplex manifold for image and action classificationProceedings of the 2011 International Conference on Computer Vision10.1109/ICCV.2011.6126404(1473-1480)Online publication date: 6-Nov-2011
https://dl.acm.org/doi/10.1109/ICCV.2011.6126404
Yang LJin RMummert LSukthankar RGoode AZheng BHoi SSatyanarayanan M(2010)A Boosting Framework for Visuality-Preserving Distance Metric Learning and Its Application to Medical Image RetrievalIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2008.27332:1(30-44)Online publication date: 1-Jan-2010
https://dl.acm.org/doi/10.1109/TPAMI.2008.273
Bouguila NZiou D(2010)A Dirichlet process mixture of generalized Dirichlet distributions for proportional data modelingIEEE Transactions on Neural Networks10.1109/TNN.2009.203485121:1(107-122)Online publication date: 1-Jan-2010
https://dl.acm.org/doi/10.1109/TNN.2009.2034851
Boutemedjet SBouguila NZiou D(2009)A Hybrid Feature Extraction Selection Approach for High-Dimensional Non-Gaussian Data ClusteringIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2008.15531:8(1429-1443)Online publication date: 1-Aug-2009
https://dl.acm.org/doi/10.1109/TPAMI.2008.155
Wang HYang QZha HCohen WMcCallum ARoweis S(2008)Adaptive p-posterior mixture-model kernels for multiple instance learningProceedings of the 25th international conference on Machine learning10.1145/1390156.1390299(1136-1143)Online publication date: 5-Jul-2008
https://dl.acm.org/doi/10.1145/1390156.1390299
Wang HYang QQin HZha HCohen WMcCallum ARoweis S(2008)Dirichlet component analysisProceedings of the 25th international conference on Machine learning10.1145/1390156.1390298(1128-1135)Online publication date: 5-Jul-2008
https://dl.acm.org/doi/10.1145/1390156.1390298
Bouguila NZiou D(2008)A Dirichlet process mixture of dirichlet distributions for classification and prediction2008 IEEE Workshop on Machine Learning for Signal Processing10.1109/MLSP.2008.4685496(297-302)Online publication date: Oct-2008
https://doi.org/10.1109/MLSP.2008.4685496

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten