research-article

Learning non-redundant codebooks for classifying complex objects

Authors:

Thomas DietterichAuthors Info & Claims

ICML '09: Proceedings of the 26th Annual International Conference on Machine Learning

Pages 1241 - 1248

https://doi.org/10.1145/1553374.1553533

Published: 14 June 2009 Publication History

Abstract

Codebook-based representations are widely employed in the classification of complex objects such as images and documents. Most previous codebook-based methods construct a single codebook via clustering that maps a bag of low-level features into a fixed-length histogram that describes the distribution of these features. This paper describes a simple yet effective framework for learning multiple non-redundant codebooks that produces surprisingly good results. In this framework, each codebook is learned in sequence to extract discriminative information that was not captured by preceding codebooks and their corresponding classifiers. We apply this framework to two application domains: visual object categorization and document classification. Experiments on large classification tasks show substantial improvements in performance compared to a single codebook or codebooks learned in a bagging style.

References

[1]

Baker, L. D. & McCallum, A. K. (1998). Distributional clustering of words for text classification. In Proc. SIGIR conf. Resear. and develo. infor. retriev., pp 96--103.

Digital Library

[2]

Bekkerman, R., El-yaniv, R., Tishby, N., Winter, Y., Guyon, I., & Elisseeff, A. (2003). Distributional word clusters vs. words for text categorization, J. of Machine Learning Research, Vol 3, pp 1183--1208.

Digital Library

[3]

Breiman, L. (1996). Bagging predictors. Machine Learning, 24 (2), pp 123--140.

[4]

Chechik, G. & Tishby, N. (2002). Extracting relevant structures with side information. Proc. Advances in Neural Information Processing Systems, pp 857--864.

[5]

Csurka, G., Dance, C. R., Fan L., Willamowski, J., & Bray, C. (2004). Visual categorization with bags of keypoints. Euro. Conf. Comput. Vision Workshop, pp 59--74.

[6]

Cui, Y., Fern, X. Z., & Dy, J. G. (2007). Non-redundant Multi-view Clustering via Orthogonalization. IEEE Int'l Conf. on Data Mining, pp 133--142.

Digital Library

[7]

Deng, H., Zhang W., Mortensen, E., Dietterich, T. & Shapiro, L. (2007). Principal curvature-based region detector for object recognition. Proc. IEEE Conf. Comput. Vision Pattern Recognition, pp 1--8.

[8]

Dhillon, I., Mallela, S. & Kumar, R. (2003). A divisive information-theoretic feature clustering algorithm for text classification. J. of Machine Learning Research, Vol 3, pp 1265--1287.

Digital Library

[9]

Dietterich, T. G., Lathrop, R. H., & Lozano-Perez, T. (1997). Solving the multiple-instance problem with axis-parallel rectangles. Artificial Intelligence, Vol 89, pp 31--71.

Digital Library

[10]

Dietterich, T. G. (1998). Approximate statistical tests for comparing supervised classification learning algorithms. Neural Computation, pp 1895--1923.

Digital Library

[11]

Dorko, G. & Schmid, C. (2005). Object class recognition using discriminative local features. Technical Report RR-5497, INRIA-Rhone-Alpes.

[12]

Freund, Y. & Schapire, R. (1996). Experiments with a new boosting algorithm. Proc. Int'l Conf. Machine Learning, pp 148--156.

[13]

Jain, P., Meka R., & Dhillon I. S. (2008). Simultaneous Unsupervised Learning of Disparate Clusterings. Statistical Analysis and Data Mining. Vol 1, pp 195--210.

Digital Library

[14]

Jurie, F. & Triggs, B. (2005). Creating efficient codebooks for visual recognition. Proc. IEEE Int'l Conf. Comput. Vision, Vol 1, pp 604--610.

Digital Library

[15]

Kadir, T., Zisserman A., & Brady, M. (2004). An affine invariant salient region detector. Proc. Euro. Conf. Comput. Vision, pp 228--241.

[16]

Kalal, Z., Matas, J., & Mikolajczyk K. (2008). Weighted sampling for large-scale boosting. Proc. Brit. Machine Vision Conf.

[17]

Larios, N. et al. (2008). Automated insect identification through concatenated histograms of local appearance features. Machine Vis. and App., 19(2), pp 105--123.

Digital Library

[18]

Lowe, D. (2004). Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision., 2(60), pp 91--110.

Digital Library

[19]

McCallum, A. K. (2002). MALLET: A machine learning for language toolkit. http://mallet.cs.umass.edu.

[20]

Mikolajczyk, K., & Schmid, C. (2004). Scale and affine invariant interest point detectors. Int. J. Comput. Vision., Vol 60, pp 63--86.

Digital Library

[21]

Mikolajczyk, K., Tuytelaars, T., Schmid, C., Zisserman, A., Matas, J., Schaffalitzky, F., Kadir, T., & Van Gool, L. (2005). A comparison of affine region detectors. Int. J. Comput. Vision., Vol 65, pp 43--72.

Digital Library

[22]

Moosmann, F., Triggs, B. & Jurie, F. (2007). Fast discriminative visual codebooks using randomized clustering forests. Proc. Advances in Neural Information Processing Systems, pp 985--992.

[23]

Opelt, A, Pinz A, Fussenegger. M, & Auer P. (2006). Generic Object Recognition with Boosting. IEEE Trans. Pattern Anal. Mach. Intell., 28(3), pp 416--431.

Digital Library

[24]

Perronnin, F., Dance, C., Csurka, G. & Bressan, M. (2006). Adapted vocabularies for generic visual categorization. Proc. Euro. Conf. Comput. Vision, pp 464--475.

Digital Library

[25]

Quinlan, J. R. (1993). C4.5: Programs for machine learning. Morgan Kaufmann Publishers Inc.

Digital Library

[26]

Salton, G. & Buckley, C. (1988). Term-weighting approaches in automatic text retrieval. Information Processing & Management, 24(5), pp 513--523.

Digital Library

[27]

Slonim, N. & Tishby, N. (2001). The power of word clusters for text classification. In Proc. Euro. Colloq. Information Retrieval Research.

[28]

Slonim, N., Friedman, N., & Tishby, N. (2002). Unsupervised document classification using sequential information maximization. Proc. SIGIR conf. Resear. and develo. infor. retriev., pp 129--136.

Digital Library

[29]

Viola, P. & Jones, M. (2001). Rapid object detection using a boosted cascade of simple features. Proc. IEEE Conf. Comput. Vision Pattern Recognition, pp 511--518.

[30]

Winn, J., Criminisi, A. & Minka, T. (2005). Object categorization by learned universal visual dictionary. Proc. IEEE Int'l Conf. Comput. Vision, Vol 2, pp 1800--1807.

Digital Library

[31]

Yang, L., Jin, R., Sukthankar R., & Jurie, F. (2008). Unifying discriminative visual codebook generation with classifier training for object category recognition. Proc. IEEE Conf. Comput. Vision Pattern Recognition, pp 1--8.

Cited By

Tribelhorn BDillon H(2022)Mapping the Chaotic Transitions of the Lorenz Equations With Unsupervised Machine LearningASME Journal of Heat and Mass Transfer10.1115/1.4055937145:1Online publication date: 17-Nov-2022
https://doi.org/10.1115/1.4055937
Wang LLi SWang SKong DYin B(2021)Hardness-Aware Dictionary Learning: Boosting Dictionary for RecognitionIEEE Transactions on Multimedia10.1109/TMM.2020.301791623(2857-2867)Online publication date: 2021
https://doi.org/10.1109/TMM.2020.3017916
Rong YXiong SGao Y(2020)Double Graph Regularized Double Dictionary Learning for Image ClassificationIEEE Transactions on Image Processing10.1109/TIP.2020.300424629(7707-7721)Online publication date: 2020
https://doi.org/10.1109/TIP.2020.3004246
Show More Cited By

Index Terms

Learning non-redundant codebooks for classifying complex objects
1. Computing methodologies
  1. Machine learning
  2. Modeling and simulation
    1. Model development and analysis
      1. Model verification and validation
      2. Modeling methodologies
2. Mathematics of computing
  1. Probability and statistics
    1. Statistical paradigms
      1. Exploratory data analysis

Recommendations

Organization of fixed rate vector quantization codebooks
A derailment-free finite-state vector quantizer with optimized state codebooks
DCC '95: Proceedings of the Conference on Data Compression

A new approach to the design of a finite-state vector quantizer (FSVQ) is proposed. FSVQ essentially exploits correlations between adjacent blocks for efficient coding. Previous FSVQ design schemes had ad-hoc features in defining states and resource ...
Design of Asymmetric VQ Codebooks Incorporating Channel Coding

In this paper, a communication system using vector quantization (VQ) and channel coding is considered. Here, a design scheme has been proposed to optimize source codebooks in the transmitter and the receiver. In the proposed algorithm, the overall ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICML '09: Proceedings of the 26th Annual International Conference on Machine Learning

June 2009

1331 pages

ISBN:9781605585161

DOI:10.1145/1553374

General Chair:
Andrea Danyluk
Williams College
,
Program Chairs:
Léon Bottou
NEC Laboratories America
,
Michael Littman
Rutgers University

Copyright © 2009 Copyright 2009 by the author(s)/owner(s).

Sponsors

NSF
Microsoft Research: Microsoft Research
MITACS

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 June 2009

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Funding Sources

Division of Information and Intelligent Systems

Conference

ICML '09

Sponsor:

Microsoft Research

ICML '09: The 26th Annual International Conference on Machine Learning held in conjunction with the 2007 International Conference on Inductive Logic Programming

June 14 - 18, 2009

Quebec, Montreal, Canada

Acceptance Rates

Overall Acceptance Rate 140 of 548 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

53
Total Citations
View Citations
395
Total Downloads

Downloads (Last 12 months)20
Downloads (Last 6 weeks)1

Reflects downloads up to 25 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Tribelhorn BDillon H(2022)Mapping the Chaotic Transitions of the Lorenz Equations With Unsupervised Machine LearningASME Journal of Heat and Mass Transfer10.1115/1.4055937145:1Online publication date: 17-Nov-2022
https://doi.org/10.1115/1.4055937
Wang LLi SWang SKong DYin B(2021)Hardness-Aware Dictionary Learning: Boosting Dictionary for RecognitionIEEE Transactions on Multimedia10.1109/TMM.2020.301791623(2857-2867)Online publication date: 2021
https://doi.org/10.1109/TMM.2020.3017916
Rong YXiong SGao Y(2020)Double Graph Regularized Double Dictionary Learning for Image ClassificationIEEE Transactions on Image Processing10.1109/TIP.2020.300424629(7707-7721)Online publication date: 2020
https://doi.org/10.1109/TIP.2020.3004246
Chebbout SMerouani H(2020)A hybrid codebook model for object categorization using two-way clustering based codebook generation methodInternational Journal of Computers and Applications10.1080/1206212X.2020.171277544:2(178-186)Online publication date: 12-Jan-2020
https://doi.org/10.1080/1206212X.2020.1712775
Passalis NTefas A(2018)Information Clustering Using Manifold-Based Optimization of the Bag-of-Features RepresentationIEEE Transactions on Cybernetics10.1109/TCYB.2016.262358148:1(52-63)Online publication date: Jan-2018
https://doi.org/10.1109/TCYB.2016.2623581
Shu XTang JQi GLi ZJiang YYan S(2018)Image Classification With Tailored Fine-Grained DictionariesIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2016.260734528:2(454-467)Online publication date: Feb-2018
https://doi.org/10.1109/TCSVT.2016.2607345
Fernando TSridharan SFookes CDenman S(2018)Deep Decision Trees for Discriminative Dictionary Learning with Adversarial Multi-agent Trajectories2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)10.1109/CVPRW.2018.00224(1803-180309)Online publication date: Jun-2018
https://doi.org/10.1109/CVPRW.2018.00224
Tan XFan YGuo R(2018)Local features and manifold ranking coupled method for sketch-based 3D model retrievalFrontiers of Computer Science: Selected Publications from Chinese Universities10.1007/s11704-017-6595-612:5(1000-1012)Online publication date: 1-Oct-2018
https://dl.acm.org/doi/10.1007/s11704-017-6595-6
Passalis NTsantekidis ATefas AKanniainen JGabbouj MIosifidis A(2017)Time-series classification using neural Bag-of-Features2017 25th European Signal Processing Conference (EUSIPCO)10.23919/EUSIPCO.2017.8081217(301-305)Online publication date: Aug-2017
https://doi.org/10.23919/EUSIPCO.2017.8081217
Bhattacharjee PBanerjee SGulati MMajumdar ARam S(2017)Supervised Analysis Dictionary LearningProceedings of the 4th ACM IKDD Conferences on Data Sciences10.1145/3041823.3041825(1-10)Online publication date: 9-Mar-2017
https://dl.acm.org/doi/10.1145/3041823.3041825
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten