Abstract
Feature extraction methods do generally not account for hierarchical structure in the data. For example, PCA and ICA provide transformations that solely depend on global properties of the overall dataset. We here present a general approach for the extraction of feature hierarchies from datasets and their use for classification or clustering. A hierarchy of features extracted from a dataset thereby constitutes a compact representation of the set that on the one hand can be used to characterize and understand the data and on the other hand serves as a basis to classify or cluster a collection of datasets. As a proof of concept, we demonstrate the feasibility of this approach with an application to mixtures of Gaussians with varying degree of structuredness and to a clinical EEG recording.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Jolliffe, I.: Principal Component Analysis. Springer, New York (1986)
Hyvarinen, A., Karhunen, J., Oja, E.: Independent Component Analysis. Wiley, Chichester (2001)
Fukushima, K.: Neural network model for a mechanism of pattern recognition unaffected by shift in position — neocognitron. Transactions IECE 62-A(10), 658–665 (1979)
Bregler, C., Omohundro, S.: Surface learning with applications to lipreading. In: Cowan, J., Tesauro, G., Alspector, J. (eds.) Advances in Neural Information Precessing Systems, vol. 6, pp. 43–50. Morgan Kaufmann Publishers, San Mateo (1994)
Bach, F., Jordan, M.: Beyond independent components: Trees and clusters. Journal of Machine Learning Research 4, 1205–1233 (2003)
Bishop, C., Tipping, M.: A hierarchical latent variable model for data visualization. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(3), 281–293 (1998)
Wang, Y., Luo, L., Freedman, M., Kung, S.: Probabilistic principal component subspaces: A hierarchical finite mixture model for data visualization. IEEE Transactions on Neural Networks 11(3), 625–636 (2000)
Tipping, M., Bishop, C.: Probabilistic principal component analysis. Journal of the Royal Statistical Society: Series B 61(3), 611–622 (1999)
Kondor, R., Jebara, T.: A kernel between sets of vectors. In: Fawcett, T., Mishra, N. (eds.) Proceedings of the ICML, pp. 361–368. AAAI Press, Menlo Park (2003)
Desobry, F., Davy, M., Fitzgerald, W.: A class of kernels for sets of vectors. In: Proceedings of the ESANN, pp. 461–466 (2005)
Jain, A., Dubes, R.: Algorithms for Clustering Data. Prentice Hall, Inc., Englewood Cliffs (1988)
Duda, R., Hart, P., Stork, D.: Pattern Classification. Wiley–Interscience, Chichester (2000)
Ward, J.: Hierarchical grouping to optimize an objective function. Journal of the American Statistical Association 58, 236–244 (1963)
Fred, A., Leitao, J.: Clustering under a hypothesis of smooth dissimilarity increments. In: Proceedings of the ICPR, vol. 2, pp. 190–194 (2000)
Dempster, A., Laird, N., Rubin, D.: Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society: Series B 39, 1–38 (1977)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Schubert, M., Kohlmorgen, J. (2008). Hierarchical Feature Extraction for Compact Representation and Classification of Datasets. In: Ishikawa, M., Doya, K., Miyamoto, H., Yamakawa, T. (eds) Neural Information Processing. ICONIP 2007. Lecture Notes in Computer Science, vol 4984. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-69158-7_58
Download citation
DOI: https://doi.org/10.1007/978-3-540-69158-7_58
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-69154-9
Online ISBN: 978-3-540-69158-7
eBook Packages: Computer ScienceComputer Science (R0)