Article

Efficient kernel feature extraction for massive data sets

Authors:

Ivor W. Tsang,

Andras Kocsor,

James T. KwokAuthors Info & Claims

KDD '06: Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining

Pages 724 - 729

https://doi.org/10.1145/1150402.1150494

Published: 20 August 2006 Publication History

Get Access

Abstract

Maximum margin discriminant analysis (MMDA) was proposed that uses the margin idea for feature extraction. It often outperforms traditional methods like kernel principal component analysis (KPCA) and kernel Fisher discriminant analysis (KFD). However, as in other kernel methods, its time complexity is cubic in the number of training points m, and is thus computationally inefficient on massive data sets. In this paper, we propose an (1+ε)²-approximation algorithm for obtaining the MMDA features by extending the core vector machines. The resultant time complexity is only linear in m, while its space complexity is independent of m. Extensive comparisons with the original MMDA, KPCA, and KFD on a number of large data sets show that the proposed feature extractor can improve classification accuracy, and is also faster than these kernel-based methods by more than an order of magnitude.

References

[1]

M. Bǎdoiu and K. L. Clarkson. Optimal core-sets for balls. In DIMACS Workshop on Computational Geometry, 2002.]]

Google Scholar

[2]

T. Friess, N. Cristianini, and C. Campbell. The kernel-adatron: a fast and simple learning procedure for support vector machines. In Proceeding of the Fifteenth International Conference on Machine Learning, pages 188--196, 1998.]]

Digital Library

Google Scholar

[3]

W. Kienzle and B. Schölkopf. Training support vector machines with multiple equality constraints. In Proceedings of the European Conference on Machine Learning, 2005.]]

Digital Library

Google Scholar

[4]

H.-C. Kim, S. Pang, H.-M. Je, D. Kim, and S. Bang. Constructing support vector machine ensemble. Pattern Recognition, 36(12):2757--2767, 2003.]]

Crossref

Google Scholar

[5]

A. Kocsor, K. Kovács, and C. Szepesvári. Margin maximizing discriminant analysis. In Proceedings of the 15th European Conference on Machine Learning, pages 227--238, Pisa, Italy, Sept. 2004.]]

Digital Library

Google Scholar

[6]

O. Mangasarian and E. Wild. Multisurface proximal support vector machine classification via generalized eigenvalues. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(1):69--74, 2006.]]

Digital Library

Google Scholar

[7]

S. Mika, G. Rätsch, J. Weston, B. Schölkopf, and K.-R. Müller. Fisher discriminant analysis with kernels. In Y.-H. Hu, J. Larsen, E. Wilson, and S. Douglas, editors, Neural Networks for Signal Processing IX, pages 41--48, 1999.]]

Google Scholar

[8]

J. Platt. Fast training of support vector machines using sequential minimal optimization. In B. Schölkopf, C. Burges, and A. Smola, editors, Advances in Kernel Methods - Support Vector Learning, pages 185--208. MIT Press, Cambridge, MA, 1999.]]

Digital Library

Google Scholar

[9]

B. Schölkopf and A. Smola. Learning with Kernels. MIT Press, Cambridge, MA, 2002.]]

Google Scholar

[10]

I. W. Tsang, J. T. Kwok, and P.-M. Cheung. Core vector machines: Fast SVM training on very large data sets. Journal of Machine Learning Research, 6:363--392, 2005.]]

Digital Library

Google Scholar

[11]

I. W. Tsang, J. T. Kwok, and K. T. Lai. Core vector regression for very large regression problems. In Proceedings of the Twentieth-Second International Conference on Machine Learning, pages 913--920, Bonn, Germany, Aug. 2005.]]

Digital Library

Google Scholar

Cited By

View all

Ganaie MTanveer MJangir J(2022)EEG signal classification via pinball universum twin support vector machineAnnals of Operations Research10.1007/s10479-022-04922-x328:1(451-492)Online publication date: 19-Aug-2022
https://doi.org/10.1007/s10479-022-04922-x
Li DXu XWang ZCao CWang M(2022)Boundary‐based Fuzzy‐SVDD for one‐class classificationInternational Journal of Intelligent Systems10.1002/int.2277337:3(2266-2292)Online publication date: 25-Jan-2022
https://dl.acm.org/doi/10.1002/int.22773
Hosseini BHammer B(2019)Large-Margin Multiple Kernel Learning for Discriminative Features Selection and Representation Learning2019 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN.2019.8851982(1-8)Online publication date: Jul-2019
https://doi.org/10.1109/IJCNN.2019.8851982
Show More Cited By

Index Terms

Efficient kernel feature extraction for massive data sets
1. Computing methodologies
  1. Machine learning

Recommendations

A novel local sensitive frontier analysis for feature extraction
ICIC'09: Proceedings of the Intelligent computing 5th international conference on Emerging intelligent computing technology and applications

In this paper, an efficient feature extraction method, named local sensitive frontier analysis (LSFA), is proposed. LSFA tries to find instances near the crossing of the multi-manifold, which are sensitive to classification, to form the frontier ...
Constrained discriminant neighborhood embedding for high dimensional data feature extraction

When handling pattern classification problem such as face recognition and digital handwriting identification, image data is always represented to high dimensional vectors, from which discriminant features are extracted using dimensionality reduction ...
Efficient and robust feature extraction by maximum margin criterion

In pattern recognition, feature extraction techniques are widely employed to reduce the dimensionality of data and to enhance the discriminatory information. Principal component analysis (PCA) and linear discriminant analysis (LDA) are the two most ...

Comments

Information & Contributors

Information

Published In

KDD '06: Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining

August 2006

986 pages

ISBN:1595933395

DOI:10.1145/1150402

Conference Chair:
Tina Eliassi-Rad
LLNL
,
General Chair:
Lyle Ungar
University of Pennsylvania
,
Program Chairs:
Mark Craven
University of Wisconsin
,
Dimitrios Gunopulos
University of California, Riverside

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 August 2006

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

KDD06

Sponsor:

KDD06: The 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

August 20 - 23, 2006

PA, Philadelphia, USA

Acceptance Rates

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Upcoming Conference

KDD '25

Sponsor:
sigkdd
sigkdd

The 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 3 - 7, 2025

Toronto , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

29
Total Citations
View Citations
650
Total Downloads

Downloads (Last 12 months)5
Downloads (Last 6 weeks)1

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Ganaie MTanveer MJangir J(2022)EEG signal classification via pinball universum twin support vector machineAnnals of Operations Research10.1007/s10479-022-04922-x328:1(451-492)Online publication date: 19-Aug-2022
https://doi.org/10.1007/s10479-022-04922-x
Li DXu XWang ZCao CWang M(2022)Boundary‐based Fuzzy‐SVDD for one‐class classificationInternational Journal of Intelligent Systems10.1002/int.2277337:3(2266-2292)Online publication date: 25-Jan-2022
https://dl.acm.org/doi/10.1002/int.22773
Hosseini BHammer B(2019)Large-Margin Multiple Kernel Learning for Discriminative Features Selection and Representation Learning2019 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN.2019.8851982(1-8)Online publication date: Jul-2019
https://doi.org/10.1109/IJCNN.2019.8851982
Yang BChen YShao QYu RLi WGuo GJiang JPan L(2019)Schizophrenia Classification Using fMRI Data Based on a Multiple Feature Image Capsule Network EnsembleIEEE Access10.1109/ACCESS.2019.29335507(109956-109968)Online publication date: 2019
https://doi.org/10.1109/ACCESS.2019.2933550
Li ZWang LYang YDu XSong H(2019)Health Evaluation of MVB Based on SVDD and Sample ReductionIEEE Access10.1109/ACCESS.2019.2904600(1-1)Online publication date: 2019
https://doi.org/10.1109/ACCESS.2019.2904600
Richhariya BGupta D(2019)Facial expression recognition using iterative universum twin support vector machineApplied Soft Computing10.1016/j.asoc.2018.11.04676(53-67)Online publication date: Mar-2019
https://doi.org/10.1016/j.asoc.2018.11.046
Richhariya BSharma ATanveer M(2018)Improved universum twin support vector machine2018 IEEE Symposium Series on Computational Intelligence (SSCI)10.1109/SSCI.2018.8628671(2045-2052)Online publication date: Nov-2018
https://doi.org/10.1109/SSCI.2018.8628671
Li DWang ZCao CLiu Y(2018)Information entropy based sample reduction for support vector data descriptionApplied Soft Computing10.1016/j.asoc.2018.02.05371(1153-1160)Online publication date: Oct-2018
https://doi.org/10.1016/j.asoc.2018.02.053
Gupta DRichhariya BBorah P(2018)A fuzzy twin support vector machine based on information entropy for class imbalance learningNeural Computing and Applications10.1007/s00521-018-3551-9Online publication date: 24-May-2018
https://doi.org/10.1007/s00521-018-3551-9
Richhariya BTanveer M(2018)A Fuzzy Universum Support Vector Machine Based on Information EntropyMachine Intelligence and Signal Analysis10.1007/978-981-13-0923-6_49(569-582)Online publication date: 8-Aug-2018
https://doi.org/10.1007/978-981-13-0923-6_49
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Index Terms

Recommendations

A novel local sensitive frontier analysis for feature extraction

Constrained discriminant neighborhood embedding for high dimensional data feature extraction

Efficient and robust feature extraction by maximum margin criterion

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Upcoming Conference

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations