Skip to main content
SpringerLink
Log in

Ultra Fast Classification and Regression of High-Dimensional Problems Projected on 2D

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

We propose the two-dimensional visual map classifier and regressor, which project the high-dimensional patterns on a 2D map, for human visualization and understanding of the data, and afterwards define a classification or regression map that predicts, for each 2D pattern, the class label (in classification) or the output value (in regression). The 2D projection is performed using the linear discriminant analysis, due to its high performance, speed and ability to project unseen (out-of-sample) patterns. The map is defined in an efficient way by assigning the proper output value to each square (or pixel) in the 2D map. The experiments show that the maps defined by both methods: (1) allow to understand visually the data distribution of a classification or regression problem; (2) their performances are very near to the state-of-the-art support vector classification and regression, including wrappers; and (3) they are very fast, between 1 and 5 orders of magnitude faster than the other approaches, spending less than 1 min to classify datasets with 5 million patterns. Matlab code is available.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. persoal.citius.usc.es/manuel.fernandez.delgado/papers/uf2dcr.

  2. faculty.ucmerced.edu/mcarreira-perpinan/research/software/ ldsvm.tar.gz

References

  1. Alawadi S, Fernández-Delgado M, Mera D, Barro S (2017) Polynomial kernel discriminant analysis for 2D visualization of classification problems. Neural Comput Appl 8:1–17

    Google Scholar 

  2. Assi KC, Labelle H, Cheriet F (2014) Modified large margin nearest neighbor metric learning for regression. IEEE Signal Process Lett 21(3):292–296

    Article  Google Scholar 

  3. Barshan E, Ghodsi A, Azimifar Z, Jahromi M (2011) Supervised principal component analysis: visualization, classification and regression on subspaces and submanifolds. Pattern Recogn 44(7):1357–1371

    Article  MATH  Google Scholar 

  4. Carletta J (1996) Assessing agreement on classification tasks: the kappa statistic. Comput Linguist 22(2):249–254

    Google Scholar 

  5. Chang C, Lin C (2011) LIBSVM: a library for support vector machines. ACM Trans Intel Syst Technol 2:27:1-27:27

    Google Scholar 

  6. Dua D, Graff C (2017) UCI machine learning repository. http://archive.ics.uci.edu/ml

  7. Fan R, Chang K, Hsieh C, Wang X, Lin C (2008) LIBLINEAR: A library for large linear classification. J Mach Learn Res 9:1871–1874

    MATH  Google Scholar 

  8. Fernández-Delgado M, Cernadas E, Amorim D, Barro S (2014) Do we need hundreds of classifiers to solve real world classification problems? J Mach Learn Res 15:3133–3181

    MathSciNet  MATH  Google Scholar 

  9. Fernández-Delgado M, Sirsat M, Cernadas E, Alawadi S, Barro S, Febrero-Bande M (2016) An extensive experimental survey of regression methods. Neural Netw 55:215–230

    Google Scholar 

  10. Geng X, Xuan D, Zhou Z (2005) Supervised nonlinear dimensionality reduction for visualization and classification. IEEE Trans Syst Man Cybern Part B 35(6):1098–1107

    Article  Google Scholar 

  11. Ghassabeh Y, Rudzicz F, Moghaddam H (2015) Fast incremental LDA feature extraction. Pattern Recogn 48:1999–2012

    Article  Google Scholar 

  12. González R, Woods R, Eddins S (2004) Digital image processing using Matlab. Prentice-Hall, New Jersey

    Google Scholar 

  13. Guan S, Fang Q, Guan T (2021) Application of a novel PNN evaluation algorithm to a greenhouse monitoring system. IEEE Trans Instrum Meas 70:1–12

    Article  Google Scholar 

  14. Jalilvand A, Salim N (2017) Feature unionization: a novel approach for dimension reduction. Appl Soft Comput 52:1253–1261

    Article  Google Scholar 

  15. Li C, Shao Y, Yin W, Liu M (2020) Robust and sparse linear discriminant analysis via and alternative direction method of multipliers. IEEE Trans Neural Netw Learn Syst 31(3):915–926

    Article  Google Scholar 

  16. Maaten L (2007) An introduction to dimensionality reduction using Matlab. Tech. Rep. 2579-2605, Universiteit Maastricht. http://lvdmaaten.github.io

  17. Mai Q (2013) A review of discriminant analysis in high dimensions. WIREs Comput Stat 5(3):190–197

    Article  Google Scholar 

  18. Mendels O, Stern H, Berman S (2014) User identification for home entertainment based on free-air hand motion signatures. IEEE Trans Syst Man Cybern Part A 44(11):1461–1473

    Article  Google Scholar 

  19. Nataraj L, Karthikeyan S, Jacob G, Manjunath B (2011) Malware images: visualization and automatic classification. In: Proceedings of the international symposium on visualization for cyber security, pp. 4–10

  20. Pasolli E, Yang HL, Crawford MM (2016) Active-metric learning for classification of remotely sensed hyperspectral images. IEEE Trans Geosci Remote Sens 54(4):1925–1939

    Article  Google Scholar 

  21. Shui P, Shi X, Li C, Feng T, Xia X, Han Y (2022) GRNN-based predictors of UHF-band sea clutter reflectivity at low grazing angle. IEEE Geosci Remote Sens Lett 19:1–5

    Article  Google Scholar 

  22. Stathakis D, Perakis K (2007) Feature evolution for classification of remotely sensed data. IEEE Geosci Remote Sens Lett 4(3):354–358

    Article  Google Scholar 

  23. Tallón-Ballesteros A, Riquelme J, Ruiz R (2019) Semi-wrapper feature subset selector for feed-forward neural networks: applications to binary and multi-class classification problems. Neurocomputing 353:28–44

    Article  Google Scholar 

  24. Tang E, Suganthan P, Yao X, Qin A (2005) Linear dimensionality reduction using relevance weighted LDA. Pattern Recogn 38:485–493

    Article  MATH  Google Scholar 

  25. Tao C, Feng J (2017) Canonical kernel dimensionality reduction. Comput Stat Data Anal 107:131–148

    Article  MATH  Google Scholar 

  26. Turchetti C, Falaschetti L (2019) A manifold learning approach to dimensionality reduction for modeling data. Inf Sci 491:16–29

    Article  MathSciNet  MATH  Google Scholar 

  27. Wang A, An N, Chen G, Li L, Alterovitz G (2015) Accelerating wrapper-based feature selection with k-nearest-neighbor. Knowl-Based Syst 83:81–91

    Article  Google Scholar 

  28. Wang L, Mao Q (2019) Probabilistic dimensionality reduction via structure learning. IEEE Trans Pattern Anal Mach Intell 41(1):205–219

    Article  Google Scholar 

  29. Wang W, Carreira-Perpinan M (2004) The role of dimensionality reduction in classification. In: Proceedings of the AAAI conference on artificial inteligence, pp. 2128–2134 . arXiv:1405.6444

  30. Wei X, Shen H, Li Y, Tang X, Wang F, Kleinsteuber M, Murphey YL (2019) Reconstructible nonlinear dimensionality reduction via joint dictionary learning. IEEE Trans Neural Netw Learn Syst 30(1):175–189

    Article  MathSciNet  Google Scholar 

  31. Xiao G, Li J, Chen Y, Li K (2020) MalFCS: An effective malware classification framework with automated feature extraction based on deep convolutional neural networks. J Parallel Distrib Comput 141:49–58

    Article  Google Scholar 

  32. Yang B, Xiang M, Zhang Y (2016) Multi-manifold discriminant Isomap for visualization and classification. Pattern Recogn 55:215–230

    Article  MATH  Google Scholar 

  33. Zhang Z, Chow TWS, Zhao M (2013) Trace ratio optimization-based semi-supervised nonlinear dimensionality reduction for marginal manifold visualization. IEEE Trans Knowl Data Eng 25(5):1148–1161

    Article  Google Scholar 

Download references

Acknowledgements

This work has received financial support from the Consellería de Educación, Universidade e Formación Profesional, Xunta de Galicia (accreditation 2019-2022 ED431G-2019/04) and the European Regional Development Fund (ERDF), which acknowledges the CiTIUS - Centro Singular de Investigación en Tecnoloxías Intelixentes da Universidade de Santiago de Compostela as a Research Center of the Galician University System.

Author information

Authors and Affiliations

  1. Centro Singular de Investigación en Tecnoloxías Intelixentes da USC (CiTIUS), University of Santiago de Compostela, 15782, Santiago de Compostela, Spain

    Heba Alateyat, Manuel Fernández-Delgado, Eva Cernadas & Senén Barro

Authors
  1. Heba Alateyat
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Manuel Fernández-Delgado
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Eva Cernadas
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Senén Barro
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Manuel Fernández-Delgado.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file 1 (pdf 87 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Alateyat, H., Fernández-Delgado, M., Cernadas, E. et al. Ultra Fast Classification and Regression of High-Dimensional Problems Projected on 2D. Neural Process Lett 55, 5377–5400 (2023). https://doi.org/10.1007/s11063-022-11090-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-022-11090-3

Keywords

Search

Navigation