research-article

Pinball Loss Twin Support Vector Clustering

Authors: M. Tanveer, Tarun Gupta, Miten Shah, For the Alzheimer’s Disease Neuroimaging Initiative Authors Info & Claims

ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), Volume 17, Issue 2s

Article No.: 63, Pages 1 - 23

https://doi.org/10.1145/3409264

Published: 21 June 2021 Publication History

Abstract

Twin Support Vector Clustering (TWSVC) is a clustering algorithm inspired by the principles of Twin Support Vector Machine (TWSVM). TWSVC has already outperformed other traditional plane based clustering algorithms. However, TWSVC uses hinge loss, which maximizes shortest distance between clusters and hence suffers from noise-sensitivity and low re-sampling stability. In this article, we propose Pinball loss Twin Support Vector Clustering (pinTSVC) as a clustering algorithm. The proposed pinTSVC model incorporates the pinball loss function in the plane clustering formulation. Pinball loss function introduces favorable properties such as noise-insensitivity and re-sampling stability. The time complexity of the proposed pinTSVC remains equivalent to that of TWSVC. Extensive numerical experiments on noise-corrupted benchmark UCI and artificial datasets have been provided. Results of the proposed pinTSVC model are compared with TWSVC, Twin Bounded Support Vector Clustering (TBSVC) and Fuzzy c-means clustering (FCM). Detailed and exhaustive comparisons demonstrate the better performance and generalization of the proposed pinTSVC for noise-corrupted datasets. Further experiments and analysis on the performance of the above-mentioned clustering algorithms on structural MRI (sMRI) images taken from the ADNI database, face clustering, and facial expression clustering have been done to demonstrate the effectiveness and feasibility of the proposed pinTSVC model.

References

[1]

Alzheimer’s Association2018. 2018 Alzheimer’s disease facts and figures. Alzh. Dement. 14, 3 (2018), 367–429.

[2]

Lan Bai, Yuan-Hai Shao, Wei-Jie Chen, Zhen Wang, and Nai-Yang Deng. 2020. Multiple flat projections for cross-manifold clustering. arXiv preprint arXiv:2002.06739 (2020).

[3]

Lan Bai, Yuan-Hai Shao, Zhen Wang, and Chun-Na Li. 2019. Clustering by twin support vector machine and least square twin support vector classifier with uniform output coding. Knowl.-based Syst. 163 (2019), 227–240.

[4]

James C. Bezdek, Robert Ehrlich, and William Full. 1984. FCM: The fuzzy c-means clustering algorithm. Comput. Geosci. 10, 2--3 (1984), 191–203.

[5]

Catherine L. Blake and C. J. Merz. 1998. UCI repository of machine learning databases. http://www.ics.uci.edu/∼mlearn/MLRepository.html.

[6]

Paul S. Bradley and Olvi L. Mangasarian. 2000. K-plane clustering. J. Glob. Optim. 16, 1 (2000), 23–32.

Digital Library

[7]

Alistair Burns and Steve Iliffe. 2009. Alzheimer’s disease. BMJ 338 (2009), b158.

[8]

Vladimir Cherkassky and Yunqian Ma. 2004. Practical selection of SVM parameters and noise estimation for SVM regression. Neural Netw. 17, 1 (2004), 113–126.

Digital Library

[9]

Andreas Christmann and Ingo Steinwart. 2008. How SVMs can estimate quantiles and the median. In Proceedings of the International Conference on Advances in Neural Information Processing Systems. 305–312.

Digital Library

[10]

Corinna Cortes and Vladimir Vapnik. 1995. Support-vector networks. Mach. Learn. 20, 3 (01 Sep. 1995), 273–297.

Digital Library

[11]

Christos Davatzikos, Susan M. Resnick, X. Wu, P. Parmpi, and Christopher M. Clark. 2008. Individual patient diagnosis of AD and FTD via high-dimensional pattern classification of MRI. Neuroimage 41, 4 (2008), 1220–1227.

[12]

Janez Demšar. 2006. Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7, Jan. (2006), 1–30.

Digital Library

[13]

Roger Fletcher. 2013. Practical Methods of Optimization. John Wiley & Sons.

[14]

Giovanni B. Frisoni, Nick C. Fox, Clifford R. Jack, Philip Scheltens, and Paul M. Thompson. 2010. The clinical use of structural MRI in Alzheimer disease. Nat. Rev. Neurol. 6, 2 (2010), 67–77.

[15]

Gao Xinbo, Li Jie, and Xie Weixin. 2000. Parameter optimization in FCM clustering algorithms. In Proceedings of the 5th International Conference on Signal Processing Proceedings and 16th World Computer Congress 2000, Vol. 3. 1457–1461.

[16]

Navin Goel, George Bebis, and Ara Nefian. 2005. Face recognition experiments with random projection. In Biometric Technology for Human Identification II, Anil K. Jain and Nalini K. Ratha (Eds.), Vol. 5779. International Society for Optics and Photonics, SPIE, 426–437.

[17]

Guodong Guo, Stan Li, and Kapluk Chan. 1970. Face recognition by support vector machines. In Proceedings of the 4th IEEE International Conference on Automatic Face and Gesture Recognition.

Digital Library

[18]

Sariel Har-Peled and Soham Mazumdar. 2004. On coresets for k-means and k-median clustering. In Proceedings of the 36th ACM Symposium on Theory of Computing. 291–300.

Digital Library

[19]

John A. Hartigan and Manchek A. Wong. 1979. Algorithm AS 136: A k-means clustering algorithm. J. Roy. Statist. Soc. Series C (Appl. Statist.) 28, 1 (1979), 100–108.

[20]

Mohamad H. Hassoun. 1995. Fundamentals of Artificial Neural Networks. The MIT Press.

Digital Library

[21]

Xiaolin Huang, Lei Shi, and Johan A. K. Suykens. 2013. Support vector machine classifier with pinball loss. IEEE Trans. Pattern Anal. Mach. Intell. 36, 5 (2013), 984–997.

[22]

Anil K. Jain, M. Narasimha Murty, and Patrick J. Flynn. 1999. Data clustering: A review. ACM Comput. Surv. 31, 3 (1999), 264–323.

Digital Library

[23]

Jayadeva, R. Khemchandani, and Suresh Chandra. 2007. Twin support vector machines for pattern classification. IEEE Trans. Pattern Anal. Mach. Intell. 29, 5 (2007), 905–910.

Digital Library

[24]

Miyuki Kamachi, Michael Lyons, and Jiro Gyoba. 1997. The Japanese female facial expression (JAFFE) database. Retrieved from http://www.kasrl.org/jaffe.html.

[25]

R. U. Khan, M. Tanveer, R. B. Pachori, and Alzheimer’s Disease Neuroimaging Initiative (ADNI). 2021. A novel method for the classification of Alzheimer’s disease from normal controls using magnetic resonance imaging. Exp. Syst. 38, 1 (2021), e12566.

[26]

Sang-Ki Kim, Youn Jung Park, Kar-Ann Toh, and Sangyoun Lee. 2010. SVM-based feature extraction for face recognition. Pattern Recog. 43, 8 (2010), 2871–2881.

Digital Library

[27]

M. Arun Kumar and Madan Gopal. 2009. Least squares twin support vector machines for pattern classification. Exp. Syst. Applic. 36, 4 (2009), 7535–7543.

Digital Library

[28]

Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. Nature 521, 7553 (2015), 436–444.

[29]

Vikramjit Mitra, Chia-Jiu Wang, and Satarupa Banerjee. 2007. Text classification: A least square support vector machine approach. Appl. Soft Comput. 7, 3 (2007), 908–914.

Digital Library

[30]

Md Mahmudur Rahman, Sameer K. Antani, and George R. Thoma. 2011. A learning-based similarity fusion and filtering approach for biomedical image retrieval using SVM classification and relevance feedback. IEEE Trans. Inf. Technol. Biomed. 15, 4 (2011), 640–646.

Digital Library

[31]

B. Richhariya and M. Tanveer. 2018. EEG Signal Classification Using Universum Support Vector Machine. Exp. Syst. Applic. 106 (2018), 169–182.

[32]

B. Richhariya and M. Tanveer. 2020. Least squares projection twin support vector clustering (LSPTSVC). Inf. Sci. 533 (2020), 1–23.

[33]

B. Richhariya and M. Tanveer. 2020. A reduced universum twin support vector machine for class imbalance learning. Pattern Recog. 102 (2020), 107150.

Digital Library

[34]

B. Richhariya, M. Tanveer, A. H. Rashid, and Alzheimer’s Disease Neuroimaging Initiative. 2020. Diagnosis of Alzheimer’s disease using universum support vector machine based recursive feature elimination (USVM-RFE). Biomed. Sig. Proc. Contr. 59 (2020), 101903.

[35]

Craig Saunders, Alexander Gammerman, and Volodya Vovk. 1998. Ridge regression learning algorithm in dual variables. In Proceedings of the Fifteenth International Conference on Machine Learning (ICML'98). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 515--521.

Digital Library

[36]

Bernhard Schölkopf, Alexander J. Smola. 2002. Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. The MIT Press.

Digital Library

[37]

Yuan-Hai Shao, Lan Bai, Zhen Wang, Xiang-Yu Hua, and Nai-Yang Deng. 2013. Proximal plane clustering via eigenvalues. In Proceedings of the International Conference on Information Technology and Quantitative Management (ITQM’13). 41–47.

[38]

Yuan-Hai Shao, Chun-Hua Zhang, Xiao-Bo Wang, and Nai-Yang Deng. 2011. Improvements on twin support vector machines. IEEE Trans. Neural Netw. 22, 6 (2011), 962–968.

Digital Library

[39]

Sweta Sharma, Reshma Rastogi, and Suresh Chandra. 2021. Large-scale twin parametric support vector machine using pinball loss function. IEEE Trans. Syst., Man, Cybern.: Syst. 51, 2 (2021), 987--1003.

[40]

Aixin Sun, Ee-Peng Lim, and Wee-Keong Ng. 2002. Web classification using support vector machine. In Proceedings of the 4th International Workshop on Web Information and Data Management (WIDM’02). Association for Computing Machinery, New York, NY, 96–99.

Digital Library

[41]

M. Tanveer. 2015. Application of smoothing techniques for linear programming twin support vector machines. Knowl. Inf. Syst. 45, 1 (01 Oct. 2015), 191–214.

Digital Library

[42]

M. Tanveer. 2015. Robust and sparse linear programming twin support vector machines. Cog. Comput. 7, 1 (2015), 137–149.

[43]

M. Tanveer, C. Gautam, and P. N. Suganthan. 2019. Comprehensive evaluation of twin SVM based classifiers on UCI datasets. Appl. Soft Comput. 83 (2019), 105617.

Digital Library

[44]

M. Tanveer, Mohammad Asif Khan, and Shen-Shyang Ho. 2016. Robust energy-based least squares twin support vector machines. Appl. Intell. 45, 1 (2016), 174–186.

Digital Library

[45]

M. Tanveer, T. Rajani, and M. A. Ganaie. 2019. Improved sparse pinball twin SVM. In Proceedings of the IEEE International Conference on Systems, Man and Cybernetics (SMC’19). 3287–3291.

[46]

M. Tanveer, B. Richhariya, R. U. Khan, A. H. Rashid, P. Khanna, M. Prasad, and C. T. Lin. 2020. Machine learning techniques for the diagnosis of Alzheimer’s disease: A review. ACM Trans. Multim. Comput., Commun. Applic. 16, 1s (2020), 1–35.

Digital Library

[47]

M. Tanveer, A. Sharma, and P. N. Suganthan. 2019. General twin support vector machine with pinball loss function. Inf. Sci. 494 (2019), 311–327.

Digital Library

[48]

M. Tanveer, Aruna Tiwari, Rahul Choudhary, and Sanchit Jalan. 2019. Sparse pinball twin support vector machines. Appl. Soft Comput. 78 (2019), 164–175.

Digital Library

[49]

Zhen Wang, Xu Chen, Yuan-Hai Shao, and Chun-Na Li. 2020. Ramp-based twin support vector clustering. Neural Comput. Applic. (2020), 9885--9896. https://doi.org/10.1007/s00521-019-04511-3

[50]

Zhen Wang, Yuan-Hai Shao, Lan Bai, and Nai-Yang Deng. 2015. Twin support vector machine for clustering. IEEE Trans. Neural Netw. Learn. Syst. 26, 10 (2015), 2583–2588.

[51]

Zhen Wang, Yuan-Hai Shao, Lan Bai, Chun-Na Li, and Li-Ming Liu. 2019. A general model for plane-based clustering with loss function. arXiv preprint arXiv:1901.09178 (2019).

[52]

Eric Westman, J.-Sebastian Muehlboeck, and Andrew Simmons. 2012. Combining MRI and CSF measures for classification of Alzheimer’s disease and prediction of mild cognitive impairment conversion. NeuroImage 62, 1 (2012), 229–238.

[53]

Svante Wold, Kim Esbensen, and Paul Geladi. 1987. Principal component analysis. Chemomet. Intell. Labor. Syst. 2, 1--3 (1987), 37–52.

[54]

Yichao Wu and Yufeng Liu. 2007. Robust truncated hinge loss support vector machines. J. Amer. Statist. Assoc. 102, 479 (2007), 974–983.

[55]

Hwanjo Yu and Sungchul Kim. 2012. SVM tutorial-classification, regression and ranking. Handb. Nat. Comput. 1 (2012), 479–506.

[56]

Alan L. Yuille and Anand Rangarajan. 2002. The concave-convex procedure (CCCP). In Proceedings of the International Conference on Advances in Neural Information Processing Systems. 1033–1040.

Digital Library

Cited By

Feng QChen CLiu L(2024)A Review of Convex Clustering From Multiple Perspectives: Models, Optimizations, Statistical Properties, Applications, and ConnectionsIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2023.327639335:10(13122-13142)Online publication date: Oct-2024
https://doi.org/10.1109/TNNLS.2023.3276393
Huang CLuo SYang GWang SCai JZhou L(2024)Weighted Intuitionistic Fuzzy Twin Support Vector Machines With Truncated Pinball LossIEEE Access10.1109/ACCESS.2024.346296412(136041-136053)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3462964
Wang HYu GMa J(2024)Fast sparse twin learning framework for large-scale pattern classificationEngineering Applications of Artificial Intelligence10.1016/j.engappai.2023.107730130(107730)Online publication date: Apr-2024
https://doi.org/10.1016/j.engappai.2023.107730
Show More Cited By

Index Terms

Pinball Loss Twin Support Vector Clustering
1. Computer systems organization
  1. Dependable and fault-tolerant systems and networks
    1. Redundancy
  2. Embedded and cyber-physical systems
    1. Embedded systems
    2. Robotics
2. Networks
  1. Network properties
    1. Network reliability

Recommendations

General twin support vector machine with pinball loss function
Highlights
- A novel general twin support vector machine with pinball loss is proposed.
- The ...
Abstract
The standard twin support vector machine (TSVM) uses the hinge loss function which leads to noise sensitivity and instability. In this paper, we propose a novel general twin support vector machine with pinball loss (Pin-GTSVM) for ...
Sparse pinball twin support vector machines
Abstract
The original twin support vector machine (TWSVM) formulation works by solving two smaller quadratic programming problems (QPPs) as compared to the traditional hinge-loss SVM (C-SVM) which solves a single large QPP — this makes the ...
Highlights
- A novel twin support vector machine with sparse pinball loss (SPTWSVM) is proposed.
Multi-view universum support vector machines with insensitive pinball loss
Abstract
Multi-view learning (MVL) is a promising field that seeks to make the most of information shared across different views, ultimately improving generalization performance. Numerous multi-view support vector machines (SVMs) have been proposed and ...
Highlights
- We propose two novel multi-view learning methods.
- They enhance generalization with the assistance of universum data.
- The insensitive pinball loss is introduced into multi-view learning.
- Experimental results validate the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Multimedia Computing, Communications, and Applications

ACM Transactions on Multimedia Computing, Communications, and Applications Volume 17, Issue 2s

June 2021

349 pages

ISSN:1551-6857

EISSN:1551-6865

DOI:10.1145/3465440

Editor:
Alberto Del Bimbo
University of Firenze, Italy

Issue’s Table of Contents

Copyright © 2021 Association for Computing Machinery.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 June 2021

Accepted: 01 June 2020

Revised: 01 May 2020

Received: 01 February 2020

Published in TOMM Volume 17, Issue 2s

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Refereed

Funding Sources

Science & Engineering Research Board (SERB) Government of INDIA under Ramanujan fellowship
Early Career Research Award (ECRA)
Council of Scientific & Industrial Research (CSIR), New Delhi, INDIA under Extra Mural Research (EMR) Scheme
Indian Institute of Technology Indore
Alzheimer’s Disease Neuroimaging Initiative (ADNI) (National Institutes of Health)
DOD ADNI (Department of Defense)
National Institute on Aging
National Institute of Biomedical Imaging and Bioengineering
AbbVie
Alzheimer’s Association
Alzheimer’s Drug Discovery Foundation
Araclon Biotech
BioClinica, Inc.
Bristol-Myers Squibb Company
CereSpir, Inc.
Cogstate
Eisai Inc.
Elan Pharmaceuticals, Inc.
Eli Lilly and Company
EuroImmun
F. Hoffmann-La Roche Ltd and its affiliated company Genentech, Inc.
Fujirebio
GE Healthcare
IXICO Ltd.
Janssen Alzheimer Immunotherapy Research & Development, LLC.
Johnson & Johnson Pharmaceutical Research & Development LLC.
Lumosity
Lundbeck
Merck & Co., Inc.
Meso Scale Diagnostics, LLC.
NeuroRx Research
Neurotrack Technologies
Novartis Pharmaceuticals Corporation
Pfizer Inc.
Piramal Imaging
Servier
Takeda Pharmaceutical Company
Transition Therapeutics
Canadian Institutes of Health Research
National Institutes of Health (www.fnih.org)

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

20
Total Citations
View Citations
217
Total Downloads

Downloads (Last 12 months)23
Downloads (Last 6 weeks)2

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Feng QChen CLiu L(2024)A Review of Convex Clustering From Multiple Perspectives: Models, Optimizations, Statistical Properties, Applications, and ConnectionsIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2023.327639335:10(13122-13142)Online publication date: Oct-2024
https://doi.org/10.1109/TNNLS.2023.3276393
Huang CLuo SYang GWang SCai JZhou L(2024)Weighted Intuitionistic Fuzzy Twin Support Vector Machines With Truncated Pinball LossIEEE Access10.1109/ACCESS.2024.346296412(136041-136053)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3462964
Wang HYu GMa J(2024)Fast sparse twin learning framework for large-scale pattern classificationEngineering Applications of Artificial Intelligence10.1016/j.engappai.2023.107730130(107730)Online publication date: Apr-2024
https://doi.org/10.1016/j.engappai.2023.107730
Zhu JChen SLiu YHu C(2024)Energy-based structural least squares twin support vector clusteringEngineering Applications of Artificial Intelligence10.1016/j.engappai.2023.107467128:COnline publication date: 14-Mar-2024
https://dl.acm.org/doi/10.1016/j.engappai.2023.107467
Liu YChen SZhu JHu C(2024)Plane-based clustering with asymmetric distribution lossApplied Soft Computing10.1016/j.asoc.2023.110893148:COnline publication date: 27-Feb-2024
https://dl.acm.org/doi/10.1016/j.asoc.2023.110893
Li DLi XGu FPan ZChen DMadden A(2023)A Universality–Distinction Mechanism-Based Multi-Step Sales Forecasting for Sales Prediction and Inventory OptimizationSystems10.3390/systems1106031111:6(311)Online publication date: 19-Jun-2023
https://doi.org/10.3390/systems11060311
Damminsed VPanup WWangkeeree R(2023)Laplacian Twin Support Vector Machine With Pinball Loss for Semi-Supervised ClassificationIEEE Access10.1109/ACCESS.2023.326227011(31399-31416)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3262270
Li DGu FLi XDu RChen DMadden A(2023)Dynamic sales prediction with auto-learning and elastic-adjustment mechanism for inventory optimizationInformation Systems10.1016/j.is.2023.102259119(102259)Online publication date: Oct-2023
https://doi.org/10.1016/j.is.2023.102259
Gu SChung FWang S(2023)Fuzzy style flat-based clusteringInformation Sciences10.1016/j.ins.2023.119321644(119321)Online publication date: Oct-2023
https://doi.org/10.1016/j.ins.2023.119321
Malik AGao RGanaie MTanveer MSuganthan P(2023)Random vector functional link networkApplied Soft Computing10.1016/j.asoc.2023.110377143:COnline publication date: 1-Aug-2023
https://dl.acm.org/doi/10.1016/j.asoc.2023.110377
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Issue’s Table of Contents