Skip Abstract Section
Abstract
Twin Support Vector Clustering (TWSVC) is a clustering algorithm inspired by the principles of Twin Support Vector Machine (TWSVM). TWSVC has already outperformed other traditional plane based clustering algorithms. However, TWSVC uses hinge loss, which maximizes shortest distance between clusters and hence suffers from noise-sensitivity and low re-sampling stability. In this article, we propose Pinball loss Twin Support Vector Clustering (pinTSVC) as a clustering algorithm. The proposed pinTSVC model incorporates the pinball loss function in the plane clustering formulation. Pinball loss function introduces favorable properties such as noise-insensitivity and re-sampling stability. The time complexity of the proposed pinTSVC remains equivalent to that of TWSVC. Extensive numerical experiments on noise-corrupted benchmark UCI and artificial datasets have been provided. Results of the proposed pinTSVC model are compared with TWSVC, Twin Bounded Support Vector Clustering (TBSVC) and Fuzzy c-means clustering (FCM). Detailed and exhaustive comparisons demonstrate the better performance and generalization of the proposed pinTSVC for noise-corrupted datasets. Further experiments and analysis on the performance of the above-mentioned clustering algorithms on structural MRI (sMRI) images taken from the ADNI database, face clustering, and facial expression clustering have been done to demonstrate the effectiveness and feasibility of the proposed pinTSVC model.
- Alzheimer’s Association2018. 2018 Alzheimer’s disease facts and figures. Alzh. Dement. 14, 3 (2018), 367–429.Google Scholar
Cross Ref
- Lan Bai, Yuan-Hai Shao, Wei-Jie Chen, Zhen Wang, and Nai-Yang Deng. 2020. Multiple flat projections for cross-manifold clustering. arXiv preprint arXiv:2002.06739 (2020).Google Scholar
- Lan Bai, Yuan-Hai Shao, Zhen Wang, and Chun-Na Li. 2019. Clustering by twin support vector machine and least square twin support vector classifier with uniform output coding. Knowl.-based Syst. 163 (2019), 227–240.Google Scholar
- James C. Bezdek, Robert Ehrlich, and William Full. 1984. FCM: The fuzzy c-means clustering algorithm. Comput. Geosci. 10, 2--3 (1984), 191–203.Google ScholarCross Ref
- Catherine L. Blake and C. J. Merz. 1998. UCI repository of machine learning databases. http://www.ics.uci.edu/∼mlearn/MLRepository.html.Google Scholar
- Paul S. Bradley and Olvi L. Mangasarian. 2000. K-plane clustering. J. Glob. Optim. 16, 1 (2000), 23–32. Google ScholarDigital Library
- Alistair Burns and Steve Iliffe. 2009. Alzheimer’s disease. BMJ 338 (2009), b158.Google ScholarCross Ref
- Vladimir Cherkassky and Yunqian Ma. 2004. Practical selection of SVM parameters and noise estimation for SVM regression. Neural Netw. 17, 1 (2004), 113–126. Google ScholarDigital Library
- Andreas Christmann and Ingo Steinwart. 2008. How SVMs can estimate quantiles and the median. In Proceedings of the International Conference on Advances in Neural Information Processing Systems. 305–312. Google ScholarDigital Library
- Corinna Cortes and Vladimir Vapnik. 1995. Support-vector networks. Mach. Learn. 20, 3 (01 Sep. 1995), 273–297. DOI:https://doi.org/10.1007/BF00994018 Google ScholarDigital Library
- Christos Davatzikos, Susan M. Resnick, X. Wu, P. Parmpi, and Christopher M. Clark. 2008. Individual patient diagnosis of AD and FTD via high-dimensional pattern classification of MRI. Neuroimage 41, 4 (2008), 1220–1227.Google ScholarCross Ref
- Janez Demšar. 2006. Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7, Jan. (2006), 1–30. Google ScholarDigital Library
- Roger Fletcher. 2013. Practical Methods of Optimization. John Wiley & Sons. Google ScholarDigital Library
- Giovanni B. Frisoni, Nick C. Fox, Clifford R. Jack, Philip Scheltens, and Paul M. Thompson. 2010. The clinical use of structural MRI in Alzheimer disease. Nat. Rev. Neurol. 6, 2 (2010), 67–77.Google ScholarCross Ref
- Gao Xinbo, Li Jie, and Xie Weixin. 2000. Parameter optimization in FCM clustering algorithms. In Proceedings of the 5th International Conference on Signal Processing Proceedings and 16th World Computer Congress 2000, Vol. 3. 1457–1461. DOI:https://doi.org/10.1109/ICOSP.2000.893376Google ScholarCross Ref
- Navin Goel, George Bebis, and Ara Nefian. 2005. Face recognition experiments with random projection. In Biometric Technology for Human Identification II, Anil K. Jain and Nalini K. Ratha (Eds.), Vol. 5779. International Society for Optics and Photonics, SPIE, 426–437. DOI:https://doi.org/10.1117/12.605553Google Scholar
- Guodong Guo, Stan Li, and Kapluk Chan. 1970. Face recognition by support vector machines. In Proceedings of the 4th IEEE International Conference on Automatic Face and Gesture Recognition. Google ScholarDigital Library
- Sariel Har-Peled and Soham Mazumdar. 2004. On coresets for k-means and k-median clustering. In Proceedings of the 36th ACM Symposium on Theory of Computing. 291–300. Google ScholarDigital Library
- John A. Hartigan and Manchek A. Wong. 1979. Algorithm AS 136: A k-means clustering algorithm. J. Roy. Statist. Soc. Series C (Appl. Statist.) 28, 1 (1979), 100–108.Google ScholarCross Ref
- Mohamad H. Hassoun. 1995. Fundamentals of Artificial Neural Networks. The MIT Press. Google ScholarDigital Library
- Xiaolin Huang, Lei Shi, and Johan A. K. Suykens. 2013. Support vector machine classifier with pinball loss. IEEE Trans. Pattern Anal. Mach. Intell. 36, 5 (2013), 984–997.Google ScholarCross Ref
- Anil K. Jain, M. Narasimha Murty, and Patrick J. Flynn. 1999. Data clustering: A review. ACM Comput. Surv. 31, 3 (1999), 264–323. Google ScholarDigital Library
- Jayadeva, R. Khemchandani, and Suresh Chandra. 2007. Twin support vector machines for pattern classification. IEEE Trans. Pattern Anal. Mach. Intell. 29, 5 (2007), 905–910. Google ScholarDigital Library
- Miyuki Kamachi, Michael Lyons, and Jiro Gyoba. 1997. The Japanese female facial expression (JAFFE) database. Retrieved from http://www.kasrl.org/jaffe.html.Google Scholar
- R. U. Khan, M. Tanveer, R. B. Pachori, and Alzheimer’s Disease Neuroimaging Initiative (ADNI). 2021. A novel method for the classification of Alzheimer’s disease from normal controls using magnetic resonance imaging. Exp. Syst. 38, 1 (2021), e12566.Google Scholar
- Sang-Ki Kim, Youn Jung Park, Kar-Ann Toh, and Sangyoun Lee. 2010. SVM-based feature extraction for face recognition. Pattern Recog. 43, 8 (2010), 2871–2881. Google ScholarDigital Library
- M. Arun Kumar and Madan Gopal. 2009. Least squares twin support vector machines for pattern classification. Exp. Syst. Applic. 36, 4 (2009), 7535–7543. Google ScholarDigital Library
- Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. Nature 521, 7553 (2015), 436–444.Google Scholar
- Vikramjit Mitra, Chia-Jiu Wang, and Satarupa Banerjee. 2007. Text classification: A least square support vector machine approach. Appl. Soft Comput. 7, 3 (2007), 908–914. Google ScholarDigital Library
- Md Mahmudur Rahman, Sameer K. Antani, and George R. Thoma. 2011. A learning-based similarity fusion and filtering approach for biomedical image retrieval using SVM classification and relevance feedback. IEEE Trans. Inf. Technol. Biomed. 15, 4 (2011), 640–646. Google ScholarDigital Library
- B. Richhariya and M. Tanveer. 2018. EEG Signal Classification Using Universum Support Vector Machine. Exp. Syst. Applic. 106 (2018), 169–182. DOI:https://doi.org/10.1016/j.eswa.2018.03.053Google ScholarCross Ref
- B. Richhariya and M. Tanveer. 2020. Least squares projection twin support vector clustering (LSPTSVC). Inf. Sci. 533 (2020), 1–23. DOI:https://doi.org/10.1016/j.ins.2020.05.001Google ScholarCross Ref
- B. Richhariya and M. Tanveer. 2020. A reduced universum twin support vector machine for class imbalance learning. Pattern Recog. 102 (2020), 107150. DOI:https://doi.org/10.1016/j.patcog.2019.107150Google ScholarDigital Library
- B. Richhariya, M. Tanveer, A. H. Rashid, and Alzheimer’s Disease Neuroimaging Initiative. 2020. Diagnosis of Alzheimer’s disease using universum support vector machine based recursive feature elimination (USVM-RFE). Biomed. Sig. Proc. Contr. 59 (2020), 101903.Google ScholarCross Ref
- Craig Saunders, Alexander Gammerman, and Volodya Vovk. 1998. Ridge regression learning algorithm in dual variables. In Proceedings of the Fifteenth International Conference on Machine Learning (ICML'98). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 515--521. Google ScholarDigital Library
- Bernhard Schölkopf, Alexander J. Smola. 2002. Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. The MIT Press. Google ScholarDigital Library
- Yuan-Hai Shao, Lan Bai, Zhen Wang, Xiang-Yu Hua, and Nai-Yang Deng. 2013. Proximal plane clustering via eigenvalues. In Proceedings of the International Conference on Information Technology and Quantitative Management (ITQM’13). 41–47.Google ScholarCross Ref
- Yuan-Hai Shao, Chun-Hua Zhang, Xiao-Bo Wang, and Nai-Yang Deng. 2011. Improvements on twin support vector machines. IEEE Trans. Neural Netw. 22, 6 (2011), 962–968. Google ScholarDigital Library
- Sweta Sharma, Reshma Rastogi, and Suresh Chandra. 2021. Large-scale twin parametric support vector machine using pinball loss function. IEEE Trans. Syst., Man, Cybern.: Syst. 51, 2 (2021), 987--1003. DOI:0.1109/TSMC.2019.2896642Google ScholarCross Ref
- Aixin Sun, Ee-Peng Lim, and Wee-Keong Ng. 2002. Web classification using support vector machine. In Proceedings of the 4th International Workshop on Web Information and Data Management (WIDM’02). Association for Computing Machinery, New York, NY, 96–99. DOI:https://doi.org/10.1145/584931.584952 Google ScholarDigital Library
- M. Tanveer. 2015. Application of smoothing techniques for linear programming twin support vector machines. Knowl. Inf. Syst. 45, 1 (01 Oct. 2015), 191–214. DOI:https://doi.org/10.1007/s10115-014-0786-3 Google ScholarDigital Library
- M. Tanveer. 2015. Robust and sparse linear programming twin support vector machines. Cog. Comput. 7, 1 (2015), 137–149.Google ScholarCross Ref
- M. Tanveer, C. Gautam, and P. N. Suganthan. 2019. Comprehensive evaluation of twin SVM based classifiers on UCI datasets. Appl. Soft Comput. 83 (2019), 105617. DOI:https://doi.org/10.1016/j.asoc.2019.105617Google ScholarDigital Library
- M. Tanveer, Mohammad Asif Khan, and Shen-Shyang Ho. 2016. Robust energy-based least squares twin support vector machines. Appl. Intell. 45, 1 (2016), 174–186. Google ScholarDigital Library
- M. Tanveer, T. Rajani, and M. A. Ganaie. 2019. Improved sparse pinball twin SVM. In Proceedings of the IEEE International Conference on Systems, Man and Cybernetics (SMC’19). 3287–3291.Google Scholar
- M. Tanveer, B. Richhariya, R. U. Khan, A. H. Rashid, P. Khanna, M. Prasad, and C. T. Lin. 2020. Machine learning techniques for the diagnosis of Alzheimer’s disease: A review. ACM Trans. Multim. Comput., Commun. Applic. 16, 1s (2020), 1–35. Google ScholarDigital Library
- M. Tanveer, A. Sharma, and P. N. Suganthan. 2019. General twin support vector machine with pinball loss function. Inf. Sci. 494 (2019), 311–327. DOI:https://doi.org/10.1016/j.ins.2019.04.032Google ScholarDigital Library
- M. Tanveer, Aruna Tiwari, Rahul Choudhary, and Sanchit Jalan. 2019. Sparse pinball twin support vector machines. Appl. Soft Comput. 78 (2019), 164–175. DOI:https://doi.org/10.1016/j.asoc.2019.02.022Google ScholarDigital Library
- Zhen Wang, Xu Chen, Yuan-Hai Shao, and Chun-Na Li. 2020. Ramp-based twin support vector clustering. Neural Comput. Applic. (2020), 9885--9896. https://doi.org/10.1007/s00521-019-04511-3Google Scholar
- Zhen Wang, Yuan-Hai Shao, Lan Bai, and Nai-Yang Deng. 2015. Twin support vector machine for clustering. IEEE Trans. Neural Netw. Learn. Syst. 26, 10 (2015), 2583–2588.Google ScholarCross Ref
- Zhen Wang, Yuan-Hai Shao, Lan Bai, Chun-Na Li, and Li-Ming Liu. 2019. A general model for plane-based clustering with loss function. arXiv preprint arXiv:1901.09178 (2019).Google Scholar
- Eric Westman, J.-Sebastian Muehlboeck, and Andrew Simmons. 2012. Combining MRI and CSF measures for classification of Alzheimer’s disease and prediction of mild cognitive impairment conversion. NeuroImage 62, 1 (2012), 229–238. DOI:https://doi.org/10.1016/j.neuroimage.2012.04.056Google ScholarCross Ref
- Svante Wold, Kim Esbensen, and Paul Geladi. 1987. Principal component analysis. Chemomet. Intell. Labor. Syst. 2, 1--3 (1987), 37–52.Google ScholarCross Ref
- Yichao Wu and Yufeng Liu. 2007. Robust truncated hinge loss support vector machines. J. Amer. Statist. Assoc. 102, 479 (2007), 974–983.Google ScholarCross Ref
- Hwanjo Yu and Sungchul Kim. 2012. SVM tutorial-classification, regression and ranking. Handb. Nat. Comput. 1 (2012), 479–506.Google ScholarCross Ref
- Alan L. Yuille and Anand Rangarajan. 2002. The concave-convex procedure (CCCP). In Proceedings of the International Conference on Advances in Neural Information Processing Systems. 1033–1040. Google ScholarDigital Library
Index Terms
- Pinball Loss Twin Support Vector Clustering
Recommendations
General twin support vector machine with pinball loss function
Highlights- A novel general twin support vector machine with pinball loss is proposed.
- The ...
AbstractThe standard twin support vector machine (TSVM) uses the hinge loss function which leads to noise sensitivity and instability. In this paper, we propose a novel general twin support vector machine with pinball loss (Pin-GTSVM) for ...
Sparse pinball twin support vector machines
AbstractThe original twin support vector machine (TWSVM) formulation works by solving two smaller quadratic programming problems (QPPs) as compared to the traditional hinge-loss SVM (C-SVM) which solves a single large QPP — this makes the ...
Highlights- A novel twin support vector machine with sparse pinball loss (SPTWSVM) is proposed.
Multitask centroid twin support vector machines
Twin support vector machines are a recently proposed learning method for binary classification. They learn two hyperplanes rather than one as in conventional support vector machines and often bring performance improvements. However, an inherent shortage ...
Comments