Abstract
Assigning a confidence and a credibility measures is a challenging stochastic inference problem. Some algorithms only yield the predicted value without evaluating the measure of confidence or credibility over the decision. Support vector machines (SVM) is one algorithm that showed state-of-the-art decision accuracy but lacks a measure of confidence and credibility over the decisions. In this paper we propose a new confidence measure based on the Vapnik and Chervonenkis (VC) dimension of a learning algorithm and the notion of complexity as defined by Kolmogorov. We also propose a new credibility measure based on the VC dimension. The resulting confidence and credibility measures are then tested on the well-known US postal handwritten digit recognition, on the Wisconsin breast cancer dataset and are also tested for agitation detection. The results show high and improved correlation between the decision and the confidence/credibility measures compared to Vovk’s and Platt’s methods.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aronszajn N (1950) Introduction to the theory of Hilbert spaces. Reasearch [sic] Foundation, Stillwater
Brier G (1950) Verification of forecasts expressed in terms of probability. Mon Weather Rev 78(1):1–3
Burges C (1998) A tutorial on support vector machines for pattern recognition. Data Mining Knowl Discov 2(2):121–167
Cun Y, Boser B, Denker J, Howard R, Habbard W, Jackel L, Henderson D (1990) Handwritten digit recognition with a back-propagation network. In: Advances in neural information processing systems 2, Morgan Kaufmann Publishers Inc. pp 396–404
Dawid A (1983) Inference, statistical: I. Encycl Stat Sci 4:89–105
Dumais S, Platt J, Heckerman D, Sahami M (1998) Inductive learning algorithms and representations for text categorization. In: Proceedings of the seventh international conference on Information and knowledge management, ACM. pp 148–155
Ercan, Secil, Kayakutlu G (2014) Patent value analysis using support vector machines. In: Soft computing, vol 18. Springer, pp 313–328
Gammerman A, Vapnik V., Vowk V (1998) Learning by transduction. In: Uncertainty in Artificial Intelligence, Morgan Kaufmann. pp 148–155
Joachims T (1998) Text categorization with support vector machines: Learning with many relevant features. Machine Learning: ECML-98. pp 137–142
Kecman V (2001) Learning and soft computing: support vector machines, neural networks, and fuzzy logic models. The MIT press, London
Kolmogorov A (1968) Three approaches to the quantitative definition of information. Int J Comput Math 2(1):157–168
Li L, Chen J (2006) Emotion recognition using physiological signals. Advances in Artificial Reality and Tele-Existence. pp 437–446
Liu, Zhi and Xu, Shuqiong and Zhang, Yun and Chen, Xin and Chen, Philip CL (2013) Interval type-2 fuzzy kernel based support vector machine algorithm for scene classification of humanoid robot. In: Soft Computing, Springer, pp 1–18
Melluish T, Saunders C, Nouretdinov I, Vovk V (2001) The typicalness framework: a comparison with the bayesian approach. University of London, Royal Holloway
Mitra P, Murthy C, Pal S (2004) A probabilistic active support vector learning algorithm. IEEE Trans Pattern Anal Mach Intell 26(3):413–418
Platt J (1999) Probabilities for SV machines. Advances in Neural Information Processing Systems. pp 61–74
Sakr G, Elhajj I, Huijer H (2010) Support Vector Machines to Define and Detect Agitation Transition. IEEE Transactions on Affective Computing
Sakr G, Elhajj I, Huijer H, Riley-Doucet C, Debnath D (2008) Subject independent agitation detection. In: IEEE/ASME International Conference on Advanced Intelligent Mechatronics, AIM 2008. pp 200–204
Sakr G, Elhajj I, Wejinya U (2009) multi level SVM for subject independent agitation detection. In: IEEE/ASME International Conference on Advanced Intelligent Mechatronics, AIM 2009, pp 538–543
Schölkopf B, Smola A (2002) Learning with kernels. The MIT Press
Spielberger CD, Gorsuch RL, Edward LR (1970) STAI manual for the State-Trait Anxiety Inventory (“self-evaluation questionnaire”). In: Consulting Psychologists Press
Street N, Wolberg W, Mangasarian O (1992) Nuclear feature extraction for breast tumor diagnosis. University of Wisconsin-Madison, Computer Sciences Department
Stroop JR (1935) Studies of interference in serial verbal reactions. J Exp Psychol 18:643–662
Vapnik V (1995) The nature of statistical learnin. Springer, New York
Vapnik V (1998) Statistical learning theory. Wiley, New York
Vovk V, Gammerman A, Saunders C (1999) Machine-learning applications of algorithmic randomness. In: Proceedings of the Sixteenth International Conference on Machine Learning, Morgan Kaufmann Publishers Inc. pp 444–453
Acknowledgments
This research was funded by the American University of Beirut University Research Board, Dar Al-Handassah (Shair & Partners) Research Fund and the Rathman (Kadifa) Fund. We would like to thank Dr. Cheryl Riley-Doucet and Dr. Debatosh Debnath from Oakland University for providing the data used in this research.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by V. Loia.
Rights and permissions
About this article
Cite this article
Sakr, G.E., Elhajj, I.H. VC-based confidence and credibility for support vector machines. Soft Comput 20, 133–147 (2016). https://doi.org/10.1007/s00500-014-1485-4
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-014-1485-4