Abstract
Support vector machines (SVMs) have good accuracy and generalization properties, but they tend to be slow to classify new examples. In contrast to previous work that aims to reduce the time required to fully classify all examples, we present a method that provides the best-possible classification given a specific amount of computational time. We construct two SVMs: a “full” SVM that is optimized for high accuracy, and an approximation SVM (via reduced-set or subset methods) that provides extremely fast, but less accurate, classifications. We apply the approximate SVM to the full data set, estimate the posterior probability that each classification is correct, and then use the full SVM to reclassify items in order of their likelihood of misclassification. Our experimental results show that this method rapidly achieves high accuracy, by selectively devoting resources (reclassification) only where needed. It also provides the first such progressive SVM solution that can be applied to multiclass problems.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Burges CJC (1996) Simplified support vector decision rules. In: Proceedings of the thirteenth international conference on machine learning, pp 71–77
Burges CJC (1998) A tutorial on support vector machines for pattern recognition. Data Min Knowl Discov 2(2): 121–167
Castano R, Wagstaff KL, Chien S, Stough TM, Tang B (2007) On-board analysis of uncalibrated data for a spacecraft at Mars. In: Proceedings of the thirteenth international conference on knowledge discovery and data mining, pp 922–930
Chien S, Sherwood R, Tran D, Cichy B, Rabideau G, Castaño R, Davies A, Mandel D, Frye S, Trout B, Shulman S, Boyer D (2005) Using autonomy flight software to improve science return on Earth observing one. J Aerosp Comput Inf Commun 2(4): 196–216
Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20: 273–297
DeCoste D (2002) Anytime interval-valued outputs for kernel machines: fast support vector machine classification via distance geometry. In: Proceedings of the nineteenth international conference on machine learning, pp 99–106
DeCoste D (2003) Anytime query-tuned kernel machines via Cholesky factorization. In: Proceedings of the SIAM international conference on data mining (SIAMDM-03)
DeCoste D, Mazzoni D (2003) Fast query-optimized kernel machine classification via incremental approximate nearest support vectors. In: Proceedings of the twentieth international conference on machine learning, pp 115–122
DeCoste D, Scholkopf B (2002) Training invariant support vector machines. Mach Learn 26(1–3): 161–190
Diner DJ, Beckert JC, Reilly TH, Bruegge CJ, Conel JE, Kahn R, Martonchik JV, Ackerman TP, Gordon HR, Muller J-P, Myneni R, Sellers RJ, Pinty B, Verstraete MM (1998) Multiangle imaging spectroradiometer (MISR) instrument description and experiment overview. IEEE Trans Geosci Remote Sens 36: 1072–1087
Duan K, Keerthi SS (2005) Which is the best multiclass SVM method? An empirical study. In: Proceedings of multiple classifier systems, pp 278–285
Hastie T, Tibshirani R (1998) Classification by pairwise coupling. Ann Appl Stat 26(2): 451–471
Lin HT, Lin CJ, Weng RC (2003) A note on Platt’s probabilistic outputs for support vector machines. Technical report, National Taiwan University
Mazzoni D, Garay MJ, Davies R, Nelson D (2007) An operational MISR pixel classifier using support vector machines. Remote Sens Environ 107(1–2): 149–158
Newman DJ, Hettich S, Blake CL, Merz CJ (1998) UCI repository of machine learning databases. http://www.ics.uci.edu/~mlearn/MLRepository.html
Platt JC (1999) Probabilities for SV machines. In: Smola AJ, Bartlett P, Schölkopf B, Schuurmans D (eds) Advances in large margin classifiers. MIT Press, Cambridge, pp 61–74
Ratsch M, Romdhani S, Vetter T (2004) Efficient face detection by a cascaded support vector machine using Haar-like features. In: Proceedings of the German pattern recognition symposium, pp 62–70
Romdhani S, Torr PHS, Schölkopf B, Blake A (2001) Computationally efficient face detection. In: Proceedings of the eighth international conference on computer vision, pp 695–700
Tang B, Mazzoni D (2006) Multiclass reduced-set support vector machines. In: Proceedings of the twenty-third international conference on machine learning, pp 921–928
Author information
Authors and Affiliations
Corresponding author
Additional information
Responsible editor: Geoffrey Webb.
Rights and permissions
About this article
Cite this article
Wagstaff, K.L., Kocurek, M., Mazzoni, D. et al. Progressive refinement for support vector machines. Data Min Knowl Disc 20, 53–69 (2010). https://doi.org/10.1007/s10618-009-0149-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10618-009-0149-y