Abstract
We propose the new multi-class of bagged proximal support vector machines (MC-Bag-PSVM) for handling the ImageNet challenging problem with very large number of images and a thousand classes. Our MC-Bag-PSVM trains in the parallel manner ensemble binary PSVM classifiers used for the One-Versus-All (OVA) multi-class strategy on multi-core computer with GPUs. The binary PSVM model is constructed by bagged binary PSVM models built in under-sampling training dataset. The numerical test results on ILSVRC 2010 dataset show that our MC-Bag-PSVM algorithm is faster and more accurate than the state-of-the-art linear SVM algorithm. An example of its effectiveness is given with an accuracy of 75.64% obtained in the classification of ImageNet-1000 dataset having 1,261,405 images in 2048 deep features into 1,000 classes in 29.5 min using a PC Intel(R) Core i7-4790 CPU, 3.6 GHz, 4 cores and Gigabyte GeForce RTX 2080Ti 11 GB GDDR6, 4352 CUDA cores.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bosch, A., Zisserman, A., Muñoz, X.: Scene classification Via pLSA. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3954, pp. 517–530. Springer, Heidelberg (2006). https://doi.org/10.1007/11744085_40
Breiman, L.: Bagging predictors. Mach. Learn. 24(2), 123–140 (1996)
Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2(27), 1–27 (2011)
Chollet, F.: Xception: deep learning with depthwise separable convolutions. CoRR abs/1610.02357 (2016)
Deng, J., Berg, A.C., Li, K., Fei-Fei, L.: What does classifying more than 10,000 image categories tell us? In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6315, pp. 71–84. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15555-0_6
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Li, F.F.: ImageNet: a large-scale hierarchical image database. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009)
Do, T.-N.: Parallel multiclass stochastic gradient descent algorithms for classifying million images with very-high-dimensional signatures into thousands classes. Vietnam J. Comput. Sci. 1(2), 107–115 (2014). https://doi.org/10.1007/s40595-013-0013-2
Do, T.-N., Nguyen, V.-H., Poulet, F.: Speed up SVM algorithm for massive classification tasks. In: Tang, C., Ling, C.X., Zhou, X., Cercone, N.J., Li, X. (eds.) ADMA 2008. LNCS (LNAI), vol. 5139, pp. 147–157. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-88192-6_15
Do, T.-N., Poulet, F.: Parallel multiclass logistic regression for classifying large scale image datasets. In: Le Thi, H.A., Nguyen, N.T., Do, T.V. (eds.) Advanced Computational Methods for Knowledge Engineering. AISC, vol. 358, pp. 255–266. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-17996-4_23
Do, T.-N., Tran-Nguyen, M.-T.: Incremental parallel support vector machines for classifying large-scale multi-class image datasets. In: Dang, T.K., Wagner, R., Küng, J., Thoai, N., Takizawa, M., Neuhold, E. (eds.) FDSE 2016. LNCS, vol. 10018, pp. 20–39. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-48057-2_2
Doan, T.-N., Do, T.-N., Poulet, F.: Large scale classifiers for visual classification tasks. Multimed. Tools Appl. 74(4), 1199–1224 (2014). https://doi.org/10.1007/s11042-014-2049-4
Fan, R.E., Chang, K.W., Hsieh, C.J., Wang, X.R., Lin, C.J.: LIBLINEAR: a library for large linear classification. J. Mach. Learn. Res. 9(4), 1871–1874 (2008)
Fung, G., Mangasarian, O.L.: Proximal support vector machine classifiers. In: Proceedings of the seventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, 26–29 August 2001, pp. 77–86 (2001)
Fung, G., Mangasarian, O.L.: Multicategory proximal support vector machine classifiers. Mach. Learn. 59(1–2), 77–97 (2005)
He, H., Garcia, E.A.: Learning from imbalanced data. IEEE Trans. Knowl. Data Eng. 21(9), 1263–1284 (2009)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. CoRR abs/1512.03385 (2015)
Herrero-Lopez, S., Williams, J.R., Sanchez, A.: Parallel multiclass classification using SVMs on GPUs. In: Proceedings of the 3rd Workshop on General-Purpose Computation on Graphics Processing Units, pp. 2–11. ACM, New York (2010)
Hido, S., Kashima, H.: Roughly balanced bagging for imbalanced data. In: SIAM International Conference on Data Mining, pp. 143–152 (2008)
Japkowicz, N. (ed.): AAAI’Workshop on Learning from Imbalanced Data Sets. No. WS-00-05 in AAAI Tech Report (2000)
Kreßel, U.H.G.: Pairwise classification and support vector machines. In: Schölkopf, B., Burges, C.J.C., Smola, A.J. (eds.) Advances in Kernel Methods, pp. 255–268. MIT Press, Cambridge (1999)
Lenca, P., Lallich, S., Do, T.-N., Pham, N.-K.: A comparison of different off-centered entropies to deal with class imbalance for decision trees. In: Washio, T., Suzuki, E., Ting, K.M., Inokuchi, A. (eds.) PAKDD 2008. LNCS (LNAI), vol. 5012, pp. 634–643. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-68125-0_59
Li, F., Perona, P.: A Bayesian hierarchical model for learning natural scene categories. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005), 20–26 June 2005, San Diego, CA, USA, pp. 524–531 (2005)
Lowe, D.: Object recognition from local scale invariant features. In: Proceedings of the 7th International Conference on Computer Vision, pp. 1150–1157 (1999)
Lowe, D.: Distinctive image features from scale invariant keypoints. Int. J. Comput. Vis. 60, 91–110 (2004)
OpenMP Architecture Review Board: OpenMP application program interface version 3.0 (2008). http://www.openmp.org/mp-documents/spec30.pdf
Perronnin, F., Sánchez, J., Liu, Y.: Large-scale image categorization with explicit data embedding. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 2297–2304 (2010)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. CoRR abs/1409.1556 (2014)
Sivic, J., Zisserman, A.: Video Google: a text retrieval approach to object matching in videos. In: 9th IEEE International Conference on Computer Vision (ICCV 2003), 14–17 October 2003, Nice, France, pp. 1470–1477 (2003)
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. CoRR abs/1512.00567 (2015)
Vapnik, V.: The Nature of Statistical Learning Theory. Springer, Heidelberg (1995). https://doi.org/10.1007/978-1-4757-2440-0
Whaley, R., Dongarra, J.: Automatically tuned linear algebra software. In: Ninth SIAM Conference on Parallel Processing for Scientific Computing (1999). cD-ROM Proceedings
Wu, J.: Power mean SVM for large scale visual classification. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 2344–2351 (2012)
Acknowledgments
This work has received support from the College of Information Technology, Can Tho University. We would like to thank very much the Big Data and Mobile Computing Laboratory.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Do, TN. (2021). Multi-class Bagged Proximal Support Vector Machines for the ImageNet Challenging Problem. In: Dang, T.K., Küng, J., Chung, T.M., Takizawa, M. (eds) Future Data and Security Engineering. FDSE 2021. Lecture Notes in Computer Science(), vol 13076. Springer, Cham. https://doi.org/10.1007/978-3-030-91387-8_7
Download citation
DOI: https://doi.org/10.1007/978-3-030-91387-8_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-91386-1
Online ISBN: 978-3-030-91387-8
eBook Packages: Computer ScienceComputer Science (R0)