Abstract
The amount of image data becomes larger and larger, both image size (due the higher resolution) and image number. It is estimated for personal use only, an average single user will take 100,000 images during his life. The growth of image data is illustrated by the dataset size, for example ImageNet benchmark dataset is made of more than 14 million images and more than 21,000 classes. This is very challenging for classification algorithms. They have to deal with time and space complexity and very imbalanced data when using SVM algorithms. We present extensions of Power Mean SVM to deal with such data. The first one is an incremental version to deal with the space complexity, the second one is a parallel version of the incremental version to deal with time complexity and the last one is the use of a balanced bagging algorithm for training binary classifiers to deal with imbalanced data. We evaluate our parallel incremental version of balanced bagging PmSVM on the 1,000 classes of ImageNet (ILSVRC 2010). The results show that our algorithm can be run on standard PC (with eg. 2 or 4 GB RAM); it is 255 times faster than the original version and 1,276 times faster than state-of-the-art linear classifier, LIBLINEAR with 80 cores.
Similar content being viewed by others
References
Berg A, Deng J, Li FF (2010) Large scale visual recognition challenge 2010. Tech Rep. http://www.image-net.org/challenges/LSVRC/2010/index
Chua TS, Tang J, Hong R, Li H, Luo Z, Zheng, YT (2009) Nus-wide: a real-world web image database from national university of Singapore. In: Proceedings of ACM Conference on Image and Video Retrieval (CIVR’09). Santorini, Greece
Csurka G, Dance CR, Fan L, Willamowski J, Bray C (2004) Visual categorization with bags of keypoints. In: Workshop on Statistical Learning in Computer Vision, ECCV, pp 1–22
Deng J, Berg AC, Li K, Li FF (2010) What does classifying more than 10, 000 image categories tell us? In: Daniilidis K, Maragos P, Paragios N (eds) ECCV, Part V. Lecture Notes in Computer Science, vol 6315. Springer pp 71–84
Do TN, Nguyen VH, Poulet F (2008) Speed up SVM algorithm for massive classification tasks. In: Tang C, Ling CX, Zhou X, Cercone N, Li X (eds) ADMA. Lecture Notes in Computer Science, vol 5139. Springer
Everingham M, Van Gool L, Williams CKI, Winn J, Zisserman A (2010) The Pascal visual object classes (VOC) challenge. Int J Comput Vis 88(2):303–338
Griffin G, Holub A, Perona P (2007) Caltech-256 object category dataset. Tech. Rep. CNS-TR-2007-001. California Institute of Technology. http://authors.library.caltech.edu/7694
Guermeur Y (2007) SVM multiclasses, théorie et applications
Hsieh CJ, Chang KW, Lin CJ, Keerthi SS, Sundararajan S (2008) A dual coordinate descent method for large-scale linear SVM. In: International Conference on Machine Learning, pp 408–415
Huiskes MJ, Thomee B, Lew MS (2010) New trends and ideas in visual concept detection: The mir Flickr retrieval evaluation initiative. In: Proceedings of the International Conference on Multimedia Information Retrieval, MIR ’10. ACM, New York, pp 527–536. doi:10.1145/1743384.1743475. http://doi.acm.org/10.1145/1743384.1743475
Krebel UH-G (1999) Pairwise classification and support vector machines. In: Schölkopf B, Burges CJC, Smola AJ (eds) Advances in Kernel methods. MIT Press, Cambridge, pp 255–268
Lenca P, Lallich S, Do TN, Pham NK (2008) A comparison of different off-centered entropies to deal with class imbalance for decision trees. In: The Pacific-Asia Conference on Knowledge Discovery and Data Mining, LNAI 5012. Springer, New York, pp 634–643
Li FF, Fergus R, Perona P (2007) Learning generative visual models from few training examples: an incremental Bayesian approach tested on 101 object categories. Comput Vis Image Underst 106(1):59–70
Lin Y, Lv F, Zhu S, Yang M, Cour T, Yu K, Cao L, Huang TS (2011) Large-scale image classification: fast feature extraction and SVM training. In: CVPR. IEEE pp 1689–1696
Lowe DG (2004) Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60(2), 91–110. http://dx.doi.org/10.1023/B:VISI.0000029664.99615.94
MPI-Forum.: MPI: a message-passing interface standard URL http://www.mpi-forum.org
OpenMP Architecture Review Board: OpenMP application program interface version 3.0 (2008). http://www.openmp.org/mp-documents/spec30.pdf
Perronnin F, Sánchez J, Liu Y (2010) Large-scale image categorization with explicit data embedding. In: CVPR. IEEE, pp 2297–2304
Pham NK, Do TN, Lenca P, Lallich S (2008) Using local node information in decision trees: coupling a local decision rule with an off-centered entropy. In: International Conference on Data Mining. CSREA Press, Las Vegas, pp 117–123
Platt J, Cristianini N, Shawe-Taylor J (2000) Large margin dags for multiclass classification. Adv Neural Inf Process Syst 12:547–553
Vapnik V (1995) The nature of statistical learning theory. Springer, New York
Vedaldi A, Zisserman A (2012) Efficient additive kernels via explicit feature maps. IEEE Trans Pattern Anal Mach Intell 34(3):480–492
Visa S, Ralescu A (2005) Issues in mining imbalanced data sets—a review paper. In: Midwest Artificial Intelligence and Cognitive Science Conference. Dayton, USA pp 67–73
Weiss GM, Provost F (2003) Learning when training data are costly: the effect of class distribution on tree induction. J Artif Intell Res 19:315–354
Weston J, Watkins C (1999) Support vector machines for multi-class pattern recognition. In: Proceedings of the Seventh European Symposium on Artificial, Neural Networks, pp 219–224
Wu J (2010) A fast dual method for hik svm learning. In: Daniilidis K, Maragos P, Paragios N (eds) European Conference on Computer Vision, Lecture Notes in Computer Science. Springer, New York, vol 6312, pp 552–565
Wu J (2012) Power mean svm for large scale visual classification. In: CVPR. IEEE pp 2344–2351
Wu J, Tan WC, Rehg JM (2011) Efficient and effective visual codebook generation using additive kernels. J Mach Learn Res 12:3097–3118
Yu HF, Hsieh CJ, Chang KW, Lin CJ (2012) Large linear classification when data cannot fit in memory. TKDD 5(4):23
Yuan GX, Ho CH, Lin CJ (2012) Recent advances of large-scale linear classification. Proc IEEE 100(9):2584–2603
Acknowledgments
This work was partially funded by Region Bretagne (France) and VIED (Vietnam International Education Development).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Doan, TN., Do, TN. & Poulet, F. Parallel incremental power mean SVM for the classification of large-scale image datasets. Int J Multimed Info Retr 3, 89–96 (2014). https://doi.org/10.1007/s13735-014-0053-0
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13735-014-0053-0