Skip to main content
SpringerLink
Log in

Hierarchical feature concatenation-based kernel sparse representations for image categorization

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

In order to obtain improved performance in complicated visual categorization tasks, considerable research has adopted multiple kernel learning based on dozens of different features. However, it is a complex process that needs to extract a multitude of features and seeks the optimal combination of multiple kernels. Inspired by the key idea of hierarchical learning, in this paper, we propose to find sparse representation based on feature concatenation using hierarchical kernel orthogonal matching pursuit (HKOMP). In addition to commonly used spatial pyramid feature for kernel representation, our method only employs one type of generic image feature, i.e., p.d.f gradient-based orientation histogram for concatenation of sparse codes. Next, the resulting concatenated features kernelized with widely used Gaussian radial basis kernel function form compact sparse representations in the second layer for linear support vector machine. HKOMP algorithm combines the advantages of building image representations layer-by-layer and kernel learning. Several publicly available image datasets are used to evaluate the presented approach and empirical results for various datasets show that the proposed scheme outperforms many kernel learning based and other competitive image categorization algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Zhang, L., Zhao, Y., Zhu, Z.: Extracting shared subspace incrementally for multi-label image classification. Vis. Comput. 30(12), 1359–1371 (2014)

    Article  Google Scholar 

  2. Liu, X., Shi, Z., Shi, Z.: A co-boost framework for learning object categories from Google Images with 1st and 2nd order features. Vis. Comput. 30(1), 5–17 (2013)

    Article  Google Scholar 

  3. Ji, R., Gao, Y., Hong, R., Liu, Q., Tao, D., Li, X.: Spectral-spatial constraint hyperspectral image classification. IEEE Trans. Geosci. Remote Sens. 52(3), 1811–1824 (2014)

    Article  Google Scholar 

  4. Gao, Y., Wang, M., Zha, Z., Shen, J., Li, X., Wu, X.: Visual-textual joint relevance learning for tag-based social image search. IEEE Trans. Image Process. 22(1), 363–376 (2013)

    Article  MathSciNet  Google Scholar 

  5. Gao, Y., Wang, M., Tao, D., Ji, R., Dai, Q.: 3D object retrieval and recognition with hypergraph analysis. IEEE Trans. Image Process. 21(9), 4290–4303 (2012)

    Article  MathSciNet  Google Scholar 

  6. Gao, Y., Wang, M., Ji, R., Wu, X., Dai, Q.: 3D object retrieval with hausdorff distance learning. IEEE Trans. Ind. Electron. 61(4), 2088–2098 (2014)

    Article  Google Scholar 

  7. Guan, T., He, Y., Duan, L.: Efficient BOF generation and compression for on-device mobile visual location recognition. IEEE Multimed. 21(2), 32–41 (2014)

    Article  Google Scholar 

  8. Guan, T., He, Y., Gao, J., Yang, J., Yu, J.: On-device mobile visual location recognition by integrating vision and inertial sensors. IEEE Trans. Multimed. 21(2), 32–41 (2014)

    Article  Google Scholar 

  9. Guan, T., Wang, Y., Duan, L., Ji, R.: On-device mobile landmark recognition using binarized sescriptor with multifeature fusion. ACM Trans. Intell. Syst. Technol. 7(1), 1–28 (2015)

    Article  Google Scholar 

  10. Zhao, Y., Yang, J.: Hyperspectral image denoising via sparse representation and low-rank constraint. IEEE Trans. Geosci. Remote Sens. 53(1), 296–308 (2015)

    Article  Google Scholar 

  11. Zhao, Z., Glotin, H., Xie, Z., Gao, J., Wu, X.: Cooperative sparse representation in two opposite directions for semi-supervised image annotation. IEEE Trans. Image Process. 21(9), 4218–4231 (2012)

    Article  MathSciNet  Google Scholar 

  12. Chiang, C.-K., Liu, C.-H., Duan, C.-H., Lai, S.-H.: Learning component-level sparse representation for image and video categorization. IEEE Trans. Image Process. 22(12), 4775–4787 (2013)

    Article  MathSciNet  Google Scholar 

  13. Wang, L., Yan, H., Lv, K., Pan, C.: Visual tracking via kernel sparse representation with multikernel fusion. IEEE Trans. Circuits Syst. Video Technol. 24(7), 1132–1141 (2014)

    Article  Google Scholar 

  14. Zhang, L., Zhou, W., Chang, P.-C., Liu, J., Yan, Z., Wang, T., Li, F.: Kernel sparse representation-based classifier. IEEE Trans. Signal Process. 60(4), 1684–1695 (2012)

    Article  MathSciNet  Google Scholar 

  15. Gehler, P., Nowozin, S.: On feature combination for multiclass object classification. In: IEEE International Conference on Computer Vision, pp. 221–228 (2009)

  16. Zheng, J., Huang, Q., Chen, S., Wang, W.: Efficient kernel discriminative common vectors for classification. Vis. Comput. 31(5), 643–655 (2015)

    Article  Google Scholar 

  17. Nguyen, H., Patel, V., Nasrabad, N., Chellappa, R.: Design of non-linear kernel dictionaries for object recognition. IEEE Trans. Image Process. 22(12), 5123–5135 (2013)

    Article  MathSciNet  Google Scholar 

  18. Gao, S., Tsang, I.W., Chia, L.-T.: Sparse representation with kernels. IEEE Trans. Image Process. 22(2), 423–434 (2013)

    Article  MathSciNet  Google Scholar 

  19. Jian, M., Jung, C.: Class-discriminative kernel sparse representation-based classification using multi-objective optimization. IEEE Trans. Signal Process. 61(18), 4416–4427 (2013)

    Article  MathSciNet  Google Scholar 

  20. Shawe-Taylor, J., Cristianini, N.: Kernel Methods for Pattern Analysis. Cambridge University Press, Cambridge (2004)

    Book  MATH  Google Scholar 

  21. Bosch, A., Zisserman, A., Munoz, X.: Representing shape with a spatial pyramid kernel. In: Proceedings of the 6th ACM International Conference on Image Video Retrieval, pp. 401–408 (2007)

  22. Shechtman, E., Irani, M.: Matching local self-similarities across images and videos. In: IEEE International Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007)

  23. Tuytelaars, T.: Dense interest points. In: IEEE International Conference on Computer Vision and Pattern Recognition, pp. 2281–2288 (2010)

  24. Boureau, Y., Bach, F., Yann, L., Ponce, J.: Learning mid-level features for recognition. In: IEEE International Conference on Computer Vision and Pattern Recognition, pp. 2559–2566 (2010)

  25. Sonnenburg, S., Rätsch, G., Schäfer, C., Schölkopf, B.: Large scale multiple kernel learning. J. Mach. Learn. Res. 7, 1531–1565 (2006)

    MathSciNet  MATH  Google Scholar 

  26. Rakotomamonjy, A., Bach, F., Canu, S., Grandvalet, Y.: More efficiency in multiple kernel learning. In: Proceedings of the 24th International Conference on Machine Learning, pp. 775–782 (2007)

  27. Vedaldi, A., Gulshan, V., Varma, M., Zisserman, A.: Multiple kernels for object detection. In: IEEE International Conference on Computer Vision, pp. 606–613 (2009)

  28. Xiao, J., Hays, J., Ehinger, K.A., Oliva, A., Torralba, A.: SUN database: large-scale scene recognition from abbey to zoo. In: IEEE International Conference on Computer Vision and Pattern Recognition, pp. 3485–3492 (2010)

  29. Patterson, G., Hays, J.: SUN attribute database: discovering, annotating, and recognizing scene attributes. In: IEEE International Conference on Computer Vision and Pattern Recognition, pp. 2751–2758 (2012)

  30. Yang, J., Tian, Y., Duan, L.-Y., Huang, T., Gao, W.: Group-sensitive multiple kernel learning for object recognition. IEEE Trans. Image Process. 21(5), 2838–2852 (2012)

    Article  MathSciNet  Google Scholar 

  31. Jain, A., Vishwanathan, S.V.N., Varma, M.: SPF-GMKL: generalized multiple kernel learning with a million kernels. In: Proceedings of the 18th ACM International Conference on Knowledge Discovery and Data Mining, pp. 750–758 (2012)

  32. Gönen, M., Alpaydin, E.: Multiple kernel learning algorithms. J. Mach. Learn. Res. 12, 2211–2268 (2011)

    MathSciNet  MATH  Google Scholar 

  33. Ojala, T., Pietikainen, M., Maenpaa, T.: Multiresolution grayscale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 971–987 (2002)

    Article  MATH  Google Scholar 

  34. Li, C., Zhou, W., Yuan, S.: Iris recognition based on a novel variation of local binary pattern. Vis. Comput. 31(10), 1419–1429 (2015)

    Article  Google Scholar 

  35. Tuzel, O., Porikli, F., Meer, P.: Human detection via classification on riemannian manifolds. In: IEEE International Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007)

  36. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  37. Murtza, I., Abdullah, D., Khan, A., Arif, M., Mirza, S.M.: Cortex-inspired multilayer hierarchy based object detection system using PHOG descriptors and ensemble classification. Vis. Comput. 1–14 (2015)

  38. Oliva, A., Torralba, A.: Modeling the shape of the scene: a holistic representation of the spatial envelope. Int. J. Comput. Vis. 42(3), 145–175 (2001)

    Article  MATH  Google Scholar 

  39. Pinto, N., Cox, D.D., Dicarlo, J.J.: Why is real-world visual object recognition hard. PLOS Comput. Biol. 4(1), e27 (2008)

    Article  MathSciNet  Google Scholar 

  40. Berg, A., Malik, J.: Geometric blur for template matching. In: IEEE International Conference on Computer Vision and Pattern Recognition, pp. 607–614 (2001)

  41. Varma, M., Ray, D.: Learning the discriminative power-invariance trade-off. In: IEEE International Conference on Computer Vision, pp. 1–8 (2007)

  42. Bucak, S., Jin, R., Jain, A.: Multiple kernel learning for visual object recognition: a review. IEEE Trans. Pattern Anal. Mach. Intell. 36(7), 1354–1369 (2014)

    Article  Google Scholar 

  43. Aiolli, F., Donini, M.: EasyMKL: a scalable multiple kernel learning algorithm. Neural Comput. 169, 215–224 (2015)

    Google Scholar 

  44. Kobayashi, T.: BFO meets HOG: feature extraction based on histograms of oriented p.d.f. gradients for image classification. In: IEEE International Conference on Computer Vision and Pattern Recognition, pp. 947–954 (2013)

  45. Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: IEEE International Conference on Computer Vision and Pattern Recognition, pp. 2169–2178 (2006)

  46. Yin, J., Liu, Z., Jin, Z., Yang, W.: Kernel sparse representation based classification. Neural Comput. 77, 120–128 (2011)

    Google Scholar 

  47. Li, H., Gao, Y., Sun, J.: Fast kernel sparse representation. In: Proceedings of International Conference on Digital Image Computing Techniques and Applications, pp. 72–77 (2011)

  48. Nguyen, H., Patel, V., Nasrabadi, N.M., Chellappa, R.: Kernel dictionary learning. In: Proceedings of IEEE International Conference on Acoustics, Speech, Signal Process, pp. 2021–2024 (2012)

  49. Yu, K., Lin, Y., Lafferty, J.: Learning image representations from the pixel level via hierarchical Sparse coding. In: IEEE International Conference on Computer Vision and Pattern Recognition, pp. 1713–1720 (2011)

  50. Bo, L., Ren, X., Fox, D.: Hierarchical matching pursuit for image classification: architecture and fast algorithms. In: Advances in neural information processing systems, pp. 2115–2123

  51. Bo, L., Ren, X., Fox, D.: Multipath sparse coding using hierarchical matching pursuit. In: IEEE International Conference on Computer Vision and Pattern Recognition, pp. 660–667 (2013)

  52. Liu, B., Liu, J., Bai, X., Lu, H.: Regularized hierarchical feature learning with non-negative sparsity and selectivity for image classification. In: Proceedings of the 22nd International Conference on Pattern Recognition, pp. 4293–4298 (2014)

  53. Wu, J., Rehg, J.M.: Beyond the Euclidean distance: creating effective visual codebooks using the histogram intersection kernel. In: IEEE International Conference on Computer Vision, pp. 630–637 (2009)

  54. Shrivastava, A., Patel, V., Chellappa, R.: Multiple kernel learning for sparse representation-based classification. IEEE Trans. Image Process. 23(7), 3013–3024 (2014)

    Article  MathSciNet  Google Scholar 

  55. Zhang, L., Zhen, X., Shao, L.: Learning object-to-class kernels for scene classification. IEEE Trans. Image Process. 23(8), 3241–3253 (2014)

    Article  MathSciNet  Google Scholar 

  56. Wang, P., Wang, J., Zeng, G., Xu, W., Zha, H., Li, S.: Supervised kernel descriptors for visual recognition. In: IEEE International Conference on Computer Vision and Pattern Recognition, pp. 2858–2865 (2013)

  57. Han, Y., Liu, G.: Probability-confidence-kernel-based localized multiple kernel learning with lp norm. IEEE Trans. Syst. Man Cybern. B 42(3), 827–837 (2012)

    Article  Google Scholar 

  58. Han, Y., Yang, K., Ma, Y., Liu, G.: Localized multiple kernel learning via sample-wise alternating optimization. IEEE Trans. Cybern. 44(1), 137–148 (2014)

    Article  Google Scholar 

  59. Yan, S., Xu, X., Xu, D., Lin, S., Li, X.: Image classification with densely sampled image windows and generalized adaptive multiple kernel learning. IEEE Trans. Cybern. 45(3), 395–404 (2015)

    Article  Google Scholar 

  60. Thiagarajan, J., Ramamurthy, K., Spanias, A.: Multiple kernel sparse representations for supervised and unsupervised learning. IEEE Trans. Image Process. 23(7), 2905–2915 (2014)

    Article  MathSciNet  Google Scholar 

  61. Nilsback, M.-E., Zisserman, A.: Automated flower classification over a large number of classes. In: Proceedings of the 6th Indian Conference on Computer Vision, Graphics and Image Processing, pp. 722–729 (2008)

  62. Yuan, X.-T., Yan, S.: Visual classification with multi-task joint sparse representation. In: IEEE International Conference on Computer Vision and Pattern Recognition, pp. 3493–3500 (2010)

Download references

Acknowledgments

This project is supported by the National Program on Key Basic Research Project (No. 2014CB340403) and Natural Science Foundation of Tianjin (No. 15JCYBJC15500). The authors would like to be grateful to the editors’ and reviewers’ valuable comments which improved the quality of this paper.

Author information

Authors and Affiliations

  1. School of Electronic Information Engineering, Tianjin University, Tianjin, 300072, China

    Bo Wang, Jichang Guo, Yan Zhang & Chongyi Li

Authors
  1. Bo Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jichang Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chongyi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bo Wang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, B., Guo, J., Zhang, Y. et al. Hierarchical feature concatenation-based kernel sparse representations for image categorization. Vis Comput 33, 647–663 (2017). https://doi.org/10.1007/s00371-016-1215-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-016-1215-2

Keywords

Search

Navigation