Skip to main content
SpringerLink
Log in

An Image Mosaic Method Based on Convolutional Neural Network Semantic Features Extraction

  • Published:
Journal of Signal Processing Systems Aims and scope Submit manuscript

Abstract

Since traditional image feature extraction methods rely on features such as corner points, a new method based on semantic feature extraction is proposed inspiring by convolution neural attack. The semantic features of each pixel in an image are computed and quantified by neural network to represent the contribution of each pixel to the image semantics. According to the quantization results, the semantic contribution values of each pixel are sorted, and the semantic feature points are selected from high to low and the image mosaic is completed. Experimental results show that this method can effectively extract image features and complete image mosaic.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5

Similar content being viewed by others

References

  1. Szeliski, R. (1996). Video mosaics for virtual environments. IEEE Computer Graphics and Applications, 16(2), 22–30.

    Article  Google Scholar 

  2. Peleg, S., Rousso, B., Rav-Acha, A., et al. (2000). Mosaicing on Adaptive Manifolds. IEEE Trans on Pami, 22(10), 1144–1154.

    Article  Google Scholar 

  3. Zokai, S., & Wolberg, G. (2005). Image registration using log-polar mappings for recovery of large-scale similarity and projective transformations. IEEE Transactions on Image Processing, 14(10), 1422–1434.

    Article  MathSciNet  Google Scholar 

  4. Pratt, W. (1974). Correlation Techniques of Image Registration. IEEE Trans Aes, 10(3), 353–358.

    Google Scholar 

  5. Harris, C., & Stephens, M. (1988). A combined corner and edge detector. In Proceedings of Fourth Alvey Vision Conference (pp. 147–151).

    Google Scholar 

  6. Dalal, N., Triggs, B. (2005). Histograms of oriented gradients for human detection. IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE, 886-893.

  7. Lowe, D.,. G. (1999). Object Recognition from Local Scale-Invariant Features. IEEE International Conference on Computer Vision, 1150.

  8. Bay, H., Ess, A., Tuytelaars, T., et al. (2008). Speeded-Up Robust Features. Computer Vision and Image Understanding, 110(3), 404–417.

    Article  Google Scholar 

  9. Lécun, Y., Bottou, L., Bengio, Y., et al. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278–2324.

    Article  Google Scholar 

  10. Simonyan, K., Zisserman, A. (2015). Very Deep Convolutional Networks for Large-Scale Image Recognition. International Conference of Learning Representation.

  11. Girshick, R., Donahue, J., Darrell, T., Malik, J. (2014). Rich feature hierarchies for accurate object detection and semantic segmentation. Conference on Computer Vision and Pattern Recognition.

  12. Ren, S., He, K., Girshick, R., et al. (2016). Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6), 1137–1149.

    Article  Google Scholar 

  13. Sarkar, S., Venugopalan, V., Reddy, K., et al. (2017). Deep Learning for Automated Occlusion Edge Detection in RGB-D Frames. Journal of Signal Processing Systems, 88(2), 205–217.

    Article  Google Scholar 

  14. Nakjai, P., & Katanyukul, T. (2018). Hand Sign Recognition for Thai Finger Spelling: An Application of Convolution Neural Network. Journal of Signal Processing Systems, 91(3), 131–146.

    Google Scholar 

  15. Long, J., Shelhamer, E., & Darrell, T. (2014). Fully convolutional networks for semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(4), 640–651.

    Google Scholar 

  16. Zheng, S., Jayasumana, S., Romera-Paredes, B., et al. (2015). Conditional Random Fields as Recurrent Neural Networks, 2015 IEEE International Conference on Computer Vision, 1529-1537.

  17. Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). ImageNet classification with deep convolutional neural networks. International Conference on Neural Information Processing Systems, 1097–1105.

  18. Szegedy, C., Zaremba, W., Sutskever, I., et al. (2013). Intriguing properties of neural networks. International Conference of Learning Representation, 2014, 1–9.

    Google Scholar 

  19. Moosavidezfooli, S. M., Fawzi, A., & Frossard, P. (2016). DeepFool: A Simple and Accurate Method to Fool Deep Neural Networks. Computer Vision and Pattern Recognition, 2574–2582.

  20. Goodfellow, I. (2014). J., Shlens, J., Szegedy, C. Explaining and Harnessing Adversarial Examples. International Conference of Learning Representation, 2015, 1–11.

    Google Scholar 

  21. Papernot, N., Mcdaniel, P., Jha, S., et al. (2016). The Limitations of Deep Learning in Adversarial Settings. IEEE European Symposium on Security and Privacy, 372–387.

  22. Papernot, N., Mcdaniel, P., Goodfellow, I., et al. (2017). Practical Black-Box Attacks against Machine Learning, Asia CCS (pp. 506–519).

    Google Scholar 

  23. Narodytska, N., Kasiviswanathan, S. (2017). Simple Black-Box Adversarial Attacks on Deep Neural Networks. Computer Vision and Pattern Recognition Workshops, 1310-1318.

  24. Li, J., Wang, Z. M., Lai, S. M., et al. (2018). Parallax-Tolerant Image Stitching Based on Robust Elastic Warping. IEEE Transactions on Multimedia, 20(7), 1672–1687.

    Article  Google Scholar 

  25. Brown, M., Lowe, D. G. (2003). Recognising Panoramas. Brown, M., & Lowe, D. G. (2003). Recognising Panoramas. 9th IEEE International Conference on Computer Vision (ICCV 2003).

  26. Brown, M., & Lowe, D. G. (2007). Automatic panoramic image stitching using invariant features. International Journal of Computer Vision, 74(1), 59–73.

    Article  Google Scholar 

  27. Gao, J., Kim, S. J., Brown, M. S. (2011). Constructing image panoramas using dual-homography warping. 2011 IEEE Conference on Computer Vision & Pattern Recognition (CVPR).

  28. Verdie, Y., Yi, K. M., Fua, P., Lepetit, V. (2015). Tilde: a temporally invariant learned detector. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

  29. Yi, K. M., Verdie, Y., Fua, P., Lepetit, V. (2015). Learning to Assign Orientations to Feature Points, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

  30. Simo-Serra, E., Trulls, E., Ferraz, L., Kokkinos, I., Fua, P., Moreno-Noguer, F. (2015). Discriminative Learning of Deep Convolutional Feature Point Descriptors. 2015 IEEE International Conference on Computer Vision (ICCV). IEEE Computer Society.

  31. Domingos, P. (2012). A few useful things to know about machine learning. Communications of the ACM, 55(10), 78–87.

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (NO.61674115), and the Natural Science Foundation of Tianjin City, China (No.17JCYBJC15900).

Author information

Authors and Affiliations

  1. School of Microelectronics, Tianjin University, 92 Weijin Road, Nankai District, Tianjin, 300072, China

    Zaifeng Shi, Hui Li, Huizheng Ren & Boyu Fan

  2. Tianjin Key Lab of Imaging & Sensing Microelectronics Technology, Tianjin, China

    Zaifeng Shi

  3. School of Mathematical Sciences, Tianjin Normal University, Tianjin, China

    Qingjie Cao

Authors
  1. Zaifeng Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hui Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qingjie Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Huizheng Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Boyu Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zaifeng Shi.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shi, Z., Li, H., Cao, Q. et al. An Image Mosaic Method Based on Convolutional Neural Network Semantic Features Extraction. J Sign Process Syst 92, 435–444 (2020). https://doi.org/10.1007/s11265-019-01477-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11265-019-01477-2

Keywords

Search

Navigation