Skip to main content
SpringerLink
Log in

Real-time adaptive visible and infrared image registration based on morphological gradient and C_SIFT

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

Since the visible and infrared images have different imaging mechanisms, the difficulty of image registration has greatly increased. The grayscale difference between visible and infrared images is very disadvantageous for extracting feature points in homogenous region, but they both retain the obvious contour edge in the scene. After using the morphological gradient method, the grayscale edge of visible and infrared images can be obtained and their similarity is greatly improved, and their difference may be seen as the difference in brightness or grayscale. Therefore, we proposed a novel algorithm to realise real-time adaptive registration of visible and infrared images using morphological gradient and C_SIFT. Firstly, the morphological gradient method is used to extract the rough edges of visible and infrared images for aligning their visual features as a single similar type. Secondly, the C_SIFT feature detection operator is used to detect and extract feature points from the extracted edges. The C_SIFT uses the centroid method to describe the main direction of feature points, makes rotation invariance feasible. Finally, to verify the effectiveness of the proposed algorithm, we carried out a series of experiments in eight various scenarios. The experimental results show that the proposed algorithm has achieved good experimental results. The registration of visible and infrared images can be completed quickly by the proposed algorithm, and the registration accuracy is satisfactory.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Brown, L.G.: A survey of image registration techniques. ACM Comput. Surv. 24, 325–376 (1992)

    Article  Google Scholar 

  2. Li, H., Manjunath, B.S., Mitra, S.K.: A contour-based approach to multisensor image registration. IEEE Trans. Image Process. A Publ. IEEE Signal Process. Soc. 4, 320–334 (1995)

    Article  Google Scholar 

  3. Li, H.H., Zhou, Y.T.: Automatic visual/IR image registration. Opt. Eng. 35, 391–400 (1996)

    Article  Google Scholar 

  4. Maes, F., Collignon, A.: Vandermeulen, D., Marchal, G., Suetens, P.: Multimodality image registration by maximization of mutual information. IEEE Trans. Med. Imaging 16, 187–198 (1997)

    Article  Google Scholar 

  5. Studholme, C., Hill, D.L.G., Hawkes, D.J.: An overlap invariant entropy measure of 3D medical image alignment. Pattern Recogn. 32, 71–86 (1999)

    Article  Google Scholar 

  6. Irani, M., Anandan, P.: Robust multi-sensor image alignment. In: International conference on computer vision, pp. 959–966 (1998)

  7. Coiras, E., Santamaria, J., Miravet, C.: Segment-based registration technique for visual-infrared images. Opt. Eng. 39, 202–207 (2000)

    Article  Google Scholar 

  8. Li, T., Chen, Z., Wang, R.: Fuzzy feature matching between infrared image and optical image. Proceedings of the SPIE, vol. 4552, pp 259–264 (2001)

  9. Gonzalez, C.I., Castro, J.R., Martinez, G.E., Melin, P., Castillo, O.: A new approach based on generalized type-2 fuzzy logic for edge detection. In: IFSA World Congress and Nafips Meeting, pp. 424–429 (2013)

  10. Barrenechea, E., Bustince, H., Baets, B.D., Lopez-Molina, C.: Construction of interval-valued fuzzy relations with application to the generation of fuzzy edge images. IEEE Trans. Fuzzy Syst. 19, 819–830 (2011)

    Article  Google Scholar 

  11. Colerhodes, A., Zavorin, I., Moigne, J.L.: Mutual information as a similarity measure for remote sensing image registration. Proc. SPIE 4383, 51–61 (2001)

    Article  Google Scholar 

  12. Kern, J.P., Pattichis, M.S.: Robust multispectral image registration using mutual-information models. IEEE Trans. Geosci. Remote Sens. 45, 1494–1505 (2007)

    Article  Google Scholar 

  13. Lee, J.H., Yong, S.K., Lee, D., Kang, D.G., Ra, J.B.: Robust CCD and IR image registration using gradient-based statistical information. IEEE Signal Process. Lett. 17, 347–350 (2010)

    Article  Google Scholar 

  14. Wang, L., Niu, Z., Wu, C., Xie, R., Huang, H.: A robust multisource image automatic registration system based on the SIFT descriptor. Int. J. Remote Sens. 33, 3850–3869 (2012)

    Article  Google Scholar 

  15. Keller, Y., Averbuch, A.: Multisensor image registration via implicit similarity. IEEE Trans. Pattern Anal. Mach. Intell. 28, 794–801 (2006)

    Article  Google Scholar 

  16. Pluim, J.P.W., Maintz, J.B.A., Viergever, M.A.: Image registration by maximization of combined mutual information and gradient information. IEEE Trans. Med. Imaging 19, 809–814 (2002)

    Article  Google Scholar 

  17. Bilodeau, G.A., Torabi, A., Morin, F.: Visible and infrared image registration using trajectories and composite foreground images. Image Vis. Comput. 29, 41–50 (2011)

    Article  Google Scholar 

  18. Wu, F., Wang, B., Yi, X., Li, M., Hao, J., Qin, H., Zhou, H.: Visible and infrared image registration based on visual salient features. J. Electron. Imaging 24, 053017 (2015)

    Article  Google Scholar 

  19. Yao, J., Goh, K.L.: A refined algorithm for multisensor image registration based on pixel migration. IEEE Trans. Image Process. A Publ. IEEE Signal Process. Soc. 15, 1839–1847 (2006)

    Google Scholar 

  20. Yang, B., Qiuze, Y.U., Liu, Y., Tian, J.: Image matching based on new weighted normalized cross correlation. J. Project. Rockets Missiles Guidance 28, 199–202 (2008)

    Google Scholar 

  21. Tian, T., Mei, X., Yu, Y., Zhang, C., Zhang, X.: Automatic visible and infrared face registration based on silhouette matching and robust transformation estimation. Infrared Phys. Technol. 69, 145–154 (2015)

    Article  Google Scholar 

  22. Yang, M.Y., Qiang, Y., Rosenhahn, B.: A global-to-local framework for infrared and visible image sequence registration. In: 2015 IEEE Winter Conference on Applications of Computer Vision, pp. 381–388. IEEE, Waikoloa, 5–9 January 2015

  23. Dou, J., Qin, Q., Tu, Z., Peng, X., Li, Y.: Infrared and visible image registration based on SIFT and sparse representation. In: Control and decision conference, pp. 5420–5424 (2016)

  24. Liu, G., Zhou, H., Liang, X.G., Wang, M.J.: Image registration algorithm for infrared and visible light based on non-subsampled contourlet transform. Comput. Sci. 43, 313–316 (2016)

    Google Scholar 

  25. Tang, C., Tian, G.Y., Chen, X., Wu, J., Li, K., Meng, H., Tang, C., Tian, G.Y., Chen, X., Wu, J.: Infrared and visible images registration with adaptable local-global feature integration for rail inspection. Infrared Phys. Technol. 87, 31–39 (2017)

    Article  Google Scholar 

  26. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60, 91–110 (2004)

    Article  Google Scholar 

  27. Butt, M.A.: Mathematical morphology and its applications to image and signal processing. Kluwer Academic Publishers, New York (2000)

    Google Scholar 

  28. Marr, D., Hildreth, E.: Theory of edge detection. Proc. R. Soc. Lond. 207, 187–217 (1980)

    Google Scholar 

  29. Witkin, A.P.: Scale-space filtering. In: Proceeding of International Joint Conference Artificial Intelligence Karlsruhe, pp. 1019–1022 (1983)

  30. Calonder, M., Lepetit, V., Strecha, C., Fua, P.: BRIEF: binary robust independent elementary features. In: European Conference on Computer Vision, pp. 778–792 (2010)

  31. Yi, X., Wang, B., Fang, Y., Liu, S.: Registration of infrared and visible images based on the correlation of the edges. In: International Congress on Image and Signal Processing, pp. 990–994 (2014)

Download references

Acknowledgements

This work was supported in part by the Nation Natural Science Foundation of China (Grant no. 61602064) and the Key Project of Sichuan Provincial Department of Education (18ZA0100) and the Research Project of Sichuan Science and Technology Department (2017HH0088, 2018JY0146, 2019YFH0187) and the Young Scholar Leadership Fund of CUIT (J201709).

Author information

Authors and Affiliations

  1. Software Department, Chengdu University of Information Technology, Chengdu, 610225, China

    Qiang Zeng, Jianhua Adu, Jiexin Liu, Jianxing Yang, Yuanping Xu & Mei Gong

Authors
  1. Qiang Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jianhua Adu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jiexin Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jianxing Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yuanping Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Mei Gong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jianhua Adu.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zeng, Q., Adu, J., Liu, J. et al. Real-time adaptive visible and infrared image registration based on morphological gradient and C_SIFT. J Real-Time Image Proc 17, 1103–1115 (2020). https://doi.org/10.1007/s11554-019-00858-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-019-00858-x

Keywords

Search

Navigation