Skip to main content
SpringerLink
Log in

A new SURF-based algorithm for robust registration of multimodal images data

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

In this paper, we propose an original algorithm allowing the registration of multimodal images. Indeed, the large and nonlinear intensity changes between different modalities often lead to mismatching of the interest points, which can alter the quality of the registration. Our method builds on the SURF for the identification and the description of the interest points. In order to boost the confidence on the subsequent matching operation of the registration, an applicable classification on both the distance and orientation SURF features is presented despite the low structural similarity of the multimodal acquisition. The proposed algorithm is tested and validated on a large dataset extracted from acquisitions on cultural heritage wall paintings that implement 4 imaging modalities covering UV, IR, visible and fluorescence. The comparison with conventional SIFT/SURF approaches, PIIFD, SURF-PIIFD-PRM,CFOG and HTCPR for registration highlights the effectiveness of the method and its robustness to outliers in SIFT/SURF descriptors.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Cui, Ming and Wonka, Peter and Razdan, Anshuman and Hu, Jiuxiang, A new image registration scheme based on curvature scale space curve matching, The Visual Computer, 23, 607–618(2007)

    Article  Google Scholar 

  2. Rohith, G., Kumar, L.S.: Paradigm shifts in super-resolution techniques for remote sensing applications, The Visual Computer, 37, 1965-2008 (2021)

    Article  Google Scholar 

  3. Ye, Yuanxin and Bruzzone, Lorenzo and Shan, Jie and Bovolo, Francesca and Zhu, Qing, Fast and robust matching for multimodal remote sensing image registration, IEEE Transactions on Geoscience and Remote Sensing, 57, 9059–9070(2019)

    Article  Google Scholar 

  4. Hu, Y., Modat, M., Gibson, E., Li, W., Ghavami, N., Bonmati, E., Wang, G., Bandula, S., Moore, C.M., Emberton, M. and Ourselin, S., : Weakly-supervised convolutional neural networks for multimodal image registration, Medical image analysis, 49, 1-13 (2018)

    Article  Google Scholar 

  5. Rengarajan, V., Rajagopalan, A.N., Aravind, R., Seetharaman, G.: Image registration and change detection under rolling shutter motion blur, IEEE transactions on pattern analysis and machine intelligence, 39, 959-1972(2016)

    Google Scholar 

  6. Ouerghi, H., Mourali, O., Zagrouba, E.: Glioma classification via MR images radiomics analysis. Vis. Comput. pp. 1-15 (2021)

  7. Chen, Bailiang and Lambrou, Tryphon and Offiah, Amaka and Fry, Martin and Todd-Pokropek, Andrew, Combined MR imaging towards subject-specific knee contact analysis, The Visual Computer, 27, 121–128(2011)

    Article  Google Scholar 

  8. Brock, K. K., Mutic, S., McNutt, T. R., Li, H., & Kessler, M. L. : Use of image registration and fusion algorithms and techniques in radiotherapy: Report of the AAPM Radiation Therapy Committee Task Group No 132, Medical physics, 44, 43-76(2017)

  9. Renbo Xia, Jibin Zhao and Yunpeng Liu, A robust feature-based registration method of multimodal image using phase congruency and coherent point drift, SPIE, 8919, 1–8(2013)

    Google Scholar 

  10. Liu, Y.-J., Yuen, M.M.-F., Xiong, S.: A feature-based approach for individualized human head modeling, The Visual Computer, 18, 368-381(2002)

    Article  Google Scholar 

  11. Rominger, Cédric and Martin, Arnaud, Recalage et fusion d’images sonar multivues: Utilisation du conflit, Revue Nationale des Technologies de l’information, 21, 231–246(2011)

    Google Scholar 

  12. Jingga Zhang, Jiajun Wang, Xiuying Wang and Dagan Feng, Multi-modal image registration with joint structure tensor and local entropy, International Journal of Computer Assisted Radiology and Surgery, 10, 1765–1775(2015)

    Article  Google Scholar 

  13. Cosentino, A.: Identification of pigments by multispectral imaging; a flowchart method. Heritage Science 2, 1–12 (2014)

    Article  Google Scholar 

  14. S. Saleem and R. Sablatnig, A robust SIFT descriptor for multi-spectral images, IEEE Signal Processing Letters, 21, 400–403(2014)

    Article  Google Scholar 

  15. Barrera, F., Lumbreras, F., Sappa, A.D.: Multispectral piecewise planar stereo using Manhattan-world assumption. Pattern Recognition Letter 34, 52–61 (2013)

    Article  Google Scholar 

  16. Roche, A., Malandain, G., Pennec, X.: The correlation ratio as a new similarity measure for multimodal image registration. Proc. Med. Image Comput. Computer-Assist. Interv. pp. 1115–1124 (1998)

  17. F. Maes, A. Collignon, D. Vandermeulen, G. Marchal and P. Suetens, Multimodality image registration by maximization of mutual information, IEEE Trans. Med. Imaging, 16, 187–198(1997)

    Article  Google Scholar 

  18. Roche, A., Malandain, G., Ayache, N.: Unifying Maximum Likelihood Approaches in Medical Image Registration. INRIA 11, 71–80 (2000)

    Google Scholar 

  19. Ulysses, J.N., Conci, A.: Measuring similarity in medical registration 17, 1–4 (2010)

  20. Jian Chen and Jie Tian, Real-time multi-modal rigid registration based on a novel symmetric-SIFT descriptor, Progress in Natural Science, 19, 643–651(2008)

    Article  Google Scholar 

  21. Ahmad, Sahar and Khan, Muhammad Faisal, Multimodal non-rigid image registration based on elastodynamics, The Visual Computer, 34, 21–27(2018)

    Article  Google Scholar 

  22. Xiaoyong S, Li X, Zhang Q, Jiaya J: Multi-modal and multi-spectral registration for natural images, ECCV 2014, 309–324 (2014)

    Google Scholar 

  23. Konstantinos, K., Aristeidis, S., Nikos, P.: Efficient and automated multimodal satellite data registration through MRFs and linear programming. CVPR pp. 329–336 (2014)

  24. Anzid, H., le Goic, G., Mansouri, A., Mammass, D. et al.: Improvement of Multimodal Images Classification Based on DSMT Using Visual Saliency Model Fusion With SVM, International journal of computers and technology, 18, 7418-7430 (2018)

    Article  Google Scholar 

  25. De Silva, T., Hotaling, N., Chew, E. Y., & Cukras, C.: Feature-based retinal image registration for longitudinal analysis of patients with age-related macular degeneration, Med. Imaging 11313, 113–132 (2020)

    Google Scholar 

  26. Patel, M. I., Thakar, V. K., & Shah, S. K.: Image registration of satellite images with varying illumination level using HOG descriptor based SURF, Procedia computer science, 93, 382-388(2016)

  27. Kumawat, A., Panda, S.: A robust edge detection algorithm based on feature-based image registration (FBIR) using improved canny with fuzzy logic (ICWFL). Vis. Comput. pp. 1–22 (2021)

  28. Harris, C., Stephens, M. et al.: A combined corner and edge detector, Alvey vision conference, 15, 10-5244(1988)

    Google Scholar 

  29. Feng, J., Ai, C., An, Z., Zhou, Z., Shi, Y.: A feature detection and matching algorithm based on harris algorithm, 2019 international conference on communications. Inf. Syst. Comput. Eng. (CISCE) pp. 616–621 (2019)

  30. Mistry, S., Patel, A.: Image Stitching using Harris Feature Detection. International Research Journal of Engineering and Technology (IRJET) 3, 2220–2226 (2016)

    Google Scholar 

  31. Chen, J., Tian, J., Lee, N., Zheng, J., Smith, R.: Theodore and Laine, Andrew F, A partial intensity invariant feature descriptor for multimodal retinal image registration, IEEE Transactions on Biomedical Engineering, 57, 1707-1718(2010)

    Article  Google Scholar 

  32. Lv, G., Teng, S.W., Lu, G.: COREG: a corner based registration technique for multimodal images. Multimedia Tools and Applications 77, 12607–12634 (2018)

    Article  Google Scholar 

  33. Das, A., Ghoshal, D.: Human Eye Detection using Modified Image Registration Process based on Corner Detection. International Journal of Pure and Applied Mathematics 118, 1521–1527 (2018)

    Google Scholar 

  34. Misra, I., Moorthi, S. M., Dhar, D., Ramakrishnan, R.: An automatic satellite image registration technique based on Harris corner detection and Random Sample Consensus (RANSAC) outlier rejection model. In 2012 1st International Conference on Recent Advances in Information Technology (RAIT), vol. 1, pp. 68–73 (2012)

  35. Zhuoqian, Y., Xinang, L., Yang, Y.: Harris feature and coherent point drift based remote sensing image registration. In 2017 13th IEEE International Conference on Electronic Measurement and Instruments (ICEMI), vol. 13, pp. 554–558 (2017)

  36. Sheng, H.: Medical Image Registration Method Based on Tensor Voting and Harris Corner Point Detection. Journal of Medical Imaging and Health Informatics 8, 583–589 (2018)

    Article  Google Scholar 

  37. Davies, E.R.: Application of the generalised hough transform to corner detection. IEE Proceedings E (Computers and Digital Techniques) 135, 49–54 (1988)

    Article  Google Scholar 

  38. Wang, G., Xu, H., Zhang, H.: An image registration method based on the combination of multiple image features, In EEE International Geoscience and Remote Sensing Symposium (IGARSS), pp. 2803–2806(2016)

  39. Istenic, R., Heric, D., Ribaric, S., Zazula, D.: Thermal and visual image registration in hough parameter space, In 14th International Workshop on Systems, Signals and Image Processing and 6th EURASIP Conference Focused on Speech and Image Processing, Multimedia Communications and Services, pp. 106–109 (2007)

  40. Fridman, L., Nordberg, V.: Two multimodal image registration approaches for positioning purposes, Master of Science Thesis in Electrical Engineering (2019)

  41. Gourrame, K., Douzi, H., Harba, R., Riad, R., Ros, F., Amar, M., Elhajji, M.: A zero-bit Fourier image watermarking for print-cam process. Multimedia Tools and Applications 78, 2621–2638 (2019)

    Article  Google Scholar 

  42. Lowe, D.G.: Object recognition from local scale-invariant features. In Proceedings of the International Conference on Computer Vision Corfu, vol. 5, pp. 1150–1157 (1999)

  43. Wang, V.T., Hayes, M.P.: Synthetic aperture sonar track registration using SIFT image correspondences, IEEE Journal of Oceanic Engineering, 42, 901-913(2017)

    Article  Google Scholar 

  44. Ma, Wenping and Wen, Zelian and Wu, Yue and Jiao, Licheng and Gong, Maoguo and Zheng, Yafei and Liu, Liang, Remote sensing image registration with modified SIFT and enhanced feature matching, IEEE Geoscience and Remote Sensing Letters, 14, 3–7(2016)

    Article  Google Scholar 

  45. Rister, B., Horowitz, M. A., & Rubin, D. L.: Volumetric image registration from invariant keypoints, IEEE Trans. Image Process. 26, 4900–4910 (2017)

    Article  MathSciNet  Google Scholar 

  46. Chen, S., Li, X., Zhao, L., Chang, C.-I., Xue, B.: An iterative SIFT based on intensity and spatial information for remote sensing image registration. Algorithms, Technologies, and Applications for Multispectral and Hyperspectral Imagery XXV 10986, 109–861 (2019)

    Google Scholar 

  47. Jian Chen and Jie Tian, Real-time multi-modal rigid registration based on a novel symmetric-SIFT descriptor, Progress in Natural Science, 19, 643–651(2009)

    Article  Google Scholar 

  48. Hossain, M.T., Lv, G., Teng, S. W., Lu, G., Lackmann, M.: Improved symmetric-sift for multi-modal image registration. In 2011 International Conference on Digital Image Computing: Techniques and Applications, pp. 197–202. IEEE (2011)

  49. Teng, Shyh Wei and Hossain, Md Tanvir and Lu, Guojun, Multimodal image registration technique based on improved local feature descriptors, Journal of Electronic Imaging, 24, 1–13 (2015)

    Article  Google Scholar 

  50. Paul, S., Durgam, U. K., Pati, U. C.: Multimodal optical image registration using modified SIFT, Progress in Intelligent Computing Techniques: Theory, Practice, and Applications, pp. 123–129 (2018)

  51. Ye, Yuanxin and Shan, Jie and Bruzzone, Lorenzo and Shen, Li, Robust registration of multimodal remote sensing images based on structural similarity, IEEE Transactions on Geoscience and Remote Sensing, 55, 2941–2958(2017)

    Article  Google Scholar 

  52. Ghassabi, Z., Shanbehzadeh, J., Sedaghat, A., Fatemizadeh, E.: An efficient approach for robust multimodal retinal image registration based on UR-SIFT features and PIIFD descriptors. EURASIP Journal on Image and Video Processing 2013, 1–16 (2013)

    Article  Google Scholar 

  53. Herbert Bay, Andreas Ess, Tinne Tuytelaars, and Luc Van Gool, Speeded-up robust features (SURF), Computer vision and image understanding, 110, 346–359(2008)

    Article  Google Scholar 

  54. Firmenichy, D., Brown, M., Susstrunk, S.: Multispectral interest points for RGB-NIR image registration. In Proceedings of IEEE International Conference on Image Processing, vol. 110, pp. 181–1849 (2011)

  55. Zheng, Q., Wang, Q., Ba, X., Liu, S., Nan, J., Zhang, S.: A Medical Image Registration Method Based on Progressive Images. Computational and Mathematical Methods in Medicine, Hindawi 2021, 1–10 (2021)

    Google Scholar 

  56. Zhao, D.: A Rapid Multimodal Image Registration Based on the Local Edge Histogram. Math. Prob. Eng. Hindawi 2021, 1–10 (2021)

    Google Scholar 

  57. Gang Wang and Zhicheng Wang and Yufei Chen and Weidong Zhao, Robust point matching method for multimodal retinal image registration, Biomedical Signal Processing and Control, 19, 68–76(2015)

    Article  Google Scholar 

  58. Wang, X., Liu, X., Chen, Y., Zhou Z.:, A Robust Method for Multimodal Image Registration Based on Vector Field Consensus, Intelligent Computing Methodologies. Lect. Notes Comput. Sci. 1, 128–139 (2017)

    Google Scholar 

  59. Zhao, D., Yang, Y., Ji, Z., Xiaopeng, H.: Rapid multimodality registration based on MM-SURF. Neurocomputing 131, 87–97 (2014)

  60. Pluim, J., Maintz, J., Viergever, M.: Image registration by maximization of combined mutual information and gradient information, In Proceedings of Medical Image Computing and Computer-Assisted Interventation, pp. 103–129 (2000)

  61. Bay, H., Tuytelaars, T., Gool, L.V.: SURF: speeded up robust features. In Proceedings of European Conference on Computer Vision, pp. 404–417 (2006)

  62. Anzid, H., le Goic, G., Bekkarri, A., Mansouri, A., Mammass, D.: Improving point matching on multimodal images using distance and orientation automatic filtering, In 2016 IEEE/ACS 13th International Conference of Computer Systems and Applications (AICCSA), vol. 13, pp. 1–8 (2016)

  63. Fischler, M.A., Bolles, R. C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography, Commun. ACM 24, 381–395 (1981)

    Article  MathSciNet  Google Scholar 

  64. Juan, L., Gwun, O.: A comparison of SIFT, PCA-SIFT and SURF. International Journal of Image Processing (IJIP) 3, 143–152 (2009)

    Google Scholar 

  65. Beckouche, S., Leprince, S., Sabater, N., Ayoub, F.: Robust outliers detection in image point matching, In Computer Vision Workshops (ICCV Workshops), 2011 IEEE International Conference, pp. 180–187 (2011)

  66. Chen, C.H.: Signal and image processing for remote sensing, vol. 2007, pp. 515–537. CRC Press, Boca Raton (2007)

  67. Alred, G.J., Brusaw, C.T., Oliu, W.E.: Handbook of Technical Writing, 7th. St. Martin’s, New York (2003)

  68. Gull, S.F.: Maximum Entropy and Bayesian Methods. vol. 36, pp. 53–71. Kluwer Academic, Dordrecht (1989)

  69. Hanson, K.M.: Introduction to Bayesian image analysis. Medical Imaging: Image Processing 1898, 716–731 (1993)

    Google Scholar 

Download references

Acknowledgements

The authors thank the Château de Germolles managers for providing data and expertise and the COST Action TD1201 “Colour and Space in Cultural Heritage (COSCH)” (www.cosch.info) for supporting this case study. The authors also thank the PHC Toubkal/16/31: 34676YA program for the financial support

Author information

Authors and Affiliations

  1. IRF-SIC Laboratory, Faculty of Sciences, Ibn Zohr University, Agadir, Morocco

    Hanan Anzid, Aissam Bekkari & Driss Mammass

  2. ImVia Laboratory, Imagery and Artificial Vision, University of Bourgogne-franche-comt, Besanon, France

    Gaetan le Goic & Alamin Mansouri

Authors
  1. Hanan Anzid
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gaetan le Goic
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Aissam Bekkari
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alamin Mansouri
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Driss Mammass
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Anzid, H., le Goic, G., Bekkari, A. et al. A new SURF-based algorithm for robust registration of multimodal images data. Vis Comput 39, 1667–1681 (2023). https://doi.org/10.1007/s00371-022-02435-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-022-02435-z

Keywords

Search

Navigation