Skip to main content
Springer Nature Link
Log in

A Multi-modal Registration and Visualization Software Tool for Artworks Using CraquelureNet

  • Conference paper
  • First Online:
Pattern Recognition, Computer Vision, and Image Processing. ICPR 2022 International Workshops and Challenges (ICPR 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13645))

Included in the following conference series:

  • 658 Accesses

Abstract

For art investigations of paintings, multiple imaging technologies, such as visual light photography, infrared reflectography, ultraviolet fluorescence photography, and x-radiography are often used. For a pixel-wise comparison, the multi-modal images have to be registered. We present a registration and visualization software tool, that embeds a convolutional neural network to extract cross-modal features of the crack structures in historical paintings for automatic registration. The graphical user interface processes the user’s input to configure the registration parameters and to interactively adapt the image views with the registered pair and image overlays, such as by individual or synchronized zoom or movements of the views. In the evaluation, we qualitatively and quantitatively show the effectiveness of our software tool in terms of registration performance and short inference time on multi-modal paintings and its transferability by applying our method to historical prints.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. The Bosch Research and Conservation Project. https://boschproject.org. Accessed 25 May 2022

  2. Baráth, D., Matas, J.: Graph-cut RANSAC. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6733–6741 (2018). https://doi.org/10.1109/CVPR.2018.00704

  3. Baráth, D., Noskova, J., Ivashechkin, M., Matas, J.: MAGSAC++, a fast, reliable and accurate robust estimator. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1301–1309 (2020). https://doi.org/10.1109/CVPR42600.2020.00138

  4. Cappellini, V., Del Mastio, A., De Rosa, A., Piva, A., Pelagotti, A., El Yamani, H.: An automatic registration algorithm for cultural heritage images. In: IEEE International Conference on Image Processing 2005 2, pp. II-566 (2005). https://doi.org/10.1109/ICIP.2005.1530118

  5. Chum, O., Matas, J., Kittler, J.: Locally optimized RANSAC. Pattern Recognit. 236–243 (2003). https://doi.org/10.1007/978-3-540-45243-0_31

  6. Conover, D.M., Delaney, J.K., Loew, M.H.: Automatic registration and mosaicking of technical images of old master paintings. Appl. Phys. A 119(4), 1567–1575 (2015). https://doi.org/10.1007/s00339-015-9140-1

    Article  Google Scholar 

  7. DeTone, D., Malisiewicz, T., Rabinovich, A.: SuperPoint: self-supervised interest point detection and description. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp. 224–236 (2018). https://doi.org/10.1109/CVPRW.2018.00060

  8. Dusmanu, M., et al.: D2-Net: a trainable CNN for joint description and detection of local features. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8084–8093 (2019). https://doi.org/10.1109/CVPR.2019.00828

  9. Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981). https://doi.org/10.1145/358669.358692

    Article  MathSciNet  Google Scholar 

  10. Fransen, B., Temmermans, F., Currie, C.: Imaging techniques and methodologies for acquisition, processing and distribution of multimodal image data from the oeuvre of Jan van Eyck. In: Optics, Photonics and Digital Technologies for Imaging Applications VI, vol. 11353, pp. 68–81 (2020). https://doi.org/10.1117/12.2556260

  11. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60(2), 91–110 (2004). https://doi.org/10.1023/B:VISI.0000029664.99615.94

    Article  Google Scholar 

  12. Murashov, D.: A procedure for automated registration of fine art images in visible and X-ray spectral bands. In: Proceedings of the International Conference on Computer Vision Theory and Applications (VISAPP-2011), pp. 162–167 (2011)

    Google Scholar 

  13. Myronenko, A., Song, X.: Point set registration: coherent point drift. IEEE Trans. Pattern Anal. Mach. Intell. 32(12), 2262–2275 (2010). https://doi.org/10.1109/TPAMI.2010.46

    Article  Google Scholar 

  14. Raguram, R., Chum, O., Pollefeys, M., Matas, J., Frahm, J.M.: USAC: a universal framework for random sample consensus. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 2022–2038 (2013). https://doi.org/10.1109/TPAMI.2012.257

    Article  Google Scholar 

  15. Sindel, A., et al.: A keypoint detection and description network based on the vessel structure for multi-modal retinal image registration. Bildverarbeitung für die Medizin 2022, 57–62 (2022). https://doi.org/10.1007/978-3-658-36932-3_12

    Article  Google Scholar 

  16. Sindel, A., Maier, A., Christlein, V.: A visualization tool for image fusion of artworks. In: 25th International Conference on Cultural Heritage and New Technologies (2020)

    Google Scholar 

  17. Sindel, A., Maier, A., Christlein, V.: CraquelureNet: matching the crack structure in historical paintings for multi-modal image registration. In: IEEE International Conference on Image Processing 2021, pp. 994–998 (2021). https://doi.org/10.1109/ICIP42928.2021.9506071

  18. Stein, D., Fritzsche, K., Nolden, M., Meinzer, H., Wolf, I.: The extensible open-source rigid and affine image registration module of the medical imaging interaction toolkit (MITK). Comput. Methods Programs Biomed. 100(1), 79–86 (2010). https://doi.org/10.1016/j.cmpb.2010.02.008

    Article  Google Scholar 

  19. Zacharopoulos, A., et al.: A method for the registration of spectral images of paintings and its evaluation. J. Cult. Herit. 29, 10–18 (2018). https://doi.org/10.1016/j.culher.2017.07.004

    Article  Google Scholar 

Download references

Acknowledgements

Thanks to Daniel Hess, Oliver Mack, Daniel Görres, Wibke Ottweiler, Germanisches Nationalmuseum (GNM), and Gunnar Heydenreich, Cranach Digital Archive (CDA), and Thomas Klinke, TH Köln, and Amalie Hänsch, FAU Erlangen-Nürnberg for providing image data, and to Leibniz Society for funding the research project “Critical Catalogue of Luther portraits (1519 - 1530)” with grant agreement No. SAW-2018-GNM-3-KKLB, to the European Union’s Horizon 2020 research and innovation programme within the Odeuropa project under grant agreement No. 101004469 for funding this publication, and to NVIDIA for their GPU hardware donation.

Author information

Authors and Affiliations

  1. Pattern Recognition Lab, FAU Erlangen-Nürnberg, Erlangen, Germany

    Aline Sindel, Andreas Maier & Vincent Christlein

Authors
  1. Aline Sindel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andreas Maier
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Vincent Christlein
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Aline Sindel .

Editor information

Editors and Affiliations

  1. York University, Toronto, ON, Canada

    Jean-Jacques Rousseau

  2. University of Ontario Institute of Technology, Oshawa, ON, Canada

    Bill Kapralos

Rights and permissions

Reprints and permissions

Copyright information

© 2023 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Sindel, A., Maier, A., Christlein, V. (2023). A Multi-modal Registration and Visualization Software Tool for Artworks Using CraquelureNet. In: Rousseau, JJ., Kapralos, B. (eds) Pattern Recognition, Computer Vision, and Image Processing. ICPR 2022 International Workshops and Challenges. ICPR 2022. Lecture Notes in Computer Science, vol 13645. Springer, Cham. https://doi.org/10.1007/978-3-031-37731-0_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-37731-0_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-37730-3

  • Online ISBN: 978-3-031-37731-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships