Abstract
Image blending is an integral part of many multi-image applications such as panorama stitching or remote image acquisition processes. In such scenarios, multiple images are connected at predefined boundaries to form a larger image. A convincing transition between these boundaries may be challenging, since each image might have been acquired under different conditions or even by different devices.
We propose the first blending approach based on osmosis filters. These drift-diffusion processes define an image evolution with a non-trivial steady state. For our blending purposes, we explore several ways to compose drift vector fields based on the derivatives of our input images. These vector fields guide the evolution such that the steady state yields a convincing blended result. Our method benefits from the well-founded theoretical results for osmosis, which include useful invariances under multiplicative changes of the colour values. Experiments on real-world data show that this yields better quality than traditional gradient domain blending, especially under challenging illumination conditions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
References
Burt, P.J., Adelson, E.H.: A multiresolution spline with application to image mosaics. ACM Trans. Graph. 2(4), 217–236 (1983)
d’Autume, M., Morel, J.M., Meinhardt-Llopis, E.: A flexible solution to the osmosis equation for seamless cloning and shadow removal. In: Proceedings of 2018 IEEE International Conference on Image Processing, Athens, Greece, pp. 2147–2151 (2018)
Efros, A., Freeman, W.T.: Image quilting for texture synthesis and transfer. In: Proceedings of 28th Annual Conference on Computer Graphics and Interactive Techniques, Los Angeles, CA, pp. 341–346 (2001)
Fang, F., Wang, T., Fang, Y., Zhang, G.: Fast color blending for seamless image stitching. IEEE Geosci. Remote Sens. Lett. 16(7), 1115–1119 (2019)
Fattal, R., Lischinski, D., Werman, M.: Gradient domain high dynamic range compression. In: Proceedings of SIGGRAPH 2002, San Antonio, TX, pp. 249–256 (2002)
Frankot, R., Chellappa, R.: A method for enforcing integrability in shape from shading algorithms. IEEE Trans. Pattern Anal. Mach. Intell. 10(4), 439–451 (1988)
Georgiev, T.: Covariant derivatives and vision. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3954, pp. 56–69. Springer, Heidelberg (2006). https://doi.org/10.1007/11744085_5
Gracias, N., Mahoor, M., Negahdaripour, S., Gleason, A.: Fast image blending using watersheds and graph cuts. Image Vis. Comput. 27(5), 597–607 (2009)
Hagenburg, K., Breuß, M., Vogel, O., Weickert, J., Welk, M.: A lattice Boltzmann model for rotationally invariant dithering. In: Bebis, G., et al. (eds.) ISVC 2009. LNCS, vol. 5876, pp. 949–959. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-10520-3_91
Hagenburg, K., Breuß, M., Weickert, J., Vogel, O.: Novel schemes for hyperbolic PDEs using osmosis filters from visual computing. In: Bruckstein, A.M., ter Haar Romeny, B.M., Bronstein, A.M., Bronstein, M.M. (eds.) SSVM 2011. LNCS, vol. 6667, pp. 532–543. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-24785-9_45
Illner, R., Neunzert, H.: Relative entropy maximization and directed diffusion equations. Math. Methods Appl. Sci. 16, 545–554 (1993)
Levin, A., Zomet, A., Peleg, S., Weiss, Y.: Seamless image stitching in the gradient domain. In: Pajdla, T., Matas, J. (eds.) ECCV 2004. LNCS, vol. 3024, pp. 377–389. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-24673-2_31
Meister, A.: Numerik linearer Gleichungssysteme, 5th edn. Vieweg, Braunschweig (2015)
Parisotto, S., Calatroni, L., Bugeau, A., Papadakis, N., Schönlieb, C.B.: Variational osmosis for non-linear image fusion. IEEE Trans. Image Process. 29, 5507–5516 (2020)
Parisotto, S., Calatroni, L., Caliari, M., Schönlieb, C.B., Weickert, J.: Anisotropic osmosis filtering for shadow removal in images. Inverse Probl. 35(5), Article 054001 (2019)
Parisotto, S., Calatroni, L., Daffara, C.: Digital cultural heritage imaging via osmosis filtering. In: Mansouri, A., El Moataz, A., Nouboud, F., Mammass, D. (eds.) ICISP 2018. LNCS, vol. 10884, pp. 407–415. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-94211-7_44
Pérez, P., Gagnet, M., Blake, A.: Poisson image editing. ACM Trans. Graph. 22(3), 313–318 (2003)
Risken, H.: The Fokker-Planck Equation. Springer, New York (1984)
Schmidt, M.: Linear scale-spaces in image processing: drift-diffusion and connections to mathematical morphology. Ph.D. thesis, Department of Mathematics, Saarland University, Saarbrücken, Germany (2018)
Sevcenco, I.S., Hampton, P.J., Agathoklis, P.: Seamless stitching of images based on a Haar wavelet 2D integration method. In: Proceedings of 17th International Conference on Digital Signal Processing, Kanoni, Greece (2011)
Sochen, N.A.: Stochastic processes in vision: from Langevin to Beltrami. In: Proceedings of Eighth International Conference on Computer Vision, Vancouver, Canada, vol. 1, pp. 288–293. IEEE Computer Society Press (2001)
Uyttendaele, M., Eden, A., Skeliski, R.: Eliminating ghosting and exposure artifacts in image mosaics. In: Proceedings of 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Kauai, HI, pp. 509–516 (2001)
Vogel, O., Hagenburg, K., Weickert, J., Setzer, S.: A fully discrete theory for linear osmosis filtering. In: Kuijper, A., Bredies, K., Pock, T., Bischof, H. (eds.) SSVM 2013. LNCS, vol. 7893, pp. 368–379. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-38267-3_31
Weickert, J., Hagenburg, K., Breuß, M., Vogel, O.: Linear osmosis models for visual computing. In: Heyden, A., Kahl, F., Olsson, C., Oskarsson, M., Tai, X.-C. (eds.) EMMCVPR 2013. LNCS, vol. 8081, pp. 26–39. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40395-8_3
Wu, H., Zheng, S., Zhang, J., Huang, K.: GP-GAN: towards realistic high-resolution image blending. In: Proceedings of 27th ACM International Conference on Multimedia, Nice, France, pp. 2487–2495 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Bungert, P., Peter, P., Weickert, J. (2023). Image Blending with Osmosis. In: Calatroni, L., Donatelli, M., Morigi, S., Prato, M., Santacesaria, M. (eds) Scale Space and Variational Methods in Computer Vision. SSVM 2023. Lecture Notes in Computer Science, vol 14009. Springer, Cham. https://doi.org/10.1007/978-3-031-31975-4_50
Download citation
DOI: https://doi.org/10.1007/978-3-031-31975-4_50
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-31974-7
Online ISBN: 978-3-031-31975-4
eBook Packages: Computer ScienceComputer Science (R0)