Abstract
Many tasks in image processing and computer vision are modelled by diffusion processes, variational formulations, or constrained optimisation problems. Basic iterative solvers such as explicit schemes, Richardson iterations, or projected gradient descent methods are simple to implement and well-suited for parallel computing. However, their efficiency suffers from severe step size restrictions. As a remedy we introduce a simple and highly efficient acceleration strategy, leading to so-called Fast Semi-Iterative (FSI) schemes that extrapolate the basic solver iteration with the previous iterate. To derive suitable extrapolation parameters, we establish a recursion relation that connects box filtering with an explicit scheme for 1D homogeneous diffusion. FSI schemes avoid the main drawbacks of recent Fast Explicit Diffusion (FED) and Fast Jacobi techniques, and they have an interesting connection to the heavy ball method in optimisation. Our experiments show their benefits for anisotropic diffusion inpainting, nonsmooth regularisation, and Nesterov’s worst case problems for convex and strongly convex optimisation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Acar, R., Vogel, C.R.: Analysis of bounded variation penalty methods for ill-posed problems. Inverse Prob. 10, 1217–1229 (1994)
Bertero, M., Poggio, T.A., Torre, V.: Ill-posed problems in early vision. Proc. IEEE 76(8), 869–889 (1988)
Chambolle, A.: An algorithm for total variation minimization and applications. J. Math. Imaging Vis. 20(1), 89–97 (2004)
Gentzsch, W., Schlüter, A.: Über ein Einschrittverfahren mit zyklischer Schrittweitenänderung zur Lösung parabolischer Differentialgleichungen. Zeitschrift für Angewandte Mathematik und Mechanik 58, T415–T416 (1978). in German
Golub, G.H., Varga, R.S.: Chebyshev semi-iterative methods, successive overrelaxation iterative methods, and second order Richardson iterative methods. Part I. Numerische Mathematik 3(1), 147–156 (1961)
Hellwig, G.: Partial Differential Equations. Teubner, Stuttgart (1977)
van der Houwen, P.J., Sommeijer, B.P.: On the internal stability of explicit, m-stage Runge-Kutta methods for large m-values. Zeitschrift für Angewandte Mathematik und Mechanik 60(10), 479–485 (1980)
Iijima, T.: Basic theory on normalization of pattern (in case of typical one-dimensional pattern). Bull. Electrotech. Lab. 26, 368–388 (1962). in Japanese
Nesterov, Y.: Introductory Lectures on Convex Optimization: A Basic Course, Applied Optimization, vol. 87. Kluwer, Boston (2004)
Ochs, P., Brox, T., Pock, T.: iPiasco: inertial proximal algorithm for strongly convex optimization. J. Math. Imaging Vis. 53(2), 171–181 (2015)
Perona, P., Malik, J.: Scale space and edge detection using anisotropic diffusion. IEEE Trans. Pattern Anal. Mach. Intell. 12(7), 629–639 (1990)
Polyak, B.T.: Some methods of speeding up the convergence of iteration methods. USSR Comput. Math. Math. Phys. 4(5), 1–17 (1964)
Richardson, L.F.: The approximate arithmetical solution by finite differences of physical problems involving differential equation, with an application to the stresses in a masonry dam. Philos. Trans. R. Soc. A 210, 307–357 (1911)
Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60, 259–268 (1992)
Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning internal representations by error propagation, chap. 8. In: Rumelhart, D.E., McClelland, J.L. (eds.) Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol. 1, pp. 318–362. MIT Press, Cambridge (1986)
Saul’yev, V.K.: Integration of Equations of Parabolic Type by the Method of Nets. Elsevier, Pergamon, Oxford (1964)
Setzer, S., Steidl, G., Morgenthaler, J.: A cyclic projected gradient method. Comput. Optim. Appl. 54(2), 417–440 (2013)
Sutskever, I., Martens, J., Dahl, G., Hinton, G.: On the importance of initialization and momentum in deep learning. In: Proceedings of 30th International Conference on Machine Learning, pp. 1139–1147. Atlanta, GA, June 2013
Varga, R.S.: A comparison of the successive overrelaxation method and semi-iterative methods using Chebyshev polynomials. J. Soc. Ind. Appl. Math. 5(2), 39–46 (1957)
Weickert, J.: Theoretical foundations of anisotropic diffusion in image processing. Comput. Suppl. 11, 221–236 (1996)
Weickert, J.: Anisotropic Diffusion in Image Processing. Teubner, Stuttgart (1998)
Weickert, J., Grewenig, S., Schroers, C., Bruhn, A.: Cyclic schemes for PDE-based image analysis. Int. J. Comput. Vis. 118(3), 275–299 (2016)
Wells, W.M.: Efficient synthesis of Gaussian filters by cascaded uniform filters. IEEE Trans. Pattern Anal. Mach. Intell. 8(2), 234–239 (1986)
Young, D.M.: On Richardson’s method for solving linear systems with positive definite matrices. J. Math. Phys. 32(1), 243–255 (1954)
Chzhao-Din, Y.: Some difference schemes for the solution of the first boundary value problem for linear differential equations with partial derivatives. Ph.D. thesis, Moscow State University (1958). in Russian
Acknowledgements
Our research has been partially funded by the Deutsche Forschungsgemeinschaft (DFG) through a Gottfried Wilhelm Leibniz Prize for Joachim Weickert and by the Graduate School of Computer Science at Saarland University. This is gratefully acknowledged.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this paper
Cite this paper
Hafner, D., Ochs, P., Weickert, J., Reißel, M., Grewenig, S. (2016). FSI Schemes: Fast Semi-Iterative Solvers for PDEs and Optimisation Methods. In: Rosenhahn, B., Andres, B. (eds) Pattern Recognition. GCPR 2016. Lecture Notes in Computer Science(), vol 9796. Springer, Cham. https://doi.org/10.1007/978-3-319-45886-1_8
Download citation
DOI: https://doi.org/10.1007/978-3-319-45886-1_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-45885-4
Online ISBN: 978-3-319-45886-1
eBook Packages: Computer ScienceComputer Science (R0)