Skip to main content
SpringerLink
Log in

Optimal Selection of the Regularization Function in a Weighted Total Variation Model. Part I: Modelling and Theory

  • Published:
Journal of Mathematical Imaging and Vision Aims and scope Submit manuscript

Abstract

A weighted total variation model with a spatially varying regularization weight is considered. Existence of a solution is shown, and the associated Fenchel predual problem is derived. For automatically selecting the regularization function, a bilevel optimization framework is proposed. In this context, the lower-level problem, which is parameterized by the regularization weight, is the Fenchel predual of the weighted total variation model and the upper-level objective penalizes violations of a variance corridor. The latter object relies on a localization of the image residual as well as on lower and upper bounds inspired by the statistics of the extremes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Acar, R., Vogel, C.: Analysis of bounded variation penalty methods for ill-posed problems. Inverse Probl. 10, 1217–1229 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  2. Adams, R.A., Fournier, J.J.F.: Sobolev Spaces, 2nd edn. Academic Press, London (2003)

    MATH  Google Scholar 

  3. Almansa, A., Ballester, C., Caselles, V., Haro, G.: A TV based restoration model with local constraints. J. Sci. Comput. 34(3), 209–236 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  4. Ambrosio, L., Fusco, N., Pallara, D.: Functions of Bounded Variation and Free Discontinuity Problems. Oxford Mathematical Monographs. Oxford University Press, New York (2000)

    MATH  Google Scholar 

  5. Appell, J., Zabrejko, P.P.: Nonlinear Superposition Operators. Cambridge University Press, Cambridge (1990)

    Book  MATH  Google Scholar 

  6. Athavale, P., Jerrard, R., Novaga, M., Orlandi, G.: Weighted TV minimization and applications to vortex density models. Technical report, University of Pisa, Department of Mathematics (2015)

  7. Attouch, H., Buttazzo, G., Michaille, G.: Variational Analysis in Sobolev and BV Spaces, Applications to PDEs and optimization, MPS/SIAM series on optimization, vol. 6. Society for Industrial and Applied Mathematics (SIAM), Mathematical Programming Society (MPS), Philadelphia, PA (2006)

  8. Aubert, G., Kornprobst, P.: Mathematical Problems in Image Processing: Partial Differential Equations and the Calculus of Variations, vol. 147. Springer Science & Business Media, Berlin (2006)

    MATH  Google Scholar 

  9. Barbu, V.: Optimal Control of Variational Inequalities, volume 100 of Res. Notes Math. Pitman, London (1984)

  10. Barbu, V., Precupanu, T.: Convexity and Optimization in Banach Spaces. Springer Monographs in Mathematics, 4th edn. Springer, Dordrecht (2012)

    Book  MATH  Google Scholar 

  11. Bertalmio, M., Caselles, V., Rougé, B., Solé, A.: TV based image restoration with local constraints. J. Sci. Comput. 19, 95–122 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  12. Boccardo, L., Murat, F.: Nouveaux résultats de convergence dans des problèmes unilatéraux. In: Nonlinear Partial Differential Equations and Their Applications. Collège de France Seminar, Vol. II (Paris, 1979/1980), volume 60 of Res. Notes in Math., pp. 64–85, 387–388. Pitman, Boston (1982)

  13. Boccardo, L., Murat, F.: Homogenization of nonlinear unilateral problems. In: Composite Media and Homogenization Theory (Trieste, 1990), volume 5 of Progr. Nonlinear Differential Equations Appl., pp 81–105. Birkhäuser Boston, Boston (1991)

  14. Calatroni, L., Chung, C., Reyes, J.C.D.L., Schönlieb, C.-B., Valkonen, T.: Bilevel approaches for learning of variational imaging models. 05 (2015)

  15. Cao, V.C., De los Reyes, J.C., Schoenlieb, C.B.: Learning optimal spatially-dependent regularization parameters in total variation image restoration. ArXiv e-prints, Mar. (2016)

  16. Chambolle, A.: An algorithm for total variation minimization and application. J. Math. Imaging Vis. 20, 89–97 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  17. Chambolle, A., Lions, P.-L.: Image recovery via total variation minimization and related problems. Numer. Math. 76(2), 167–188 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  18. Chan, T., Golub, G., Mulet, P.: A nonlinear primal-dual method for total variation-based image restoration. SIAM J. Sci. Comput. 20(6), 1964–1977 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  19. Chen, Y.: Learning fast and effective image restoration models. PhD thesis, Graz University of Technology (2014)

  20. Chen, Y., Yu, W., Pock, T.: On learning optimized reaction diffusion processes for effective image restoration. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June (2015)

  21. De los Reyes, J.C., Schönlieb, C.-B., Valkonen, T.: Bilevel parameter learning for higher-order total variation regularisation models. J. Math. Imaging Vis. 57, 1–25 (2016)

    Article  MathSciNet  Google Scholar 

  22. De Los Reyes, J.C., Schönlieb, C.-B., Valkonen, T.: The structure of optimal parameters for image restoration problems. J. Math. Anal. Appl. 434(1), 464–500 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  23. Dong, Y., Hintermüller, M., Rincon-Camacho, M.: Automated regularization parameter selection in multi-scale total variation models for image restoration. J. Math. Imaging Vis. 40(1), 82–104 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  24. Dong, Y., Hintermüller, M., Rincon-Camacho, M.: A multi-scale vectorial l\(^{\tau }\)-tv framework for color image restoration. Int. J. Comput. Vis. 92(3), 296–307 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  25. Dong, Y., Hintermüller, M., Rincon-Camacho, M.M.: Automated regularization parameter selection in multi-scale total variation models for image restoration. J. Math. Imaging Vis. 40(1), 82–104 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  26. Dunford, N., Schwartz, J.T.: Linear operators. I. General theory. With the assistance of Bade, W.G., Bartle, R.G. Pure and Applied Mathematics, Vol. 7. Interscience Publishers, Inc., New York (1958)

  27. Ekeland, I., Temam, R.: Convex Analysis and Variational Problems. American Elsevier, New York (1976)

    MATH  Google Scholar 

  28. Fehrenbach, J., Nikolova, M., Steidl, G., Weiss, P.: Bilevel image denoising using Gaussianity tests. In: Scale space and variational methods in computer vision, volume 9087 of Lecture Notes in Computer Science, pp. 117–128. Springer, Cham (2015)

  29. Frick, K., Marnitz, P., Munk, A.: Statistical multiresolution Dantzig estimation in imaging: fundamental concepts and algorithmic framework. Electr. J. Stat. 6, 231–268 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  30. Girault, V., Raviart, P.-A.: Finite Element Methods for Navier–Stokes Equations: Theory and Algorithms, vol. 5. Springer, Berlin (1986)

    Book  MATH  Google Scholar 

  31. Giusti, E.: Minimal Surfaces and Functions of Bounded Variation. Birkhäuser, Basel (1984)

    Book  MATH  Google Scholar 

  32. Gumbel, E.: Les valeurs extrêmes des distributions statistiques. Ann. Inst. H. Poincaré 5(2), 115–158 (1935)

    MathSciNet  MATH  Google Scholar 

  33. Gumbel, E.J.: Statistics of Extremes. Dover Publications, Inc., Mineola (2004). Reprint of the 1958 original (Columbia University Press, New York; MR0096342)

  34. Hintermüller, M., Kopacka, I.: Mathematical programs with complementarity constraints in function space: \(C\)- and strong stationarity and a path-following algorithm. SIAM J. Optim. 20(2), 868–902 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  35. Hintermüller, M., Kunisch, K.: Total bounded variation regularization as a bilaterally constrained optimization problem. SIAM J. Appl. Math. 64, 1311–1333 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  36. Hintermüller, M., Rautenberg, C.N.: On the density of classes of closed convex sets with pointwise constraints in Sobolev spaces. J. Math. Anal. Appl. 426(1), 585–593 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  37. Hintermüller, M., Rautenberg, C.N., Rösel, S.: Density of Convex Intersections and Applications. WIAS Preprint No. 2333 (2016)

  38. Hintermüller, M., Rautenberg, C.N., Wu, T., Langer, A.: Optimal selection of the regularization function in a generalized total variation model. Part II: Algorithm, its analysis and numerical tests. WIAS Preprint No. 2236 (2016)

  39. Hintermüller, M., Rincon-Camacho, M.M.: Expected absolute value estimators for a spatially adapted regularization parameter choice rule in L1-TV-based image restoration. Inverse Probl. 26(8), 1–30 (2010)

    Article  MATH  Google Scholar 

  40. Hintermüller, M., Stadler, G.: An infeasible primal-dual algorithm for total bounded variation-based inf-convolution-type image restoration. SIAM J. Sci. Comput. 28(1), 1–23 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  41. Hintermüller, M., Surowiec, T.M., Mordukhovich, B.S.: Several approaches for the derivation of stationarity conditions for elliptic MPECs with upper-level control constraints. Math. Program. 146(1–2), 555–582 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  42. Hintermüller, M., Wu, T.: Bilevel optimization for calibrating point spread functions in blind deconvolution. Inverse Probl. Imaging 9(4), 1139–1169 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  43. Hotz, T., Marnitz, P., Stichtenroth, R., Davies, L., Kabluchko, Z., Munk, A.: Locally adaptive image denoising by a statistical multiresolution criterion. Comput. Stat. Data Anal. 56(3), 543–558 (2012)

    MathSciNet  MATH  Google Scholar 

  44. Jalalzai, K.: Regularization of inverse problems in image processing. PhD thesis, Ecole Polytechnique (2012)

  45. Jalalzai, K.: Discontinuities of the minimizers of the weighted or anisotropic total variation for image reconstruction. Technical Report, Cornell University Library (2014). arXiv:1402.0026

  46. Klatzer, T., Pock, T.: Continuous hyper-parameter learning for support vector machines. In: Proceedings of the 20th Computer Vision Winter Workshop, pp. 39–47 (2015)

  47. Kunisch, K., Pock, T.: A bilevel optimization approach for parameter learning in variational models. SIAM J. Imaging Sci. 6(2), 938–983 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  48. Luo, T., Pang, J.-S., Ralph, D.: Mathematical Programs with Equilibrium Constraints. Cambridge University Press, Cambridge (1996)

    Book  MATH  Google Scholar 

  49. Mosco, U.: Convergence of convex sets and solutions of variational inequalities. Adv. Math. 3(4), 510–585 (1969)

    Article  MathSciNet  MATH  Google Scholar 

  50. Ochs, P., Ranftl, R., Brox, T., Pock, T.: Bilevel optimization with nonsmooth lower level problems. In: Scale Space and Variational Methods in Computer Vision, volume 9087 of Lecture Notes in Computer Science, pp. 654–665. Springer, Cham (2015)

  51. Outrata, J., Kocvara, M., Zowe, J.: Nonsmooth approach to optimization problems with equilibrium constraints. In: Nonconvex Optimization and its Applications, vol. 28. Kluwer Academic Publishers, Dordrecht (1998)

  52. Rodrigues, J.F.: Obstacle Problems in Mathematical Physics. Elsevier, North-Holland (1987)

    MATH  Google Scholar 

  53. Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Phys. D Nonlinear Phenom. 60(1–4), 259–268 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  54. Schmidt, U., Roth, S.: Shrinkage fields for effective image restoration. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June (2014)

  55. Schönlieb, C., De Los Reyes, J.C.: Image denoising: learning noise distribution via pde-constrained optimisation. Inverse Probl. Imaging 7(4), 1183–1214 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  56. Showalter, R.E.: Monotone Operators in Banach Space and Nonlinear Partial Differential Equations. American Mathematical Society, Providence (1997)

    MATH  Google Scholar 

  57. Tröltzsch, F.: Optimal control of partial differential equations, volume 112 of Graduate Studies in Mathematics. American Mathematical Society, Providence, RI, 2010. Theory, methods and applications, Translated from the 2005 German original by Jürgen Sprekels

  58. Vogel, C.: Computational methods for inverse problems. In: Frontiers Applied Mathematics, vol. 23. SIAM, Philadelphia (2002)

  59. Zowe, J., Kurcyusz, S.: Regularity and stability for the mathematical programming problem in Banach spaces. Appl. Math. Optim. 5(1), 49–62 (1979)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Weierstrass Institute, Mohrenstrasse 39, 10117, Berlin, Germany

    Michael Hintermüller

  2. Department of Mathematics, Humboldt-University of Berlin, Unter den Linden 6, 10099, Berlin, Germany

    Michael Hintermüller & Carlos N. Rautenberg

Authors
  1. Michael Hintermüller
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carlos N. Rautenberg
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael Hintermüller.

Additional information

This research was carried out in the framework of MATHEON supported by the Einstein Foundation Berlin within the ECMath Projects OT1, SE5 and SE15 as well as by the DFG under Grant No. HI 1466/7-1 “Free Boundary Problems and Level Set Methods”.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hintermüller, M., Rautenberg, C.N. Optimal Selection of the Regularization Function in a Weighted Total Variation Model. Part I: Modelling and Theory. J Math Imaging Vis 59, 498–514 (2017). https://doi.org/10.1007/s10851-017-0744-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10851-017-0744-2

Keywords

Mathematics Subject Classification

Search

Navigation