Abstract
In this paper, we study a non-Lipschitz and box-constrained model for both piecewise constant and natural image restoration with Gaussian noise removal. It consists of non-Lipschitz isotropic first-order \(\ell _{p}\) (\(0<p<1\)) and second-order \(\ell _{1}\) as regularization terms to keep edges and overcome staircase effects in smooth regions simultaneously. The model thus combines the advantages of non-Lipschitz and high order regularization, as well as box constraints. Although this model is quite complicated to analyze, we establish a motivating theorem, which induces an iterative support shrinking algorithm with proximal linearization. This algorithm can be easily implemented and is globally convergent. In the convergence analysis, a key step is to prove a lower bound theory for the nonzero entries of the gradient of the iterative sequence. This theory also provides a theoretical guarantee for the edge preserving property of the algorithm. Extensive numerical experiments and comparisons indicate that our method is very effective for both piecewise constant and natural image restoration.
Similar content being viewed by others
References
Attouch, H., Bolte, J., Redont, P., Soubeyran, A.: Proximal alternating minimization and projection methods for nonconvex problems: An approach based on the kurdyka-łojasiewicz inequality. Math. Oper. Res. 35, 438–457 (2008)
Attouch, H., Bolte, J., Svaiter, B.F.: Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized gauss-seidel methods. Math. Program. 137, 91–129 (2013)
Beck, A., Teboulle, M.: Fast gradient-based algorithms for constrained total variation image denoising and deblurring problems. IEEE Trans. Image Process. 18, 2419–2434 (2009)
Bian, W., Chen, X.: Linearly constrained non-lipschitz optimization for image restoration. SIAM J. Imaging Sci. 8, 2294–2322 (2015)
Bolte, J., Sabach, S., Teboulle, M.: Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Math. Program. 146, 459–494 (2014)
Candès, E.J., Romberg, J., Tao, T.: Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52, 489–509 (2006)
Chambolle, A., Lions, P.L.: Image recovery via total variation minimization and related problems. Numer. Math. 76, 167–188 (1997)
Chan, R.H., Tao, M., Yuan, X.: Constrained total variation deblurring models and fast algorithms based on alternating direction method of multipliers. SIAM J. Imaging Sci. 6, 680–697 (2013)
Chan, T.F., Esedoglu, S., Park, F.: A fourth order dual method for staircase reduction in texture extraction and image restoration problems, in ICIP, pp. 4137–4140 (2010)
Chan, T.F., Marquina, A., Mulet, P.: High-order total variation-based image restoration. SIAM J. Sci. Comput. 22, 503–516 (2000)
Chartrand, R., Yin, W.: Iteratively reweighted algorithms for compressive sensing. In: ICASSP, pp. 3869–3872 (2008)
Chen, X., Ng, M.K., Zhang, C.: Non-lipschitz \(\ell _{p}\)-regularization and box constrained model for image restoration. IEEE Trans. Image Process. 21, 4709–4721 (2012)
Chen, X., Niu, L., Yuan, Y.: Optimality conditions and a smoothing trust region newton method for nonlipschitz optimization. SIAM J. Optim. 23, 1528–1552 (2013)
Chen, X., Xu, F., Ye, Y.: Lower bound theory of nonzero entries in solutions of \(\ell _{2}\)-\(\ell _{p}\) minimization. SIAM J. Sci. Comput. 32, 2832–2852 (2010)
Chen, X., Zhou, W.: Smoothing nonlinear conjugate gradient method for image restoration using nonsmooth nonconvex minimization. SIAM J. Imaging Sci. 3, 765–790 (2010)
Chen, X., Zhou, W.: Convergence of the reweighted \(\ell _{1}\) minimization algorithm for \(\ell _{2}\)-\(\ell _{p}\) minimization. Comput. Optim. Appl. 59, 47–61 (2014)
Feng, X., Wu, C., Zeng, C.: On the local and global minimizers of \(\ell _0\) gradient regularized model with box constraints for image restoration. Inverse Probl. 34, 1–34 (2018)
Foucart, S., Lai, M.J.: Sparsest solutions of underdetermined linear systems via \(\ell _{q}\)-minimization for \(0<q<1\). Appl. Comput. Harmon. Anal. 26, 395–407 (2009)
Grasmair, M., Lenzen, F.: Anisotropic total variation filtering. Appli. Math. Optim. 62, 323–339 (2010)
Hinterberger, W., Scherzer, O.: Variational methods on the space of functions of bounded hessian for convexification and denoising. Computing 76, 109–133 (2006)
Hintermüller, M., Wu, T.: Nonconvex \(tv^{q}\)-models in image restoration: Analysis and a trust-region regularization based superlinearly convergent solver. SIAM J. Imaging Sci. 6, 1385–1415 (2013)
Lanza, A., Morigi, S., Sgallari, F.: Constrained tv\(_p\)-\(\ell _2\) model for image restoration. J. Sci. Comput. 68, 64–91 (2016)
Lysaker, M., Lundervold, A., Tai, X.: Noise removal using fourth-order partial differential equation with applications to medical magnetic resonance images in space and time. IEEE Trans. Image Process. 12, 1579–1590 (2003)
Lysaker, M., Tai, X.: Iterative image restoration combining total variation minimization and a second-order functional. Int. J. Comput. Vis. 66, 5–18 (2006)
Micchelli, C.A., Shen, L., Xu, Y.: Proximity algorithms for image models: denoising. Inverse Probl. 27, 1–30 (2011)
Nikolova, M.: Analysis of the recovery of edges in images and signals by minimizing nonconvex regularized least-squares. SIAM J. Multiscale Model. Simul. 4, 960–991 (2005)
Nikolova, M., Ng, M.K., Tam, C.P.: Fast nonconvex nonsmooth minimization methods for image restoration and reconstruction. IEEE Trans. Image Process. 19, 3073–3088 (2010)
Nikolova, M., Ng, M.K., Zhang, S., Ching, W.K.: Efficient reconstruction of piecewise constant images using nonsmooth nonconvex minimization. SIAM J. Imaging Sci. 1, 2–25 (2008)
Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Phys. D, Nonlinear Phenomena 60, 259–268 (1992)
Strong, D., Chan, T.: Edge-preserving and scale-dependent properties of total variation regularization. Inverse Probl. 19, 165–187 (2003)
Wang, Y., Yang, J., Yin, W., Zhang, Y.: A new alternating minimization algorithm for total variation image reconstruction. SIAM J. Imaging Sci. 1, 248–272 (2008)
Wang, Y., Yin, W.: Sparse signal reconstruction via iterative support detection. SIAM J. Imaging Sci. 3, 462–491 (2010)
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13, 600–612 (2004)
Wu, C., Tai, X.: Augmented lagrangian method, dual methods, and split bregman iteration for rof, vectorial tv, and high order models. SIAM J. Imaging Sci. 3, 300–339 (2010)
Xu, Z., Chang, X., Xu, F., Zhang, H.: \(\ell _{\frac{1}{2}}\) regularization: a thresholding representation theory and a fast solver. IEEE Trans. Neural Netw. Learn. Syst. 23, 1013–1027 (2012)
You, Y., Kaveh, M.: Fourth-order partial differential equations for noise removal. IEEE Trans. Image Process. 9, 1723–1730 (2000)
Zeng, C., Jia, R., Wu, C.: An iterative support shrinking algorithm for non-lipschitz optimization in image restoration. J. Math. Imaging Vis. 61, 122–139 (2018)
Zeng, C., Wu, C.: On the edge recovery property of noncovex nonsmooth regularization in image restoration. SIAM J. Numer. Anal. 56, 1168–1182 (2018)
Zeng, C., Wu, C.: On the discontinuity of images recovered by noncovex nonsmooth regularized isotropic models with box constraints. Adv. Comput. Math. 45, 589–610 (2019)
Zhong, H.: Research about image denoising and medical image segmentation based on energy minimization. Master’s thesis, Nankai University (2019)
Acknowledgements
This work was supported by the National Natural Science Foundation of China (NSFC) (Nos. 11871035, 11531013), Recruitment Program of Global Young Expert, Zhejiang Provincial Natural Science Foundation of China (No. LQ20A010007), and Grants RG(R)-RC/17-18/02-MATH, GRF-12300819, NSFC/RGC-N_HKBU214/19.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix
Appendix
1.1 The Proof of Lemma 3.4
Firstly, we introduce more index sets to clearly give our proof.
Definition 7.1
For any \(u \in {\mathscr {U}}\), we define:
- (1)
\(B_{1}(u)=\{i\in \Omega _1(u) \cap B(u): u_{i+1} \in B(u) \ \text {and} \ u_{i+n} \in B(u)\}\).
- (2)
\(L_{1}(u)= \Omega _1(u) \setminus B_{1}(u).\)
- (3)
\(B_{0}(u)=\Omega _0(u)\cap B(u)\).
- (4)
\(I_{0}(u)= \Omega _0(u) \setminus B_{0}(u).\)
Here, the subset \(B_{1}(u)\) of \(\Omega _1(u)\) includes pixels equal to \( l_{1}\) or \( l_{2}\) with its left and right pixels being \( l_{1}\) or \( l_{2}\), while pixels in the subset \(L_{1}(u)\) are in \((l_{1}, l_{2})\). Similarly, the subset \(B_{0}(u)\) of \(\Omega _0(u)\) consists of pixels equal to \( l_{1}\) or \( l_{2}\), where its left and right pixels are the same as itself. In contrast, the subset \(I_{0}(u)\) of \(\Omega _0(u)\) consists of pixels in \((l_{1}, l_{2})\).
A simple example is shown in Fig. 7 to explain these index sets in Definitions 3.2 and 7.1, where we consider the periodic boundary condition. Here, \(l_{1}=0\), \(l_{2}=1\), u is
and \((\Vert d_{i} u\Vert )_{i \in J}\) is \([\sqrt{2}, \frac{\sqrt{13}}{4}, 0, \frac{\sqrt{13}}{4}, \frac{3}{4}, \frac{\sqrt{10}}{4}, \frac{\sqrt{2}}{4}, \frac{3}{4}, 0, \frac{\sqrt{13}}{4}, 0, \frac{3}{4}, \frac{\sqrt{5}}{2}, \frac{\sqrt{5}}{4},\frac{1}{4},\frac{\sqrt{10}}{4}]^{T}.\) Then,
- (I)
\(I(u)=\{3,4,6,7,8,11,12,14,15,16\}, \ B(u)=\{1,2,5,9,10,13\};\)
- (II)
\(\Omega _1(u)=\{1,2,4,5,6,7,8,10,12,13,14,15,16\}, \ B_{1}(u)=\{1\}\),
\(L_{1}(u)=\{2,4,5,6,7,8,10,12,13,14,15,16\};\)
- (III)
\(\Omega _0(u)=\{3,9,11\}, \ B_{0}(u)=\{9\}, \ I_{0}(u)=\{3,11\}.\)
We observe that if \(I(u)=\emptyset \) or \(L_{1}(u)=\emptyset \), then either \(\Vert d_{i} u\Vert =0\) or \(\Vert d_{i} u\Vert =l_{2}-l_{1}\) or \(\Vert d_{i} u\Vert =\sqrt{2}(l_{2}-l_{1})\), for \(\forall i\in J\). Thus, we assume that \(I(u)\ne \emptyset \) and \(L_{1}(u)\ne \emptyset \) in the rest of our paper.
From the Definitions 3.2 and 7.1, we have a lemma similar to [12] as follows:
Lemma 7.2
[12] For a given vector \(u\in {\mathscr {U}}\), let \(I(u),B(u),\Omega _1(u),B_{1}(u),L_{1}(u),\Omega _0(u),B_{0}(u),I_{0}(u)\) be the index sets defined in Definitions 3.2 and 7.1. Then, the following statements hold:
- (a)
\(L_{1}(u)\cap B_{1}(u)=\emptyset \) and \(L_{1}(u)\cup B_{1}(u)=\Omega _1(u)\);
- (b)
\((d^{x}_{i})_{I(u)} = 0\) and \((d^{y}_{i})_{I(u)}=0\), for any \(i\in B_{1}(u)\);
- (c)
\((d^{x}_{i})_{I(u)}\ne 0\) or \((d^{y}_{i})_{I(u)}\ne 0\), for any \(i\in L_{1}(u)\);
- (d)
\(I_{0}(u)\cap B_{0}(u)=\emptyset \) and \(I_{0}(u)\cup B_{0}(u)=\Omega _0(u)\);
- (e)
\((d^{x}_{i})_{I(u)}=0\) and \((d^{y}_{i})_{I(u)}=0\), for any \(i\in B_{0}(u)\);
- (f)
\((d^{x}_{i})_{B(u)}=0\) and \((d^{y}_{i})_{B(u)}=0\), for any \(i\in I_{0}(u)\);
For simplifying symbols, let \(L^{*}_{1}=L_{1}(u^{*}),B^{*}_{1}=B_{1}(u^{*}), I^{*}_{0}=I_{0}(u^{*}),B^{*}_{0}=B_{0}(u^{*})\) and some descriptions in (5) be
If \(i\in I_{0}^{*}\), then \(t^{x,*}_{i}=t^{y,*}_{i}=0\) by Lemma 7.2(f); if \(i\in B_{0}^{*}\), then \(t^{x,*}_{i} = (d^{x}_{i})_{I^{*}}u^{*}_{I^{*}}+(d^{x}_{i})_{B^{*}}u^{*}_{B^{*}}= (d^{x}_{i})u^{*}=0\) and \(t^{y,*}_{i} = (d^{y}_{i})_{I^{*}}u^{*}_{I^{*}}+(d^{y}_{i})_{B^{*}}u^{*}_{B^{*}}= (d^{y}_{i})u^{*}=0\) by Lemma 7.2(e). Since \(I_{0}^{*}\cup B_{0}^{*}=\Omega _0^{*}\) by Lemma 7.2(d), we have
Since \((d_{i})_{I^{*}} z = 0\) for \(\forall i \in B_{0}^{*}\) by Lemma 7.2(e), \((d_{i})_{I^{*}} z = 0\) for all \(i \in \Omega _0^{*}\) when \(z \in \{z\in {\mathbb {R}}^{|I^{*}|}: (d_{i})_{I^{*}}z=0, \ \forall i\in I^{*}_0\}\).
Proof
When \(z^{*}=u^{*}_{I^{*}}\), \(R(z^{*})=F(u^{*})\). Next, we prove that \(z^{*}\) is a local minimizer of (5).
From \(u^{*}\) being a local minimizer of (3), we have \(F(u)> F(u^{*})\) for any \(u\in {\mathscr {U}}\) and \(\Vert u-u^{*}\Vert < \epsilon ^{*}\) with \(\epsilon ^{*}>0.\) If \(z^{*}=u^{*}_{I^{*}}\) is not a local minimizer, then there exists \({\widetilde{z}}\in \{{\widetilde{z}}: l_{1}< {\widetilde{z}}_{i} < l_{2}\}\) satisfying \(\Vert {\widetilde{z}}-z^{*}\Vert < \epsilon ^{*}\) and \((d_{i})_{I^{*}}{\widetilde{z}}=0\), for \(\forall i \in I^{*}_{0}(=\Omega _0^{*}\cap I^{*})\), such that
In particular, we define \({\widetilde{u}}=({\widetilde{z}};u^{*}_{B^{*}})\). Obviously, we have \({\widetilde{u}}\in {\mathscr {U}}\), \(\Vert {\widetilde{u}}-u^{*}\Vert < \epsilon ^{*}\) and thus \(F({\widetilde{u}})> F(u^{*})\). Moreover, the relationship between \(R({\widetilde{z}})\) and \(F({\widetilde{u}})\) is
We observe that \(R({\widetilde{z}})=F({\widetilde{u}})> F(u^{*})=R(z^{*})\), which contradicts (31). Thus, \(z^{*}\) is a local minimizer of the optimization problem (5). \(\square \)
1.2 The Proof of Theorem 3.5
1.2.1 More Definitions and Properties of \(u^{*}\)
The proof uses some important tools and technologies from [39] briefly reviewed here. We first introduce some notations of the matrix repesentation \(U^{*} \in {\mathbb {R}}^{n\times n}\) (\(m=n^2\)) corresponding to \(u^{*}\in {\mathbb {R}}^m\) and then present some definitions and existing results.
Denote \(\mathbf{I }^{*}=\{(i_{x},i_{y}):l_{2}<U^{*}_{i_{x},i_{y}}<l_{1}\}\), \(\mathbf{B }^{*}=\mathbf{J }\setminus \mathbf{I }^{*}\), \(\mathbf{B }_{l_{1}}^{*}=\{(i_{x},i_{y}):U^{*}_{i_{x},i_{y}}=l_{1}\}\), \(\mathbf{B }_{l_{2}}^{*}=\{(i_{x},i_{y}):U^{*}_{i_{x},i_{y}}=l_{2}\}\), \({\mathbf {I}}^{*}_{0}=\{(i_{x},i_{y})\in {\mathbf {I}}^{*}: \sqrt{(D_{x}^{+}U^{*})^{2}_{i_{x},i_{y}}+(D_{y}^{+}U^{*})^{2}_{i_{x},i_{y}}}= 0\}\) and \(\mathbf {\Omega }_1^{K} = \left\{ (i_{x},i_{y})\in {\mathbf {J}}: \sqrt{(D_{x}^{+}U^{*})^{2}_{i_{x},i_{y}}+(D_{y}^{+}U^{*})^{2}_{i_{x},i_{y}}} \ne 0\right\} .\) Similar to the one-to-one correspondence between J and \({\mathbf {J}}\), i.e., \(J \rightarrow {\mathbf {J}}\): \(i\mapsto (i_x,i_y)\), we have one-to-one correspondences for \(I^{*} \rightarrow {\mathbf {I}}^{*}\), \(B^{*} \rightarrow {\mathbf {B}}^{*}\), \(I_{0}^{*} \rightarrow {\mathbf {I}}_{0}^{*}\) and \(\Omega _1^{K} \rightarrow \mathbf {\Omega }_1^{K}\).
For a pixel \((i_{x},i_{y})\), let \({\mathcal {N}}_{1}(i_{x},i_{y})=\{(i_{x},i_{y}),(i_{x},i_{y}+1),(i_{x}+1,i_{y})\}\) represent its 1-neighborhood, and two edges in \({\mathcal {N}}_{1}(i_{x},i_{y})\) are given by \( e^{x}_{i_{x},i_{y}}=\big ((i_{x},i_{y}),(i_{x},i_{y}+1)\big ),\ e^{y}_{i_{x},i_{y}}=\big ((i_{x},i_{y}),(i_{x}+1,i_{y})\big ). \) We also define \(e(i_{x},i_{y})=\{e^{x}_{i_{x},i_{y}}, e^{y}_{i_{x},i_{y}}\}\), and \(e(\hat{\mathbf{J }})=\bigcup _{(i_{x},i_{y})\in \hat{\mathbf{J }}} e(i_{x},i_{y})\), where \(\hat{\mathbf{J }}\) is a subset of \(\mathbf{J }\). Moreover, the 1-neighborhood of a set \(\hat{\mathbf{J }}\) is \( {\mathcal {N}}_{1}(\hat{\mathbf{J }})= \{(i_{x},i_{y})\in \mathbf{J }: (i_{x},i_{y})\in {\mathcal {N}}_{1}({\hat{i}}_{x},{\hat{i}}_{y}) \ \text {for some} \ ({\hat{i}}_{x},{\hat{i}}_{y})\in \hat{\mathbf{J }}\}. \)
For two pixels \(((i_{s+1})_{x},(i_{s+1})_{y})\) and \(((i_{s+t})_{x},(i_{s+t})_{y})\), if there exist a sequence of pixels
such that \(((i_{s+l})_{x},(i_{s+l})_{y})\) and \(((i_{s+l+1})_{x},(i_{s+l+1})_{y})\) being adjacent pixels for \(1 \le l \le t-1\), then these pixels form a path connecting \(((i_{s+1})_{x},(i_{s+1})_{y})\) and \(((i_{s+t})_{x},(i_{s+t})_{y})\).
Definition 7.3
For a subset \(\hat{{\mathbf {J}}}\subseteq {\mathbf {J}}\), if for any two pixels \(((i_{s+1})_{x},(i_{s+1})_{y})\) and \(((i_{s+t})_{x},(i_{s+t})_{y})\) of \({\mathcal {N}}_{1}(\hat{{\mathbf {J}}})\), there exists a path \(P=\{((i_{s+1})_{x},(i_{s+1})_{y}),\cdots ,(i_{s+l})_{x},(i_{s+l})_{y}),\cdots ,((i_{s+t})_{x},(i_{s+t})_{y})\}\) such that
- (1)
\(((i_{s+l})_{x},(i_{s+l})_{y}) \in {\mathcal {N}}_{1}(\hat{{\mathbf {J}}}), \ \text {for} \ 1 \le l \le t\),
- (2)
\(\big (((i_{s+l})_{x},(i_{s+l})_{y}), ((i_{s+l+1})_{x},(i_{s+l+1})_{y})\big ) \in e(\hat{{\mathbf {J}}}), \ \text {for}\ 1 \le l \le t-1\),
connecting them, then we call \(\hat{{\mathbf {J}}}\) is a connected set and P is a path of \(\hat{{\mathbf {J}}}\).
Clearly, for any two connected subsets \(\hat{\mathbf{J }}_{a}\) and \(\hat{\mathbf{J }}_{b}\), if \(\hat{\mathbf{J }}_{a}\cup \hat{\mathbf{J }}_{b}\) is not connected, then \({\mathcal {N}}_{1}(\hat{\mathbf{J }}_{a})\cap {\mathcal {N}}_{1}(\hat{\mathbf{J }}_{a})=\emptyset \). Moreover, a subset \(\hat{\mathbf{J }}\) can be divided into several connected subsets \(\hat{\mathbf{J }}_{1}, \cdots , \hat{\mathbf{J }}_{k}\) satisfying \({\mathcal {N}}_{1}(\hat{\mathbf{J }}_{i})\cap {\mathcal {N}}_{1}(\hat{\mathbf{J }}_{j})=\emptyset \), when \(i\ne j\), and every subset \(\hat{\mathbf{J }}_{i}\) is called a connected component of \(\hat{\mathbf{J }}\).
Lemma 7.4
[39] For a connected set \(\hat{{\mathbf {J}}} \in {\mathbf {J}}\) and any \((i_{x},i_{y}) \in \hat{{\mathbf {J}}}\), suppose that \(|(D_{x}^{+}U)_{i_{x},i_{y}}| \le C\) and \(|(D_{y}^{+}U)_{i_{x},i_{y}}| \le C\) with C being a given constant, and \(U_{(i_{s})_{x},(i_{s})_{y}}=0\) for a given pixel \(((i_{s})_{x},(i_{s})_{y}) \in {\mathcal {N}}_{1}(\hat{{\mathbf {J}}})\). Then, we have
Definition 7.5
[39] For an edge \(\big (((i_{s})_{x},(i_{s})_{y}),((i_{t})_{x},(i_{t})_{y})\big )\), if \(|U^{*}_{(i_{s})_{x},(i_{s})_{y}}-U^{*}_{(i_{t})_{x},(i_{t})_{y}}|> \frac{|l_{2}-l_{1}|}{m}\), then \(\big (((i_{s})_{x},(i_{s})_{y}),((i_{t})_{x},(i_{t})_{y})\big )\) is called a leaping edge. Furthermore, if \((i_{s})_{x}+(i_{s})_{y}<(i_{t})_{x}+(i_{t})_{y}\), then \(((i_{s})_{x},(i_{s})_{y})\) is called a leaping pixel; if else, \(((i_{t})_{x},(i_{t})_{y})\) is called a leaping pixel.
This definition of leaping pixel is important in the proofs of Theorems 3.5 and 4.3.
Define \({\mathbf {L}}^{*}=\{(i_{x},i_{y})\in {\mathbf {J}}: (i_{x},i_{y}) \ \text {is a leaping pixel}\}\) as a set consisting all leaping pixels. We have the following lemma for the connected subset \({\mathbf {J}}^{*}_{a}\) of \(({\mathbf {I}}^{*}_{0} \cup \mathbf {\Omega }_1^{K})\setminus {\mathbf {L}}^{*}\).
Lemma 7.6
[39] For a connected subset \({\mathbf {J}}^{*}_{a}\) of \(({\mathbf {I}}^{*}_{0} \cup \mathbf {\Omega }_1^{K})\setminus {\mathbf {L}}^{*}\), we have \({\mathcal {N}}_{1}({\mathbf {J}}^{*}_{a})\cap {\mathbf {B}}^{*} = \emptyset \), or \({\mathcal {N}}_{1}({\mathbf {J}}^{*}_{a})\cap {\mathbf {B}}^{*} \subseteq \mathbf{B }_{l_{1}}^{*}\), or \({\mathcal {N}}_{1}({\mathbf {J}}^{*}_{a})\cap {\mathbf {B}}^{*} \subseteq \mathbf{B }_{l_{2}}^{*}\).
Note that, we similarly have one-to-one correspondences for \(L^{*} \rightarrow {\mathbf {L}}^{*}\) and \({\mathcal {N}}_{1}(J^{*}_{a}) \rightarrow {\mathcal {N}}_{1}({\mathbf {J}}^{*}_{a})\).
1.2.2 The proof
Proof
By the first-order necessary condition for the local minimizer \(z^{*}\) of (5), we have
where
In detail, we compute \(\left\langle \partial R(z^*), {\widehat{z}} \right\rangle \) as
That is,
where \({\widehat{\delta }}>0\) is a constant independent on \(u^{*}\). The reason is that, for the given \({\widehat{u}}\), we can choose its neighbourhood including \(u^*\) such that \(4\sqrt{6}m\alpha _{1}+\alpha _{2}\sqrt{F(u^*)}\Vert H_{I^{*}}\Vert <{\widehat{\delta }}\).
Moreover, we have
By analyzing (33), we prove \(\Omega _{1}^{*}=\Omega _{1}(u^*) \subseteq \Omega _1({\widehat{u}})\).
According to the definitions and properties of \(u^{*}\) and its matrix representation \(U^{*}\) in 7.2.1, we know that \(\Vert d_{i}u^{*}\Vert > \frac{l_{2}-l_{1}}{m}\) for \(\forall i \in L^{*}\). Accordingly, since \(u^*\) is very near to \({\widehat{u}}\), we can find a constant \(\vartheta >0\) such that \(\Vert d_{i}{\widehat{u}}\Vert >\vartheta \) for \(\forall i \in L^{*}\). Clearly, \(L^{*} \subseteq \Omega _1({\widehat{u}})\).
Next, we verify \((\Omega _{1}^{*}\setminus L^{*})\subseteq \Omega _1({\widehat{u}})\) by induction. From \(B_{1}^{*} \subseteq L^{*}\), we obtain \(\Omega _{1}^{*}\setminus L^{*}=L_1^{*}\setminus L^{*}\). Then, we just show \(L_1^{*}\setminus L^{*}\subseteq \Omega _1({\widehat{u}})\) by induction.
For all \(i \in L_1^{*}\setminus L^{*}\), we sort the values of \(\Vert d_{i}u^{*}\Vert \) as
where \(\varpi ^{*}_{1}>\varpi ^{*}_{2}>\cdots >\varpi ^{*}_{r}\) and \(r\le \#(L_1^{*}\setminus L^{*})<m\). Define \(E^{*}_{j}=\{i\in L_1^{*}\setminus L^{*}: \Vert d_{i}u^{*}\Vert =\varpi ^{*}_{j}\}\).
First, we select a \({z}^{1}\) such that
To easily find a solution of (34), we construct the following problem
Once (35) is solved, (34) will be solved by setting \({z}^{1}=({u}^{1})_{I^{*}}\). Suppose all connected subsets of \((I^{*}_{0}\cup \Omega _{1}^{*})\setminus L^{*}\) are \(J_{1},J_{2},\cdots ,J_{l}\). The problem (35) is further equivalent to
where \(1\le j \le l\).
Similar to [39], since either \({\mathcal {N}}_{1}(J_{j})\cap B^{*}=\emptyset \) or \({\mathcal {N}}_{1}(J_{j})\cap B^{*}\subseteq B_{l_{1}}^{*}= \{i\in J: u_{i} = l_{1}\}\) or \({\mathcal {N}}_{1}(J_{j})\cap B^{*}\subseteq B_{l_{2}}^{*}= \{i\in J: u_{i} = l_{2}\}\) from Lemma 7.6, a solution of (36) is given by
where \(c_{j}={u}^{*}_{m_{j}}\) with \(m_{j}=\mathrm {min}({\mathcal {N}}_{1}(J_{j}))\). Clearly, (36) is solvable and so is (35).
Due to \(\frac{|d^{x}_{i}u^{*}|}{\varpi ^{*}_{1}} \le 1\) and \(\frac{|d^{y}_{i}u^{*}|}{\varpi ^{*}_{1}} \le 1\), we can obtain \(|{u}^{1}_{i}| < m\) and \(\Vert {u}^{1}\Vert ^{2}<\frac{m(m+1)(2m+1)}{6}\) by Lemma 7.4. Denote \(\Gamma =\frac{m(m+1)(2m+1)}{6}\). Accordingly, we have \(\Vert {z}^{1}\Vert ^{2}<\Gamma \ \text {and} \ |{z}^{1}_{i}| < m, \forall i \in I^{*}.\) Moreover, \(|(d^{x}_{i})_{I^{*}}{z}^{1}|<2m\) and \(|(d^{y}_{i})_{I^{*}}{z}^{1}|<2m\) for \(\forall i \in J\). Thus, \( \Vert (d_{i})_{I^{*}}{z}^{1}\Vert ^2=|(d^{x}_{i})_{I^{*}}{z}^{1}|^2+|(d^{y}_{i})_{I^{*}}{z}^{1}|^2<8m^2,\ \forall i \in J. \) By (34), it yields
When choosing \({\widehat{z}}={z}^{1}\) in (33), we obtain
Then,
Based on (38), we prove \(E^{*}_{1} \subseteq \Omega _1({\widehat{u}})\) by contradiction. If there exists \(j\in E^{*}_{1}\) such that \(\Vert d_{j}{\widehat{u}}\Vert = 0\), then \(\Vert d_{j}u^*\Vert \) can be very close to \(\Vert d_{j}{\widehat{u}}\Vert =0\) because of \(u^*\) being very near to \({\widehat{u}}\). From \(\varphi _1'(t)|_{0+}=+\infty \) in Remark 3.1(a), we know that the Eq. (38) is impossible to be true. Thus, \(\Vert d_{j}{\widehat{u}}\Vert \ne 0\), i.e., \(j\in \Omega _1({\widehat{u}})\), which implies \(E^{*}_{1} \subseteq \Omega _1({\widehat{u}})\).
Now, we assume that \(E^{*}_{j}\subseteq \Omega _1({\widehat{u}})\) for any \(j\in \{1,2,\ldots , s-1\}\) with \(1<s<r\). Next, we prove that \(E^{*}_{s}\subseteq \Omega _1({\widehat{u}})\).
Let \(E^{*}=(\cup _{j=1}^{s-1}E^{*}_{j})\cup L^{*}\). Similarly, we want to select a \({z}^{s}\) such that
As the discussion of \({z}^{1}\), (39) has a solution \({z}^{s}\) satisfying
Choosing \({\widehat{z}}={z}^{s}\) in (33), we have
From the inductive hypothesis of \(E^{*}_{j}\subseteq \Omega _1({\widehat{u}})\) for any \(j\in \{1,\cdots , s-1\}\) and \(\Vert d_{i}u^{*}\Vert \) being close to \(\Vert d_{i}{\hat{u}}\Vert \), \(\Vert d_{i}u^{*}\Vert \) is bounded for \(\forall i \in E^{*}_{j}\). Thus, \(\sum _{i \in E^{*}\cap L_1^{*}}\varphi _1'(\Vert d_{i}u^{*}\Vert )\) is also bouned, and there exists a constant \(\Lambda \) such that \(\sum _{i \in E^{*}\cap L_1^{*}}\varphi _1'(\Vert d_{i}u^{*}\Vert )<\Lambda \). Then, the inequality above is respresented as
From (40), we also use contradiction to prove \(E^{*}_{s} \subseteq \Omega _1({\widehat{u}})\) as proving \(E^{*}_{1} \subseteq \Omega _1({\widehat{u}})\).
To sum up, from all \(E^{*}_{j} \subseteq \Omega _1({\widehat{u}})\) for \(\forall j \in \{1,2,\cdots ,r\}\), then \(L_1^{*}\setminus L^{*}=\Omega _1^{*}\setminus L^{*} \subseteq \Omega _1({\widehat{u}})\). In addition, we have shown \(L^{*} \subseteq \Omega _1({\widehat{u}})\). Thus, \(\Omega _1^{*}=\Omega _1(u^{*}) \subseteq \Omega _1({\widehat{u}})\), which completes the whole proof. \(\square \)
Rights and permissions
About this article
Cite this article
Wang, W., Wu, C. & Tai, XC. A Globally Convergent Algorithm for a Constrained Non-Lipschitz Image Restoration Model. J Sci Comput 83, 14 (2020). https://doi.org/10.1007/s10915-020-01190-4
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10915-020-01190-4
Keywords
- Non-Lipschitz optimization
- Box-constrained
- High order regularization
- Support shrinking
- Lower bound theory
- Kurdyka–Łojasiewicz property
- Image restoration