Skip to main content
SpringerLink
Log in

Learning sparse partial differential equations for vector-valued images

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Learning partial differential equations (LPDEs) from training data for particular tasks has been successfully applied to many image processing problems. In this paper, we propose a more effective LPDEs model for vector-valued image tasks. The PDEs are also formulated as a linear combination of fundamental differential invariants, but have several distinctions. First, we simplify the current LPDEs system by omitting a PDE which works as an indicate function in current ones. Second, instead of using \(L_2\)-norm, we use the \(L_1\)-norm to regularize the coefficients with respect to the fundamental differential invariants. Third, as the objective function is not smooth, we resort to the alternating direction method to optimize it. We illustrate the properties of our LPDEs system by several examples in denoising and demosaicking of RGB color images. The experiments demonstrate the advantage of the proposed method over other PDE-based methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. The images are padded with zeros of several pixels width around them, so that the Dirichlet boundary conditions \(u_{m}(x,y,t)=0, v_{m}(x,y,t)=0, (x,y,t)\in \varGamma\), are naturally fulfilled.

  2. The images are padded with zeros of several pixels width around them such that the Dirichlet boundary conditions \(u^c_{m}(x,y,t)=0, (x,y,t)\in \varGamma\), are naturally fulfilled.

References

  1. Arbelaez P, Fowlkes C, Martin D (2013) The Berkeley segmentation dataset [Online]. Available: http://www.eecs.berkeley.edu/Research/Projects/CS/vision/bsds/

  2. Aubert G, Kornprobst P (2006) Mathematical problems in image processing: partial differential equations and the calculus of variations, vol 147. Springer, New York

    MATH  Google Scholar 

  3. Beauchemin SS, Barron JL (1995) The computation of optical flow. ACM Comput Surv (CSUR) 27(3):433–466

    Article  Google Scholar 

  4. Beck A, Teboulle M (2009) A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J Imaging Sci 2(1):183–202

    Article  MathSciNet  MATH  Google Scholar 

  5. Bingsheng H, Xiaoming Y (2013) Linearized alternating direction method of multipliers with Gaussian back substitution for separable convex programming. Numer Algebra Control Opt 22(2):313–340

    MathSciNet  MATH  Google Scholar 

  6. Blomgren P, Chan TF (1998) Color TV: total variation methods for restoration of vector-valued images. IEEE Trans Image Process 7(3):304–309

    Article  Google Scholar 

  7. Boyd S, Parikh N, Chu E, Peleato B, Eckstein J (2011) Distributed optimization and statistical learning via the alternating direction method of multipliers. Found Trends Mach Learn 3(1):1–122

    Article  MATH  Google Scholar 

  8. Chan TF, Esedoglu S (2005) Aspects of total variation regularized \({L}^1\) function approximation. SIAM J Appl Math 65(5):1817–1837

    Article  MathSciNet  MATH  Google Scholar 

  9. Combettes PL, Wajs VR (2005) Signal recovery by proximal forward–backward splitting. Multis Model Simul 4(4):1168–1200

    Article  MathSciNet  MATH  Google Scholar 

  10. Deng W, Yin W (2016) On the global and linear convergence of the generalized alternating direction method of multipliers. J Sci Comput 66(3):1–28

    Article  MathSciNet  MATH  Google Scholar 

  11. Ehrhardt MJ, Arridge SR (2013) Vector-valued image processing by parallel level sets [online]. Available http://www0.cs.ucl.ac.uk/staff/ehrhardt/software.html

  12. Ehrhardt MJ, Arridge SR (2014) Vector-valued image processing by parallel level sets. Image Process IEEE Trans 23(1):9–18

    Article  MathSciNet  MATH  Google Scholar 

  13. Esser E (2009) Applications of Lagrangian-based alternating direction methods and connections to split Bregman. Cam Report

  14. Estrada FJ (2010) Image denoising benchmark [online]. Available http://www.cs.utoronto.ca/strider/Denoise/Benchmark/

  15. Figueiredo MA, Nowak RD, Wright SJ (2007) Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems. IEEE J Select Top Signal Process 1(4):586–597

    Article  Google Scholar 

  16. Goldstein T, Osher S (2009) The split Bregman method for L1-regularized problems. Siam J Imaging Sci 2(2):323–343

    Article  MathSciNet  MATH  Google Scholar 

  17. Gunturk BK, Altunbasak Y, Mersereau RM (2002) Color plane interpolation using alternating projections. IEEE Trans Image Process Publ IEEE Signal Process Soc 11(9):997–1013

    Article  Google Scholar 

  18. Li C, Xu C, Gui C, Fox MD (2005) Level set evolution without re-initialization: a new variational formulation. In: Computer vision and pattern recognition, 2005. CVPR 2005. IEEE Computer Society Conference on. vol 1, pp 430–436. IEEE

  19. Lin Z, Liu R, Su Z (2011) Linearized alternating direction method with adaptive penalty for low-rank representation. Advances in neural information processing systems, pp 612–620

  20. Lin Z, Zhang W, Tang X (2008) Learning partial differential equations for computer vision. Tech. rep., Technical report, Microsoft Research, MSR-TR-2008-189

  21. Lin Z, Zhang W, Tang X (2009) Designing partial differential equations for image processing by combining differential invariants. Tech. rep., Technical report, Microsoft Research, MSR-TR-2009-192

  22. Lions JL (1971) Optimal control of systems governed by partial differential equations. Springer, New York

    Book  MATH  Google Scholar 

  23. Liu R, Cao J, Lin Z, Shan S (2014) Adaptive partial differential equation learning for visual saliency detection. In: CVPR

  24. Liu R, Lin Z, Zhang W, Tang K, Su Z (2010) Learning PDEs for image restoration via optimal control. In: ECCV

  25. Liu R, Lin Z, Zhang W, Tang K, Su Z (2013) Toward designing intelligent PDES for computer vision: an optimal control approach. Image Vis Comput 31(1):43–56

    Article  Google Scholar 

  26. Liu R, Lin Z, Zhang W, Tang K, Su Z (2013) Toward designing intelligent PDEs for computer vision: An optimal control approach. Available http://www.cis.pku.edu.cn/faculty/vision/zlin/zlin.htm

  27. Malioutov DM, Cetin M, Willsky AS (2005) Homotopy continuation for sparse signal representation. In: ICASSP. vol 5, pp v-733. IEEE

  28. Martin B, Andrea CM, Stanley O, Martin R (2013) Level set and PDE based reconstruction methods in imaging. Springer, New York

    Google Scholar 

  29. Osher S, Rudin LI (1990) Feature-oriented image enhancement using shock filters. SIAM J Numer Anal 27(4):919–940

    Article  MATH  Google Scholar 

  30. Perona P, Malik J (1990) Scale-space and edge detection using anisotropic diffusion. Pattern Anal Mach Intell IEEE Trans 12(7):629–639

    Article  Google Scholar 

  31. Rudin LI, Osher S, Fatemi E (1992) Nonlinear total variation based noise removal algorithms. Phys D 60(1):259–268

    Article  MathSciNet  MATH  Google Scholar 

  32. Sapiro G (2006) Geometric partial differential equations and image analysis. Cambridge University Press, Cambridge

    MATH  Google Scholar 

  33. Sochen N, Kimmel R, Malladi R (1998) A general framework for low level vision. IEEE Trans Image Process 7(3):310–318

    Article  MathSciNet  MATH  Google Scholar 

  34. Tao M (2014) Some parallel splitting methods for separable convex programming with the \(o(1/t)\) convergence rate. Pacif J Opt 10(2):359–384

    MathSciNet  MATH  Google Scholar 

  35. Yang J, Yuan X (2011) Linearized augmented Lagrangian and alternating direction methods for nuclear norm minimization. Math Comput 82(281):301–329

    Article  MathSciNet  MATH  Google Scholar 

  36. Zhao Z, Fang C, Lin Z, Wu Y (2015) A robust hybrid method for text detection in natural scenes by learning-based partial differential equations. Neurocomputing 168:23–34

    Article  Google Scholar 

Download references

Acknowledgements

The authors thank the NSFC support (Nos. 61471369 and 61503396) and the open Research Foundation of State Key Laboratory of Astronautic Dynamics (Nos. 2013ADL-DW0101 and 2014ADL-DW0102).

Author information

Authors and Affiliations

  1. State Key Laboratory of Astronautic Dynamics, Xi’an, 710043, Shaanxi, China

    Yuanyuan Jiao & Xiaogang Pan

  2. College of Nine, National University of Defense Technology, Changsha, 410073, China

    Yuanyuan Jiao & Xiaogang Pan

  3. College of Science, National University of Defense Technology, Changsha, 410073, China

    Zhenyu Zhao & Chenping Hou

Authors
  1. Yuanyuan Jiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaogang Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhenyu Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chenping Hou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chenping Hou.

Appendix

Appendix

When each of them gets the minimum, the functional \(J(\mathbf {u}_m,\mathbf {a})\) (8) get the minimum. We simply write \(u^c_m\) as \(\varphi\) and fix \(u^k_m\) \((k\ne c)\) when we derive the formulation next. We divide \(J(\varphi ,\mathbf {a})\) into two parts

$$\begin{aligned} \left\{ \begin{array}{l} L_1(\varphi ) = \frac{1}{2}\int _{\varOmega }\left( \varphi |_{t=1}-O^c_m\right) ^2 {\text { d}}\varOmega , \\ L_2(\varphi ) = \frac{\mu }{2}\Vert D(\varphi ,\mathbf {a})+\frac{Y^c_m}{\mu }\Vert ^2. \end{array} \right. \end{aligned}$$
(20)

For the first part, we have

$$\begin{aligned} L_1(\varphi +\varepsilon \delta \varphi )-L_1(\varphi ) &= \frac{1}{2} \int _{\varOmega }\left[ \left( (\varphi +\varepsilon \delta \varphi )|_{t=1}-O^c_m\right) ^2-(\varphi |_{t=1}-O^c_m)^2\right] {\text { d}}\varOmega \\ &= \varepsilon \int _{\varOmega }\delta \varphi |_{t=1} (\varphi |_{t=1}-O^c_m) {\text { d}}\varOmega +o(\varepsilon ). \end{aligned}$$

For the second part, we first define \(F(\varphi ,\mathbf {a})=\sum _{i=0}^{36}a^c_i (t)\mathbf {inv}_i (\varphi )\), Then, we get

$$\begin{aligned} L_2(\varphi +\varepsilon \delta \varphi )-L_2(\varphi ) &= \frac{\mu }{2}\int _Q\left[ \left( D(\varphi +\varepsilon \delta \varphi ,\mathbf {a})+\frac{Y^c_m}{\mu }\right) ^2-\left( D(\varphi ,\mathbf {a})+\frac{Y^c_m}{\mu }\right) ^2\right] {\text { d}}Q \\ &= \frac{\mu }{2}\int _Q \left[ \left( \frac{\partial \varphi }{\partial t}+\varepsilon \frac{\partial \delta \varphi }{\partial t}-F(\varphi +\varepsilon \delta \varphi ,\mathbf {a})+\frac{Y^c_m}{\mu }\right) ^2\right. \\&\left. -\left( D(\varphi ,\mathbf {a})+\frac{Y^c_m}{\mu }\right) ^2\right] {\text { d}}Q\\ &= \mu \int _Q \left[ \left( \varepsilon \frac{\partial \delta \varphi }{\partial t}-F(\varphi +\varepsilon \delta \varphi ,\mathbf {a})+F(\varphi ,\mathbf {a})\right) \cdot \right. \\&\left. \left( D(\varphi ,\mathbf {a})+\frac{Y^c_m}{\mu }\right) \right] {\text { d}}Q+o(\varepsilon ), \end{aligned}$$

where \(Q=\varOmega \times [0,T]\) and \(\text {d}Q={\text { d}}\varOmega {\text { d}}t\).

We first compute \(F(\varphi +\varepsilon \delta \varphi ,\mathbf {a})-F(\varphi ,\mathbf {a})\), and it equals

$$\begin{aligned} F(\varphi +\varepsilon \delta \varphi ,\mathbf {a})-F(\varphi ,\mathbf {a}) &= \varepsilon \left( \frac{\partial F}{\partial \varphi }(\delta \varphi )+\frac{\partial F}{\partial \varphi _x}\frac{\partial \delta \varphi }{\partial x}+\cdots +\frac{\partial F}{\partial \varphi _{yy}}\frac{\partial ^2 \delta \varphi }{\partial y^2} \right) \\ &= \varepsilon \underset{(p,q)\in \mathcal {P}}{\sum }\sigma _{pq}(\varphi )\frac{\partial ^{p+q} \delta \varphi }{\partial x^p \partial y^q} +o(\varepsilon ). \end{aligned}$$

where

$$\begin{aligned} \left\{ \begin{array}{l} \sigma _{pq}(\varphi ) = \frac{\partial F}{\partial \varphi _{pq}}= \sum \limits _{j=0}^{36}a^c_j(t)\frac{\partial \mathbf {inv}_j(\varphi )}{\partial \varphi _{pq}},\\ \varphi _{pq} = \frac{\partial ^{p+q} \varphi }{\partial x^p \partial y^q} . \end{array} \right. \end{aligned}$$
(21)

From the above formulations, we can get

$$\begin{aligned} \delta L &= L(\varphi +\varepsilon \delta \varphi ,\mathbf {a})-L(\varphi ,\mathbf {a})\\ &= \varepsilon \int _{\varOmega }\delta \varphi |_{t=1} (\varphi |_{t=1}-O^c_m) {\text { d}}\varOmega \\&+ \mu \int _Q \left[ \left( \varepsilon \frac{\partial \delta \varphi }{\partial t}-F(\varphi +\varepsilon \delta \varphi ,\mathbf {a})+F(\varphi ,\mathbf {a})\right) \right. \\&\left. \left( D(\varphi ,\mathbf {a})+\frac{Y^c_m}{\mu }\right) \right] {\text { d}}Q+o(\varepsilon ). \end{aligned}$$

As the perturbation \(\delta \varphi\) should satisfy that \(\delta \varphi |_{\varGamma }=0\) and \(\delta \varphi |_{t=0}=0\). Integrating by parts, we have

$$\begin{aligned} \delta L &= \varepsilon \int _{\varOmega }\delta \varphi |_{t=1} (\varphi |_{t=1}-O^c_m) {\text { d}}\varOmega \\&+ \varepsilon \mu \int _{\varOmega }\delta \varphi \left( D(\varphi ,\mathbf {a})+\frac{Y^c_m}{\mu }\right) \left| _{t=1} {\text { d}}\varOmega \right. \\&- \varepsilon \mu \int _{\varOmega } \delta \varphi \left( D(\varphi ,\mathbf {a})+\frac{Y^c_m}{\mu }\right) \left| _{t=0} {\text { d}}\varOmega \right. \\&-\varepsilon \mu \int _Q \delta \varphi \frac{\partial }{\partial t}\left( D(\varphi ,\mathbf {a})+\frac{Y^c_m}{\mu }\right) \, \text {d}Q\\&-\varepsilon \mu \int _Q \underset{(p,q)\in \mathcal {P}}{\sum }\sigma _{pq}(\varphi )\frac{\partial ^{p+q} \delta \varphi }{\partial x^p \partial y^q} \left( D(\varphi ,\mathbf {a})+\frac{Y^c_m}{\mu }\right) \, \text {d}Q\\ &= \varepsilon \int _{\varOmega }\delta \varphi |_{t=1} (\varphi |_{t=1}-O^c_m) \text {d}\varOmega \\&+ \varepsilon \mu \int _{\varOmega }\delta \varphi \left( D(\varphi ,\mathbf {a})-\frac{Y^c_m}{\mu }\right) \left| _{t=1} {\text { d}}\varOmega \right. \\&-\varepsilon \mu \int _Q \delta u\frac{\partial }{\partial t}\left( D(\varphi ,\mathbf {a})-\frac{Y^c_m}{\mu }\right) \, \text {d}Q\\&-\varepsilon \mu \int _{T} \int _{\varGamma } \underset{(p,q)\in \mathcal {P}}{\sum }\sigma _{pq}(\varphi ) \delta \varphi \left( D(\varphi ,\mathbf {a})+\frac{Y^c_m}{\mu }\right) \, \text {d}\varGamma \,\text {d}t\\&-(-1)^{p+q}\varepsilon \mu \int _{T} \int _{\varOmega } \delta \varphi \frac{\partial ^{p+q}}{\partial x^p \partial y^q}\left[ \sigma _{pq}(\varphi )\left( D(\varphi ,\mathbf {a})+\frac{Y^c_m}{\mu }\right) \right] \, \text {d}\varOmega \, \text {d}t\\ &= \varepsilon \int _{\varOmega }\delta \varphi |_{t=1} (\varphi |_{t=1}-O^c_m) d\varOmega \\&+ \varepsilon \mu \int _{\varOmega } \delta \varphi \left( D(\varphi ,\mathbf {a})+\frac{Y^c_m}{\mu }\right) \left| _{t=1}\, \text {d}\varOmega \right. \\&-\varepsilon \mu \int _Q \delta \varphi \frac{\partial }{\partial t}\left( D(\varphi ,\mathbf {a})+\frac{Y^c_m}{\mu }\right) {\text { d}}Q\\&-(-1)^{p+q}\varepsilon \mu \int _Q \delta \varphi \frac{\partial ^{p+q}}{\partial x^p \partial y^q}\left[ \sigma _{pq}(\varphi )\left( D(\varphi ,\mathbf {a})+\frac{Y^c_m}{\mu }\right) \right] {\text { d}}Q. \end{aligned}$$

If \(\frac{\mathrm {D}L}{\mathrm {D}\varphi }\) exists, the boundary conditions must satisfy the equation as follows

$$\begin{aligned} \mu \left( D(\varphi ,\mathbf {a})+\frac{Y^c_m}{\mu }\right) \left| \right. _{t=1} = O^c_m-\varphi |_{t=1}. \end{aligned}$$
(22)

Then, we get the G\(\hat{a}\)teaux derivative

$$\begin{aligned} \frac{\mathrm {D}L}{\mathrm {D}\varphi } &= - \frac{\partial }{\partial t}\left( D(\varphi ,\mathbf {a})+\frac{Y^c_m}{\mu }\right) \\&-(-1)^{p+q}\frac{\partial ^{p+q}}{\partial x^p \partial y^q}\left[ \sigma _{pq}(\varphi )\left( D(\varphi ,\mathbf {a})+\frac{Y^c_m}{\mu }\right) \right] . \end{aligned}$$

When the functional gets the minimum, it should satisfy the follow PDE with the initial condition \(u^c_m|_{t=0}=I^c_m\),

$$\begin{aligned} \left\{ \begin{array}{l} \frac{\partial }{\partial t}\left( D(\varphi ,\mathbf {a})+\frac{Y^c_m}{\mu }\right) = -(-1)^{p+q}\frac{\partial ^{p+q}}{\partial x^p \partial y^q}\left[ \sigma _{pq}(\varphi )\left( D(\varphi ,\mathbf {a})+\frac{Y^c_m}{\mu }\right) \right] \\ \left( D(\varphi ,\mathbf {a})+\frac{Y^c_m}{\mu }\right) \left| \right. _{t=1} =\frac{1}{\mu }(O^c_m- \varphi |_{t=1}) ,\\ \varphi |_{t=0} = I^c_m. \end{array} \right. \end{aligned}$$
(23)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jiao, Y., Pan, X., Zhao, Z. et al. Learning sparse partial differential equations for vector-valued images. Neural Comput & Applic 29, 1205–1216 (2018). https://doi.org/10.1007/s00521-016-2623-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-016-2623-y

Keywords

Search

Navigation