Skip to main content
SpringerLink
Log in

Discovery the inverse variational problems from noisy data by physics-constrained machine learning

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Almost sophisticated physical phenomena and computational problems arise as variational problems. Recently, the development of neural networks (NNs), which has accomplished unbelievable success in many fields, especially in scientific computational fields. And almost sophisticated computational problems of physical phenomena can be viewed as a variational or PDE problem. In this work, we proposed learning Laplace-Beltrami-operator with physics-constrained machine learning and automatic differentiation to discover the inverse variational problems from noisy data. Also, the hidden fields are approximated by neural networks. Meanwhile, the neural networks and traditional high-order inverse variational problems are integrated, to make the traditional variational problem gain stronger vitality once again. We propose a way for sensitivity analysis, utilizing the automatic differentiation mechanism embedded in the framework. We propose a neural network approach to approximate unknown functions based on curvature regularity to learn the high order inverse variational problems under noisy data. Also, theoretically, our framework is flexible to adapt to the different high-order curvature-based variational problems, then, the NNs are used to solve the problems to improve computational efficiency. Furthermore, several experiments with different initial data and different noise levels are implemented to demonstrate the effectiveness and superiority of the proposed models. Our experiments confirm this property.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Algorithm 1
Algorithm 2
Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Li C, Yang Y, Liang H, Wu B (2021) Transfer learning for establishment of recognition of COVID-19 on CT imaging using small-sized training datasets. Knowl Based Syst 218:106849–106849

    Google Scholar 

  2. Kobler E, Effland A, Kunisch K, Pock T (2020) Total Deep Variation for Linear Inverse Problems. In: Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pp 7549–7558

  3. Gu Z, Li F, Fang F, Zhang G (2019) A novel Retinex-based fractional-order variational model for images with severely low light. IEEE Trans Image Process 29:3239–3253

    MathSciNet  MATH  Google Scholar 

  4. Capuani R, Dutta P, Nguyen KT (2021) Metric entropy for functions of bounded total generalized variation. SIAM J Math Anal 53(1):1168–1190

    MathSciNet  MATH  Google Scholar 

  5. Mu J, Xiong R, Fan X, Liu D, Wu F, Gao W (2020) Graph-based non-convex low-rank regularization for image compression artifact reduction. IEEE Trans Image Process 29:5374–5385

    MathSciNet  MATH  Google Scholar 

  6. Gao Y, Bredies K (2018) Infimal convolution of oscillation total generalized variation for the recovery of images with structured texture. SIAM J Imaging Sci 11(3):2021–2063

    MathSciNet  MATH  Google Scholar 

  7. Ma J, He J, Yang X (2020) Learning geodesic active contours for embedding object global information in segmentation CNNs. IEEE Trans Med Imaging 40(1):93–104

    Google Scholar 

  8. Liang L, Jin L, Xu Y (2020) PDE learning of filtering and propagation for Task-Aware facial intrinsic image analysis, IEEE Transactions on Cybernetics

  9. Chen X, Williams BM, Vallabhaneni S, Czanner G, Williams R, Zheng Y (2019) Learning active contour models for medical image segmentation. In: 2019 IEEE/CVF Conference on computer vision and pattern recognition (CVPR), pp 11624–11632

  10. Wu F, Zhuang X (2021) Unsupervised domain adaptation with variational approximation for cardiac segmentation, IEEE Transactions on Medical Imaging, vol PP

  11. Wang X, Chen H, Xiang H, Lin H, Heng P (2021) Deep virtual adversarial self-training with consistency regularization for semi-supervised medical image classification. Med Image Anal 70:102010

    Google Scholar 

  12. Li C, Yang Y, Liang H, Wu B (2021) Robust PCL discovery of data-driven mean-field game systems and control problems, IEEE Transactions on Circuits and Systems I: Regular Papers

  13. Wang K-J, Wang G-D (2021) Solitary and periodic wave solutions of the generalized Fourth-Order boussinesq equation via he’s variational methods. Math Methods Appl Sci 44(7):5617– 5625

    MathSciNet  MATH  Google Scholar 

  14. Kharazmi E, Zhang Z, Karniadakis G (2021) hp-VPINNs: Variational physics-informed neural networks with domain decomposition. Comput Methods Appl Mech Eng 374:113547

    MathSciNet  MATH  Google Scholar 

  15. Dupuy M-S (2021) Variational projector-augmented wave method: A full-potential approach for electronic structure calculations in solid-state physics. J Comput Phys 442:110510

    MathSciNet  MATH  Google Scholar 

  16. Boṫ RI, Csetnek ER, László SC (2021) Tikhonov Regularization of a Second Order Dynamical System with Hessian Driven Damping. Math Program 189(1):151–186

    MathSciNet  MATH  Google Scholar 

  17. Dong B, Ju H, Lu Y, Shi Z (2020) CURE: Curvature regularization for missing data recovery. SIAM J Imaging Sci 13(4):2169–2188

    MathSciNet  MATH  Google Scholar 

  18. He Y, Kang SH, Liu H (2020) Curvature Regularized Surface Reconstruction from Point Clouds. SIAM J Imaging Sci 13(4):1834–1859

    MathSciNet  MATH  Google Scholar 

  19. Gundogdu E, Constantin V, Parashar S, Seifoddini A, Dang M, Salzmann M, Fua P (2020) GarNet++: Improving Fast and Accurate Static 3D Cloth Draping by Curvature Loss, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol PP

  20. Spencer J, Chen K, Duan J (2018) Parameter-Free Selective segmentation with convex variational methods. IEEE Trans Image Process 28(5):2163–2172

    MathSciNet  Google Scholar 

  21. Jaouen V, Bert J, Boussion N, Fayad H, Hatt M, Visvikis D (2018) Image enhancement with PDEs and nonconservative advection flow fields. IEEE Trans Image Process 28(6):3075–3088

    MathSciNet  MATH  Google Scholar 

  22. Corsaro S, De Simone V, Marino Z (2021) Split Bregman iteration for multi-period mean variance portfolio optimization, vol 392

  23. Liu R, Mu P, Zhang J (2021) Investigating customization strategies and convergence behaviors of task-specific ADMM. IEEE Trans Image Process 30:8278–8292

    MathSciNet  Google Scholar 

  24. Khan S, Huh J, Ye JC (2021) Variational formulation of unsupervised deep learning for ultrasound image artifact removal. IEEE Trans Ultrason Ferroelectr Freq Control 68(6):2086– 2100

    Google Scholar 

  25. Wu Z-C, Huang T-Z, Deng L-J, Hu J-F, Vivone G (2021) VO+ Net: an adaptive approach using variational optimization and deep learning for panchromatic sharpening, IEEE Transactions on Geoscience and Remote Sensing

  26. Weinan E, Yu B (2018) The deep Ritz method: A deep learning-based numerical algorithm for solving variational problems. Commun Math Stat 6(1):1–12

    MathSciNet  MATH  Google Scholar 

  27. Thanh DNH, Prasath VS, Dvoenko S et al (2021) An adaptive image inpainting Method based on Euler’s elastica with adaptive parameters estimation and the discrete gradient method. Signal Process 178:107797

    Google Scholar 

  28. Fazel M, Hindi H, Boyd SP (2003) Log-det heuristic for matrix rank minimization with applications to hankel and euclidean distance matrices. In: Proceedings of the 2003 american control conference, 2003., vol 3, pp 2156–2162 IEEE

  29. Chambolle A, Pock T (2019) Total Roto-Translational variation. Numer Math 142(3):611–666

    MathSciNet  MATH  Google Scholar 

  30. Laurain A, Walker SW (2021) Optimal control of volume-preserving mean curvature Flow. J Comput Phys 438:110373

    MathSciNet  MATH  Google Scholar 

  31. Zhu W (2020) Image denoising using lp-norm of mean curvature of image surface. J Sci Comput 83(2):1–26

    MathSciNet  Google Scholar 

  32. Zaitzeff A, Esedoḡlu S, Garikipati K (2020) Second order threshold dynamics schemes for two phase motion by mean curvature. J Comput Phys 410,:109404

    MathSciNet  MATH  Google Scholar 

  33. Li B (2021) Convergence of dziuk’s semidiscrete finite element method for mean curvature flow of closed surfaces with high-order finite elements. SIAM J Numer Anal 59(3):1592–1617

    MathSciNet  MATH  Google Scholar 

  34. Smets BM, Portegies J, St-Onge E, Duits R (2021) Total Variation and Mean Curvature PDEs on the Homogeneous Space of Positions and Orientations. J Math Imaging Vis 63(2):237–262

    MathSciNet  MATH  Google Scholar 

  35. Nie R, Ma C, Cao J, Ding H, Zhou D (2021) A Total variation with joint norms for infrared and visible image fusion, IEEE Transactions on Multimedia

  36. Deng L-J, Glowinski R, Tai X-C (2019) A new operator splitting method for the euler elastica model for image smoothing. SIAM J Imaging Sci 12(2):1190–1230

    MathSciNet  Google Scholar 

  37. Chatterjee S, Goswami S (2021) New risk bounds for 2d total variation denoising. IEEE Trans Inf Theory 67(6):4060–4091

    MathSciNet  MATH  Google Scholar 

  38. Chen Y, Pock T (2017) Trainable nonlinear reaction diffusion: a flexible framework for fast and effective image restoration. IEEE Trans Pattern Anal Mach Intell 39:1256–1272

    Google Scholar 

  39. Zhong Q, Yin K, Duan Y (2021) Image Reconstruction by Minimizing Curvatures on Image Surface. J Math Imaging Vis 63:30–55

    MathSciNet  MATH  Google Scholar 

  40. Raissi M, Perdikaris P, Karniadakis GE (2019) Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J Comput Phys 378:686–707

    MathSciNet  MATH  Google Scholar 

  41. Meng X, Babaee H, Karniadakis G (2021) Multi-fidelity Bayesian neural networks: Algorithms and Applications. J Comput Phys 438:110361

    MathSciNet  MATH  Google Scholar 

  42. Cai S, Wang Z, Lu L, Zaki TA, Karniadakis G (2021) DeepM&Mnet: Inferring the electroconvection multiphysics fields based on operator approximation by neural networks, vol 436

  43. Wang Z, Zheng X, Chryssostomidis C, Karniadakis G (2021) A phase-field method for boiling heat transfer, vol 435

  44. Dissanayake M, Phan-Thien N (1994) Neural-Network-Based Approximations for solving partial differential equations. Commun Numer Methods Eng 10(3):195–201

    MATH  Google Scholar 

  45. Long Z, Lu Y, Ma X, Dong B (2018) PDE-Net:, Learning PDEs from Data. In: International conference on machine learning, pp 3208–3216

  46. Lu L, Jin P, Pang G, Zhang Z, Karniadakis GE (2021) Learning nonlinear operators via deepONet based on the universal approximation theorem of operators. Nat Mach Intell 3(3):218–229

    Google Scholar 

  47. Zhao L, Li Z, Wang Z, Caswell B, Ouyang J, Karniadakis G (2021) Active-and transfer-learning applied to microscale-macroscale coupling to simulate viscoelastic flows. J Comput Phys 427:110069

    MathSciNet  MATH  Google Scholar 

  48. Jin X, Cai S, Li H, Karniadakis GE (2021) NSFnets (Navier-Stokes flow nets): Physics-informed neural networks for the incompressible Navier-Stokes equations. J Comput Phys 426:109951

    MathSciNet  MATH  Google Scholar 

  49. Yang L, Meng X, Karniadakis G (2021) B-PINNs: Bayesian physics-informed neural networks for forward and inverse pde problems with noisy data. J Comput Phys 425:109913

    MathSciNet  MATH  Google Scholar 

  50. Lu L, Meng X, Mao Z, Karniadakis G (2020) DeepXDE: A deep learning library for solving differential equations. SIAM Rev 63:208–228

    MathSciNet  MATH  Google Scholar 

  51. Pang G, D’Elia M, Parks M, Karniadakis G (2020) NPINNs: Nonlocal physics-informed neural networks for a parametrized nonlocal universal laplacian operator: Algorithms and applications. J Comput Phys 422:109760

    MathSciNet  MATH  Google Scholar 

  52. Hornik K, Stinchcombe M, White H (1989) Multilayer feedforward networks are universal approximators. Neural Netw 2:359–366

    MATH  Google Scholar 

  53. Calin O (2020) Deep learning architectures springer

  54. Maziar R, Perdikaris P, Karniadakis GE (2019) Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J Comput Phys 378:686–707

    MathSciNet  MATH  Google Scholar 

  55. Hamedani EY, Aybat NS (2021) A Primal-Dual algorithm with line search for general Convex-Concave saddle point problems. SIAM J Optim 31(2):1299–1329

    MathSciNet  MATH  Google Scholar 

  56. Wang Y, Yin W, Zeng J (2019) Global convergence of ADMM in nonconvex nonsmooth optimization. J Sci Comput 78:29–63

    MathSciNet  MATH  Google Scholar 

  57. DP K, Ba J (2015) Adam: A Method for stochastic optimization

  58. Wang Z, Bovik A, Sheikh H, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13:600–612

    Google Scholar 

  59. Hore A, Ziou D (2010) Image quality metrics: PSNR vs. SSIM. In: 2010 20th international conference on pattern recognition, pp 2366–2369 IEEE

  60. Raissi M, Yazdani A, Karniadakis GE (2020) Hidden fluid mechanics: learning velocity and pressure fields from flow visualizations. Science 367(6481):1026–1030

    MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Tianjin International Engineering Institute, Tianjin University, Tianjin, 300072, China

    Hongbo Qu

  2. School of Electrical and Information Engineering, Tianjin University, Tianjin, 300072, China

    Hongchen Liu & Yonghong Hou

  3. Hebei Petroleum University of Technology, Chengde, China

    Shuang Jiang

  4. AVIC AVIONICS Co., Ltd, Beijing, China

    Jiabin Wang

Authors
  1. Hongbo Qu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hongchen Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shuang Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jiabin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yonghong Hou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hongchen Liu.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Hongbo Qu, Hongchen Liu and Shuang Jiang are contributed equally to this work.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Qu, H., Liu, H., Jiang, S. et al. Discovery the inverse variational problems from noisy data by physics-constrained machine learning. Appl Intell 53, 11229–11240 (2023). https://doi.org/10.1007/s10489-022-04079-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-04079-x

Keywords

Search

Navigation