Haoran Li
ABSTRACT
Deep neural networks (DNNs) have been extensively applied to various fields, including physical-system monitoring and control. However, the requirement of a high confidence level in physical systems made system operators hard to trust black-box type DNNs. For example, while DNN can perform well at both training data and testing data, but when the physical system changes its operation points at a completely different range, never appeared in the history records, DNN can fail. To open the black box as much as possible, we propose a Physics-Consistent Neural Network (PCNN) for physical systems with the following properties: (1) PCNN can be shrunk to physical equations for sub-areas with full observability, (2) PCNN reduces unobservable areas into some virtual nodes, leading to a reduced network. Thus, for such a network, PCNN can also represent its underlying physical equation via a specifically designed deep-shallow hierarchy, and (3) PCNN is theoretically proved that the shallow NN in the PCNN is convex with respect to physical variables, leading to a set of convex optimizations to seek for the physics-consistent initial guess for the PCNN. We also develop a physical rule-based approach for initial guesses, significantly shortening the searching time for large systems. Comprehensive experiments on diversified systems are implemented to illustrate the outstanding performance of our PCNN.
Supplemental Material
- Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Müller, and Wojciech Samek. 2015. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PloS one, Vol. 10, 7 (2015), e0130140.Google Scholar
Cross Ref
- Steven L Brunton, Joshua L Proctor, and J Nathan Kutz. 2016. Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proceedings of the national academy of sciences, Vol. 113, 15 (2016), 3932--3937.Google ScholarCross Ref
- Kathleen Champion, Bethany Lusch, J Nathan Kutz, and Steven L Brunton. 2019. Data-driven discovery of coordinates and governing equations. Proceedings of the National Academy of Sciences, Vol. 116, 45 (2019), 22445--22451.Google ScholarCross Ref
- Samuel I Daitch and Daniel A Spielman. 2008. Faster approximate lossy generalized flow via interior point algorithms. In Proceedings of the fortieth annual ACM symposium on Theory of computing. 451--460.Google ScholarDigital Library
- Timothy A Davis and Yifan Hu. 2011. The University of Florida sparse matrix collection. ACM Transactions on Mathematical Software (TOMS), Vol. 38, 1 (2011), 1--25.Google ScholarDigital Library
- Jonathan Goh, Sridhar Adepu, Marcus Tan, and Zi Shan Lee. 2017. Anomaly detection in cyber physical systems using recurrent neural networks. In 2017 IEEE 18th International Symposium on High Assurance Systems Engineering (HASE). IEEE, 140--145.Google ScholarCross Ref
- Aric Hagberg and Daniel A Schult. 2008. Rewiring networks for synchronization. Chaos: An interdisciplinary journal of nonlinear science, Vol. 18, 3 (2008), 037105.Google Scholar
- Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.Google ScholarCross Ref
- Xinyue Hu, Haoji Hu, Saurabh Verma, and Zhi-Li Zhang. 2020. Physics-Guided Deep Neural Networks for PowerFlow Analysis. arXiv preprint arXiv:2002.00097 (2020).Google Scholar
- Daniel Jakubovitz and Raja Giryes. 2018. Improving dnn robustness to adversarial attacks using jacobian regularization. In Proceedings of the European Conference on Computer Vision (ECCV). 514--529.Google ScholarDigital Library
- Xiaowei Jia, Jared Willard, Anuj Karpatne, Jordan Read, Jacob Zwart, Michael Steinbach, and Vipin Kumar. 2019. Physics guided RNNs for modeling dynamical systems: A case study in simulating lake temperature profiles. In Proceedings of the 2019 SIAM International Conference on Data Mining. SIAM, 558--566.Google ScholarCross Ref
- Xiaowei Jia, Jared Willard, Anuj Karpatne, Jordan S Read, Jacob A Zwart, Michael Steinbach, and Vipin Kumar. 2020. Physics-guided machine learning for scientific discovery: An application in simulating lake temperature profiles. arXiv preprint arXiv:2001.11086 (2020).Google Scholar
- Y. Jiang, Z. Wu, J. Wang, X. Xue, and S. Chang. 2018. Exploiting Feature and Class Relationships in Video Categorization with Regularized Deep Neural Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 40, 2 (2018), 352--364.Google ScholarDigital Library
- Anuj Karpatne, William Watkins, Jordan Read, and Vipin Kumar. 2017. Physics-guided neural networks (pgnn): An application in lake temperature modeling. arXiv preprint arXiv:1710.11431 (2017).Google Scholar
- Haoran Li, Yang Weng, Yizheng Liao, Brian Keel, and Kenneth E Brown. 2021. Distribution grid impedance & topology estimation with limited or no micro-PMUs. International Journal of Electrical Power & Energy Systems, Vol. 129 (2021), 106794.Google ScholarCross Ref
- Mingchen Li, Mahdi Soltanolkotabi, and Samet Oymak. 2020. Gradient descent with early stopping is provably robust to label noise for overparameterized neural networks. In International Conference on Artificial Intelligence and Statistics. PMLR, 4313--4324.Google Scholar
- Yin Liu and Vincent Chen. 2018. On the Generalization Effects of DenseNet Model Structures. (2018).Google Scholar
- Scott M Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. In Advances in neural information processing systems. 4765--4774.Google Scholar
- MATPOWER community. 2020. MATPOWER. (2020). https://matpower.org/.Google Scholar
- Agnieszka Mikołajczyk and Michał Grochowski. 2018. Data augmentation for improving deep learning in image classification problem. In 2018 international interdisciplinary PhD workshop (IIPhDW). IEEE, 117--122.Google Scholar
- PJM Interconnection LLC. 2018. Metered Load Data. (2018). https://dataminer2.pjm.com/feed/hrl_load_metered/definition.Google Scholar
- R. O. Saber and R. M. Murray. 2003. Consensus protocols for networks of dynamic agents. In Proceedings of the 2003 American Control Conference, 2003., Vol. 2. 951--956. https://doi.org/10.1109/ACC.2003.1239709Google ScholarCross Ref
- Shital Shah, Debadeepta Dey, Chris Lovett, and Ashish Kapoor. 2018. Airsim: High-fidelity visual and physical simulation for autonomous vehicles. In Field and service robotics. Springer, 621--635.Google Scholar
- Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. 2017. Learning important features through propagating activation differences. arXiv preprint arXiv:1704.02685 (2017).Google ScholarDigital Library
- Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2013. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034 (2013).Google Scholar
- Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research, Vol. 15, 1 (2014), 1929--1958.Google Scholar
- Silviu-Marian Udrescu and Max Tegmark. 2020. AI Feynman: A physics-inspired method for symbolic regression. Science Advances, Vol. 6, 16 (2020), eaay2631.Google Scholar
- Arjan van der Schaft. 2017. Modeling of physical network systems. Systems & Control Letters, Vol. 101 (2017), 21--27.Google ScholarCross Ref
- Zhong Yi Wan, Pantelis Vlachas, Petros Koumoutsakos, and Themistoklis Sapsis. 2018. Data-assisted reduced-order modeling of extreme events in complex dynamical systems. PloS one, Vol. 13, 5 (2018), e0197704.Google ScholarCross Ref
- Jared Willard, Xiaowei Jia, Shaoming Xu, Michael Steinbach, and Vipin Kumar. 2020. Integrating physics-based modeling with machine learning: A survey. arXiv preprint arXiv:2003.04919 (2020).Google Scholar
- Dong Yu, Kaisheng Yao, Hang Su, Gang Li, and Frank Seide. 2013. KL-divergence regularized deep neural network adaptation for improved large vocabulary speech recognition. In 2013 IEEE International Conference on Acoustics, Speech and Signal Processing. IEEE, 7893--7897.Google ScholarCross Ref
- J. Yu, Y. Weng, and R. Rajagopal. 2017. Robust mapping rule estimation for power flow analysis in distribution grids. In 2017 North American Power Symposium (NAPS). 1--6.Google Scholar
- Matthew D Zeiler and Rob Fergus. 2014. Visualizing and understanding convolutional networks. In European conference on computer vision. Springer, 818--833.Google ScholarCross Ref
- Jian Zhou and Olga G Troyanskaya. 2015. Predicting effects of noncoding variants with deep learning-based sequence model. Nature methods, Vol. 12, 10 (2015), 931--934.Google Scholar
- Luisa M Zintgraf, Taco S Cohen, Tameem Adel, and Max Welling. 2017. Visualizing deep neural network decisions: Prediction difference analysis. arXiv preprint arXiv:1702.04595 (2017).Google Scholar
Index Terms
- Physical Equation Discovery Using Physics-Consistent Neural Network (PCNN) Under Incomplete Observability
Recommendations
Numerical solution of Generalized Burger–Huxley & Huxley’s equation using Deep Galerkin neural network method
AbstractIn this paper, a deep learning algorithm based on Deep Galerkin method (DGM) is presented for the approximate solution of the generalized Burgers–Huxley equation (gBHE), and generalized Huxley’s equation (gHE). In this method, a deep ...
Neural network model of selective visual attention using Hodgkin---Huxley equation
We propose a mathematical model of selective visual attention using a two-layered neural network with neurons described by the Hodgkin---Huxley equation in order to investigate part of the assumption proposed by Desimone and Duncan. The neural network ...
Studies of stimulus parameters for seizure disruption using neural network simulations
AbstractA large scale neural network simulation with realistic cortical architecture has been undertaken to investigate the effects of external electrical stimulation on the propagation and evolution of ongoing seizure activity. This is an effort to ...
Comments