skip to main content
10.1145/3365365.3382210acmconferencesArticle/Chapter ViewAbstractPublication PagescpsweekConference Proceedingsconference-collections
research-article

Conformance verification for neural network models of glucose-insulin dynamics

Published: 22 April 2020 Publication History

Abstract

Neural networks present a useful framework for learning complex dynamics, and are increasingly being considered as components to closed loop predictive control algorithms. However, if they are to be utilized in such safety-critical advisory settings, they must be provably "conformant" to the governing scientific (biological, chemical, physical) laws which underlie the modeled process. Unfortunately, this is not easily guaranteed as neural network models are prone to learn patterns which are artifacts of the conditions under which the training data is collected, which may not necessarily conform to underlying physiological laws.
In this work, we utilize a formal range-propagation based approach for checking whether neural network models for predicting future blood glucose levels of individuals with type-1 diabetes are monotonic in terms of their insulin inputs. These networks are increasingly part of closed loop predictive control algorithms for "artificial pancreas" devices which automate control of insulin delivery for individuals with type-1 diabetes. Our approach considers a key property that blood glucose levels must be monotonically decreasing with increasing insulin inputs to the model. Multiple representative neural network models for blood glucose prediction are trained and tested on real patient data, and conformance is tested through our verification approach. We observe that standard approaches to training networks result in models which violate the core relationship between insulin inputs and glucose levels, despite having high prediction accuracy. We propose an approach that can learn conformant models without much loss in accuracy.

References

[1]
Martín Abadi, Ashish Agarwal, Paul Barham, and et al. 2015. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. https://www.tensorflow.org/ Software available from tensorflow.org.
[2]
Houssam Abbas, Georgios Fainekos, Sriram Sankaranarayanan, Franjo Ivancic, and Aarti Gupta. 2013. Probabilistic Temporal Logic Falsification of Cyber-Physical Systems. Trans. on Embedded Computing Systems (TECS) 12 (2013), 95--. Issue 12s.
[3]
Yashwanth Annpureddy, Che Liu, Georgios E. Fainekos, and Sriram Sankaranarayanan. 2011. S-TaLiRo: A Tool for Temporal Logic Falsification for Hybrid Systems. In TACAS. 254--257.
[4]
B. Wayne Bequette. 2013. Algorithms for a Closed-Loop Artificial Pancreas: The Case for Model Predictive Control. J. Diabetes Science and Technology 7 (November 2013), 1632--1643. Issue 6.
[5]
R N Bergman, L S Phillips, and C Cobelli. 1981. Physiologic evaluation of factors controlling glucose tolerance in man: measurement of insulin sensitivity and beta-cell glucose sensitivity from the response to intravenous glucose. Journal of Clinical Investigation 68, 6 (Dec 1981), 1456--1467.
[6]
Marc D. Breton, Stephen D. Patek, Dayu Lv, Elaine Schertz, Jessica Robic, Jennifer Pinnata, Laura Kollar, Charlotte Barnett, Christian Wakeman, Mary Oliveri, and et al. 2018. Continuous Glucose Monitoring and Insulin Informed AdvisorySystem with Automated Titration and Dosing of Insulin Reduces Glucose Variability in Type 1 Diabetes Mellitus. Diabetes Technology & Therapeutics 20, 8 (Aug 2018), 531--540.
[7]
Rudy Bunel, Ilker Turkaslan, Philip H. S. Torr, Pushmeet Kohli, and M. Pawan Kumar. 2017. Piecewise Linear Neural Network verification: A comparative study. CoRR abs/1711.00455 (2017). http://arxiv.org/abs/1711.00455
[8]
Razvan Bunescu, Aili Guo, and Cindy Marling. 2018. The 3rd International Workshop on Knowledge Discovery in Healthcare. In KDH.
[9]
J. Carmona, B. Van Dongen, A. Solti, and M. Weidlich. 2018. Conformance Checking: Relating Processes and Models. Springer.
[10]
H. Peter Chase and David Maahs. 2011. Understanding Diabetes (Pink Panther Book) (12 ed.). Children's Diabetes Foundation. Available online through CU Denver Barbara Davis Center for Diabetes.
[11]
Frederick Chee and Tyrone Fernando. 2007. Closed-Loop Control of Blood Glucose. Springer.
[12]
Claudio Cobelli, David Foster, and Gianna Toffolo. 2000. Tracer Kinetics in Biomedical Research. Springer Science & Business Media.
[13]
Claudio Cobelli, Chiara Dalla Man, Giovanni Sparacino, Lalo Magni, Giuseppe De Nicolao, and Boris P. Kovatchev. 2009. Diabetes: Models, Signals and Control (Methodological Review). IEEE reviews in biomedical engineering 2 (2009), 54--95.
[14]
Pierre Courrieu. 1994. Three algorithms for estimating the domain of validity of feedforward neural networks. Neural Networks 7, 1 (1994), 169 -- 174.
[15]
Patrick Cousot and Rhadia Cousot. 1977. Abstract Interpretation: A unified Lattice Model for Static Analysis of Programs by Construction or Approximation of Fixpoints. In ACM Principles of Programming Languages. 238--252.
[16]
Chiara Dalla Man, Robert A Rizza, and Claudio Cobelli. 2006. Meal simulation model of the glucose-insulin system. IEEE Transactions on Biomedical Engineering 1, 10 (2006), 1740--1749.
[17]
Francis J. Doyle, Lauren M. Huyett, Joon Bok Lee, Howard C. Zisser, and Eyal Dassau. 2014. Closed-Loop Artificial Pancreas Systems: Engineering the Algorithms. Diabetes Care 37 (2014), 1191--1197.
[18]
Tommaso Dreossi, Alexandre Donzé, and Sanjit A. Seshia. 2017. Compositional Falsification of Cyber-Physical Systems with Machine Learning Components. In NASA Formal Methods (NFM). 357--372.
[19]
Souradeep Dutta, Xin Chen, and Sriram Sankaranarayanan. 2019. Reachability Analysis for Neural Feedback Systems Using Regressive Polynomial Rule Inference. In Proc. Hybrid Systems: Computation and Control (HSCC) (HSCC '19). ACM, New York, NY, USA, 157--168.
[20]
Souradeep Dutta, Susmit Jha, Sriram Sankaranarayanan, and Ashish Tiwari. 2018. Output range analysis for deep feedforward neural networks. In NASA Formal Methods Symposium. Springer, 121--138.
[21]
Souradeep Dutta, Susmit Jha, Sriram Sankaranarayanan, and Ashish Tiwari. 2018. Output Range Analysis for Deep Neural Networks. Proceedings of NASA Formal Methods Symposium (NFM) 10811 (2018), 121--138.
[22]
Souradeep Dutta, Taisa Kushner, and Sriram Sankaranarayanan. 2018. Robust Data-Driven Control of Artificial Pancreas Systems Using Neural Networks. In International Conference on Computational Methods in Systems Biology. Springer, 183--202.
[23]
Souradeep Dutta, Taisa Kushner, and Sriram Sankaranarayanan. 2018. Robust Data-Driven Control of Artificial Pancreas Systems using Neural Networks. In Computational Methods in Systems Biology (Lecture Notes In Computer Science), Vol. 11095. Springer-Verlag, 183--202.
[24]
Rüdiger Ehlers. 2017. Formal Verification of Piece-Wise Linear Feed-Forward Neural Networks. In ATVA (Lecture Notes in Computer Science), Vol. 10482. Springer, 269--286.
[25]
T. Gehr, M. Mirman, D. Drachsler-Cohen, P. Tsankov, S. Chaudhuri, and M. Vechev. 2018. AI2: Safety and Robustness Certification of Neural Networks with Abstract Interpretation. In 2018 IEEE Symposium on Security and Privacy (SP). 3--18.
[26]
Eleni I Georga, Vasilios C Protopappas, Diego Ardigò, Michela Marina, Ivana Zavaroni, Demosthenes Polyzos, and Dimitrios I Fotiadis. 2013. Multivariate prediction of subcutaneous glucose concentration in type 1 diabetes patients based on support vector regression. IEEE journal of biomedical and health informatics 17, 1 (2013), 71--81.
[27]
Eleni I Georga, Vasilios C Protopappas, Demosthenes Polyzos, and Dimitrios I Fotiadis. 2012. A predictive model of subcutaneous glucose concentration in type 1 diabetes based on random forests. In Engineering in Medicine and Biology Society (EMBC), 2012 Annual International Conference of the IEEE. IEEE, 2889--2892.
[28]
Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning. MIT Press. http://www.deeplearningbook.org.
[29]
Roman Hovorka. 2005. Continuous Glucose Monitoring and Closed-Loop Systems. Diabetic Medicine 23, 1 (2005), 1--12.
[30]
R. Hovorka, V. Canonico, L.J. Chassin, U. Haueter, M. Massi-Benedetti, M.O. Frederici, T.R. Pieber, H.C. Shaller, L. Schaupp, T. Vering, and M.E. Wilinska. 2004. Nonlinear Model Predictive Control of Glucose Concentration in Subjects with Type 1 Diabetes. Physiological Measurement 25 (2004), 905--920.
[31]
R. Hovorka, F. Shojaee-Moradie, P.V. Carroll, L.J. Chassin, I.J. Gowrie, N.C. Jackson, R.S. Tudor, A.M. Umpleby, and R.H. Hones. 2002. Partitioning Glucose distribution/transport, disposal and endogenous production during IVGTT. Am. J. Physiol. Endocrinol. Metab. 282 (2002), 992--1007.
[32]
Chao Huang, Jiameng Fan, Wenchao Li, Xin Chen, and Qi Zhu. 2019. ReachNN: Reachability Analysis of Neural-Network Controlled Systems. CoRR abs/1906.10654 (2019). arXiv:1906.10654 http://arxiv.org/abs/1906.10654
[33]
Janice J. Hwang, Lihong Jiang, Elizabeth Sanchez Rangel, Xiaoning Fan, Yuyan Ding, Wai Lam, Jessica Leventhal, Feng Dai, Douglas L. Rothman, Graeme F. Mason, and et al. 2018. Glycemic Variability and Brain Glucose Levels in Type 1 Diabetes. Diabetes 68, 1 (Oct 2018), 163--171.
[34]
Radoslav Ivanov, James Weimer, Rajeev Alur, George J. Pappas, and Insup Lee. 2019. Verisig: Verifying Safety Properties of Hybrid Systems with Neural Network Controllers. In Proc. Hybrid Systems: Computation and Control (HSCC) (HSCC '19). ACM, New York, NY, USA, 169--178.
[35]
Kyle Julian and Mykel J. Kochenderfer. 2017. Neural Network Guidance for UAVs. In AIAA Guidance Navigation and Control Conference (GNC).
[36]
Juvenile Diabetes Research Foundation (JDRF). [n. d.]. Identification of Areas of Artificial Pancreas Algorithm Enhancements Through Big-Data Analysis (Part 1). http://grantcenter.jdrf.org/rfa/identification-of-areas-of-artificial-pancreas-algorithm-enhancements-through-big-data-analysis-part-1/.
[37]
Guy Katz, Clark Barrett, David Dill, Kyle Julian, and Mykel Kochenderfer. 2017. Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks. (02 2017).
[38]
Guy Katz, Clark Barrett, David L. Dill, Kyle Julian, and Mykel J. Kochenderfer. 2017. Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks. Springer International Publishing, Cham, 97--117.
[39]
Guy Katz, Clark Barrett, David L. Dill, Kyle Julian, and Mykel J. Kochenderfer. 2017. Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks. In Computer Aided Verification. Springer, 97--117.
[40]
Boris P Kovatchev, Marc Breton, Chiara Dalla Man, and Claudio Cobelli. 2009. In silico preclinical trials: a proof of concept in closed-loop control of type 1 diabetes.
[41]
Aaron Kowalski. 2015. Pathway to Artificial Pancreas Revisited: Moving Downstream. Diabetes Care 38 (June 2015), 1036--1043. Issue 6.
[42]
Taisa Kushner, B. Wayne Bequette, Faye Cameron, Gregory Forlenza, David Maahs, and Sriram Sankaranarayanan. 2019. Models, Devices, Properties, and Verification of Artificial Pancreas Systems. Springer, 93--131.
[43]
Taisa Kushner, David Bortz, David Maahs, and Sriram Sankaranarayanan. 2018. A Data-Driven Approach to Artificial Pancreas Verification and Synthesis. In Intl. Conference on Cyber-Physical Systems (ICCPS'18). IEEE Press.
[44]
Alessio Lomuscio and Lalit Maganti. 2017. An approach to reachability analysis for feed-forward ReLU neural networks. CoRR abs/1706.07351 (2017). arXiv:1706.07351 http://arxiv.org/abs/1706.07351
[45]
Alessio Lomuscio and Lalit Maganti. 2017. An approach to reachability analysis for feed-forward ReLU neural networks. CoRR abs/1706.07351 (2017). arXiv:1706.07351 http://arxiv.org/abs/1706.07351
[46]
Chiara Dalla Man, Francesco Micheletto, Dayu Lv, Marc Breton, Boris Kovatchev, and Claudio Cobelli. 2014. The UVA/PADOVA Type I Diabetes Simulator: New Features. Journal of Diabetes Science and Technology 8, 1 (2014), 26--34.
[47]
Hrushikesh N. Mhaskar, Sergei V. Pereverzyev, and Maria D. van der Walt. 2017. A Deep Learning Approach to Diabetic Blood Glucose Prediction. Frontiers in Applied Mathematics and Statistics 3 (Jul 2017).
[48]
Stavroula G Mougiakakou, Aikaterini Prountzou, Dimitra Iliopoulou, Konstantina S Nikita, Andriani Vazeou, and Christos S Bartsocas. 2006. Neural network based glucose-insulin metabolism models for children with type 1 diabetes. In Engineering in Medicine and Biology Society, 2006. EMBS'06. 28th Annual International Conference of the IEEE. IEEE, 3545--3548.
[49]
M. Narasimhamurthy, T. Kushner, S. Dutta, and S. Sankaranarayanan. 2019. Verifying Conformance of Neural Network Models: Invited Paper. In 2019 IEEE/ACM International Conference on Computer-Aided Design (ICCAD). 1--8.
[50]
Scott M Pappada, Brent D Cameron, Paul M Rosman, Raymond E Bourey, Thomas J Papadimos, William Olorunto, and Marilyn J Borst. 2011. Neural network-based real-time prediction of glucose in patients with insulin-dependent diabetes. Diabetes technology & therapeutics 13, 2 (2011), 135--141.
[51]
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. 2017. Automatic differentiation in PyTorch. In NIPS Workshop on Automatic Differentiation.
[52]
Marcello Pelillo. 1996. A relaxation algorithm for estimating the domain of validity of feedforward neural networks. Neural Processing Letters 3, 3 (Aug 1996), 113--121.
[53]
Carmen Pérez-Gandía, A Facchinetti, G Sparacino, C Cobelli, EJ Gómez, M Rigla, Alberto de Leiva, and ME Hernando. 2010. Artificial neural network algorithm for online glucose prediction from continuous glucose monitoring. Diabetes technology & therapeutics 12, 1 (2010), 81--88.
[54]
Luca Pulina and Armando Tacchella. 2012. Challenging SMT Solvers to Verify Neural Networks. AI Commun. 25, 2 (2012), 117--135.
[55]
Joseph Sill. 1997. Monotonic Networks. In Proceedings of the 10th International Conference on Neural Information Processing Systems (NIPS'97). MIT Press, Cambridge, MA, USA, 661--667.
[56]
Jay S. Skyler. 2004. DCCT: The Study That Forever Changed the Nature of Treatment of Type 1 Diabetes. British Journal of Diabetes and Vascular Disease 4, 1 (2004). Cf. http://http://www.medscape.com/viewarticle/470738.
[57]
Jay S. Skyler (editor). 2012. Atlas of Diabetes: Fourth Edition. Springer Science+Business Media.
[58]
Garry M. Steil. 2013. Algorithms for a Closed-Loop Artificial Pancreas: The Case for Proportional-Integral-Derivative Control. J. Diabetes Sci. Technol. 7 (November 2013), 1621--1631. Issue 6.
[59]
Xiaowu Sun, Haitham Khedr, and Yasser Shoukry. 2019. Formal Verification of Neural Network Controlled Autonomous Systems. In Proc. Hybrid Systems: Computation and Control (HSCC) (HSCC '19). ACM, New York, NY, USA, 147--156.
[60]
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfellow, and Rob Fergus. 2013. Intriguing properties of neural networks. CoRR abs/1312.6199 (2013). arXiv:1312.6199 http://arxiv.org/abs/1312.6199
[61]
Vincent Tjeng and Russ Tedrake. 2017. Verifying Neural Networks with Mixed Integer Programming. CoRR abs/1711.07356 (2017). http://arxiv.org/abs/1711.07356
[62]
Jan Tretmans. 2008. Model Based Testing with Labelled Transition Systems. Springer Berlin Heidelberg, Berlin, Heidelberg, 1--38.
[63]
Cumhur Erkan Tuncali, Georgios Fainekos, Hisahiro Ito, and James Kapinski. 2018. Simulation-based Adversarial Test Generation for Autonomous Vehicles with Machine Learning Components. In 2018 IEEE Intelligent Vehicles Symposium. 1555--1562.
[64]
Cumhur Erkan Tuncali, James Kapinski, Hisahiro Ito, and Jyotirmoy V. Deshmukh. 2018. Reasoning about safety of learning-enabled components in autonomous cyber-physical systems. In Proc. Design Automation Conference, DAC 2018. 30:1--30:6.
[65]
Shiqi Wang, Kexin Pei, Justin Whitehouse, Junfeng Yang, and Suman Jana. 2018. Formal Security Analysis of Neural Networks using Symbolic Intervals. CoRR abs/1804.10829 (2018). arXiv:1804.10829 http://arxiv.org/abs/1804.10829
[66]
M.E. Wilinska, L.J. Chassin, C. L. Acerini, J. M. Allen, D.B. Dunber, and R. Hovorka. 2010. Simulation Environment to Evaluate Closed-Loop Insulin Delivery Systems in Type 1 Diabetes. J. Diabetes Science and Technology 4 (January 2010). Issue 1.
[67]
Matthias Woehrle, Kai Lampka, and Lothar Thiele. 2013. Conformance Testing for Cyber-physical Systems. ACM Trans. Embed. Comput. Syst. 11, 4, Article 84 (Jan. 2013), 23 pages.
[68]
Weiming Xiang and Taylor T. Johnson. 2018. Reachability Analysis and Safety Verification for Neural Network Control Systems. CoRR abs/1805.09944 (2018). http://arxiv.org/abs/1805.09944
[69]
Weiming Xiang, Hoang-Dung Tran, and Taylor T. Johnson. 2017. Reachable Set Computation and Safety Verification for Neural Networks with ReLU Activations. CoRR abs/1712.08163 (2017). arXiv:1712.08163 http://arxiv.org/abs/1712.08163
[70]
Weiming Xiang, Hoang-Dung Tran, and Taylor T. Johnson. 2107. Reachable Set Computation and Safety Verification for Neural Networks with ReLU Activations. Cf. https://arxiv.org/pdf/1712.08163.pdf, posted on ArXIV Dec. 2017.
[71]
Shakiba Yaghoubi and Georgios Fainekos. 2019. Gray-box adversarial testing for control systems with machine learning components. In Proceedings of Hybrid Systems: Computation and Control. 179--184.
[72]
Seungil You, David Ding, Kevin Canini, Jan Pfeifer, and Maya Gupta. 2017. Deep Lattice Networks and Partial Monotonic Functions. In Advances in Neural Information Processing Systems. Curran Associates, Inc., 2981--2989.

Cited By

View all
  • (2023)Artificial Intelligence Based Prediction of Diabetic Foot Risk in Patients with Diabetes: A Literature ReviewApplied Sciences10.3390/app1305282313:5(2823)Online publication date: 22-Feb-2023
  • (2023)The importance of interpreting machine learning models for blood glucose prediction in diabetes: an analysis using SHAPScientific Reports10.1038/s41598-023-44155-x13:1Online publication date: 6-Oct-2023

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
HSCC '20: Proceedings of the 23rd International Conference on Hybrid Systems: Computation and Control
April 2020
324 pages
ISBN:9781450370189
DOI:10.1145/3365365
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication Notes

Badge change: Article originally badged under Version 1.0 guidelines https://www.acm.org/publications/policies/artifact-review-badging

Publication History

Published: 22 April 2020

Permissions

Request permissions for this article.

Check for updates

Badges

Author Tags

  1. artificial pancreas systems
  2. conformance verification
  3. neural networks

Qualifiers

  • Research-article

Funding Sources

Conference

HSCC '20
Sponsor:

Acceptance Rates

Overall Acceptance Rate 153 of 373 submissions, 41%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)27
  • Downloads (Last 6 weeks)4
Reflects downloads up to 08 Mar 2025

Other Metrics

Citations

Cited By

View all
  • (2023)Artificial Intelligence Based Prediction of Diabetic Foot Risk in Patients with Diabetes: A Literature ReviewApplied Sciences10.3390/app1305282313:5(2823)Online publication date: 22-Feb-2023
  • (2023)The importance of interpreting machine learning models for blood glucose prediction in diabetes: an analysis using SHAPScientific Reports10.1038/s41598-023-44155-x13:1Online publication date: 6-Oct-2023

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media