research-article

Refty: refinement types for valid deep learning models

Authors:

Mao YangAuthors Info & Claims

ICSE '22: Proceedings of the 44th International Conference on Software Engineering

Pages 1843 - 1855

https://doi.org/10.1145/3510003.3510077

Published: 05 July 2022 Publication History

Abstract

Deep learning has been increasingly adopted in many application areas. To construct valid deep learning models, developers must conform to certain computational constraints by carefully selecting appropriate neural architectures and hyperparameter values. For example, the kernel size hyperparameter of the 2D convolution operator cannot be overlarge to ensure that the height and width of the output tensor remain positive. Because model construction is largely manual and lacks necessary tooling support, it is possible to violate those constraints and raise type errors of deep learning models, causing either runtime exceptions or wrong output results. In this paper, we propose Refty, a refinement type-based tool for statically checking the validity of deep learning models ahead of job execution. Refty refines each type of deep learning operator with framework-independent logical formulae that describe the computational constraints on both tensors and hyperparameters. Given the neural architecture and hyperparameter domains of a model, Refty visits every operator, generates a set of constraints that the model should satisfy, and utilizes an SMT solver for solving the constraints. We have evaluated Refty on both individual operators and representative real-world models with various hyperparameter values under PyTorch and TensorFlow. We also compare it with an existing shape-checking tool. The experimental results show that Refty finds all the type errors and achieves 100% Precision and Recall, demonstrating its effectiveness.

References

[1]

Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. 2016. TensorFlow: A System for Large-Scale Machine Learning. In 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16). USENIX Association, Savannah, GA, 265--283. https://www.usenix.org/conference/osdi16/technical-sessions/presentation/abadi

Digital Library

[2]

Periklis Akritidis, Manuel Costa, Miguel Castro, and Steven Hand. 2009. Baggy Bounds Checking: An Efficient and Backwards-Compatible Defense against out-of-Bounds Errors. In Proceedings of the 18th Conference on USENIX Security Symposium (Montreal, Canada) (SSYM '09). USENIX Association, USA, 51--66.

[3]

Ganesh Ananthanarayanan, Sameer Agarwal, Srikanth Kandula, Albert Greenberg, Ion Stoica, Duke Harlan, and Ed Harris. 2011. Scarlett: Coping with Skewed Content Popularity in Mapreduce Clusters. In Proceedings of the Sixth Conference on Computer Systems (Salzburg, Austria) (EuroSys '11). Association for Computing Machinery, New York, NY, USA, 287--300.

Digital Library

[4]

Clark Barrett, Pascal Fontaine, and Cesare Tinelli. 2017. The SMT-LIB Standard: Version 2.6. Technical Report. Department of Computer Science, The University of Iowa. Available at www.SMT-LIB.org.

[5]

S. Ceri, G. Gottlob, and L. Tanca. 1989. What You Always Wanted to Know About Datalog (And Never Dared to Ask). IEEE Transactions on Knowledge and Data Engineering 1, 1 (mar 1989), 146--166.

Digital Library

[6]

Tongfei Chen. 2017. Typesafe Abstractions for Tensor Operations (Short Paper). In Proceedings of the 8th ACM SIGPLAN International Symposium on Scala (Vancouver, BC, Canada) (SCALA 2017). Association for Computing Machinery, New York, NY, USA, 45--50.

Digital Library

[7]

Tianqi Chen, Mu Li, Yutian Li, Min Lin, Naiyan Wang, Minjie Wang, Tianjun Xiao, Bing Xu, Chiyuan Zhang, and Zheng Zhang. 2015. MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems. CoRR abs/1512.01274 (2015). arXiv:1512.01274 http://arxiv.org/abs/1512.01274

[8]

Kyunghyun Cho, Bart van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, Doha, Qatar, 1724--1734.

[9]

François Chollet. 2017. Deep Learning with Python. Manning.

[10]

Ronan Collobert, Samy Bengio, and Johnny Mariéthoz. 2002. Torch: a modular machine learning software library. Idiap-RR Idiap-RR-46-2002. IDIAP.

[11]

Leonardo De Moura and Nikolaj Bjørner. 2008. Z3: An Efficient SMT Solver. In Proceedings of the Theory and Practice of Software, 14th International Conference on Tools and Algorithms for the Construction and Analysis of Systems (Budapest, Hungary) (TACAS '08/ETAPS '08). Springer-Verlag, Berlin, Heidelberg, 337--340.

[12]

Leonardo de Moura, Bruno Dutertre, and Natarajan Shankar. 2007. A Tutorial on Satisfiability modulo Theories. In Proceedings of the 19th International Conference on Computer Aided Verification (Berlin, Germany) (CAV '07). Springer-Verlag, Berlin, Heidelberg, 20--36.

Digital Library

[13]

Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. ImageNet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition. 248--255.

[14]

Li Deng. 2012. The MNIST Database of Handwritten Digit Images for Machine Learning Research [Best of the Web]. IEEE Signal Processing Magazine 29, 6 (2012), 141--142.

[15]

Julian Dolby, Avraham Shinnar, Allison Allain, and Jenna Reinen. 2018. Ariadne: Analysis for Machine Learning Programs. In Proceedings of the 2nd ACM SIGPLAN International Workshop on Machine Learning and Programming Languages (Philadelphia, PA, USA) (MAPL 2018). Association for Computing Machinery, New York, NY, USA, 1--10.

Digital Library

[16]

Bruno Dutertre. 2014. Yices 2.2. In Computer-Aided Verification (CAV'2014) (Lecture Notes in Computer Science, Vol. 8559), Armin Biere and Roderick Bloem (Eds.). Springer, 737--744.

Digital Library

[17]

Frederik Eaton. 2006. Statically Typed Linear Algebra in Haskell. In Proceedings of the 2006 ACM SIGPLAN Workshop on Haskell (Portland, Oregon, USA) (Haskell '06). Association for Computing Machinery, New York, NY, USA, 120--121.

Digital Library

[18]

Herbert B. Enderton. 2001. A Mathematical Introduction to Logic, Second edition. Elsevier.

[19]

Matvey Ezhov. 2021. Simple dynamic seq2seq with TensorFlow. https://notebook.community/ematvey/tensorflow-seq2seq-tutorials/1-seq2seq.

[20]

John Foderaro. 1991. LISP: Introduction. Commun. ACM 34, 9 (sep 1991), 27.

Digital Library

[21]

Tim Freeman and Frank Pfenning. 1991. Refinement Types for ML. In Proceedings of the ACM SIGPLAN 1991 Conference on Programming Language Design and Implementation (Toronto, Ontario, Canada) (PLDI '91). Association for Computing Machinery, New York, NY, USA, 268--277.

Digital Library

[22]

Vijay Ganesh and David L. Dill. 2007. A Decision Procedure for Bit-Vectors and Arrays. In Proceedings of the 19th International Conference on Computer Aided Verification (Berlin, Germany) (CAV '07). Springer-Verlag, Berlin, Heidelberg, 519--531.

[23]

Yanjie Gao, Yonghao Zhu, Hongyu Zhang, Haoxiang Lin, and Mao Yang. 2021. Resource-Guided Configuration Space Reduction for Deep Learning Models. In 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE) (Madrid, Spain) (ICSE '21). 175--187.

Digital Library

[24]

Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning. MIT Press. http://www.deeplearningbook.org.

Digital Library

[25]

Google. 2008. Protocol Buffers - Google's data interchange format. https://developers.google.com/protocol-buffers/.

[26]

Xiaodong Gu, Hongyu Zhang, Dongmei Zhang, and Sunghun Kim. 2016. Deep API Learning. In Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering (Seattle, WA, USA) (FSE 2016). Association for Computing Machinery, New York, NY, USA, 631--642.

Digital Library

[27]

Troels Henriksen and Martin Elsman. 2021. Towards Size-Dependent Types for Array Programming. In Proceedings of the 7th ACM SIGPLAN International Workshop on Libraries, Languages and Compilers for Array Programming (Virtual, Canada) (ARRAY 2021). Association for Computing Machinery, New York, NY, USA, 1--14.

Digital Library

[28]

C. A. R. Hoare. 1971. Procedures and Parameters: An Axiomatic Approach. In Symposium on Semantics of Algorithmic Languages, E. Engeler (Ed.). Springer Berlin Heidelberg, Berlin, Heidelberg, 102--116.

[29]

Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long Short-Term Memory. Neural Comput. 9, 8 (Nov. 1997), 1735--1780.

Digital Library

[30]

Md Johirul Islam, Giang Nguyen, Rangeet Pan, and Hridesh Rajan. 2019. A Comprehensive Study on Deep Learning Bug Characteristics. In Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (Tallinn, Estonia) (ESEC/FSE 2019). Association for Computing Machinery, New York, NY, USA, 510--520.

Digital Library

[31]

Srinivasan Iyer, Ioannis Konstas, Alvin Cheung, and Luke Zettlemoyer. 2016. Summarizing Source Code using a Neural Attention Model. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Berlin, Germany, 2073--2083.

[32]

C. Barry Jay. 1995. A semantics for shape. Science of Computer Programming 25, 2 (1995), 251 -- 283.

Digital Library

[33]

C. Barry Jay and Milan Sekanina. 1997. Shape Checking of Array Programs. In Proceedings of 1997 Computing: The Australasian Theory Symposium (Sydney, Australia) (CATS '97). Australian Computer Society, Inc., AUS.

[34]

Ranjit Jhala and Niki Vazou. 2020. Refinement Types: A Tutorial. CoRR abs/2010.07763 (2020). arXiv:2010.07763 https://arxiv.org/abs/2010.07763

[35]

Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, and Trevor Darrell. 2014. Caffe: Convolutional Architecture for Fast Feature Embedding. In Proceedings of the 22nd ACM International Conference on Multimedia (Orlando, Florida, USA) (MM '14). Association for Computing Machinery, New York, NY, USA, 675--678.

Digital Library

[36]

Ibrahem Kandel and Mauro Castelli. 2020. The effect of batch size on the generalizability of the convolutional neural networks on a histopathology dataset. ICT Express 6, 4 (2020), 312--315.

[37]

Martin Kellogg, Vlastimil Dort, Suzanne Millstein, and Michael D. Ernst. 2018. Lightweight Verification of Array Indexing. In Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis (Amsterdam, Netherlands) (ISSTA 2018). Association for Computing Machinery, New York, NY, USA, 3--14.

Digital Library

[38]

Wen Kokke, Ekaterina Komendantskaya, Daniel Kienitz, Robert Atkey, and David Aspinall. 2020. Neural Networks, Secure by Construction. In Proceedings of the 18th Asian Conference on Programming Languages and Systems (APLAS '20), Bruno C. d. S. Oliveira (Ed.). Springer International Publishing, Cham, 67--85.

Digital Library

[39]

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. ImageNet Classification with Deep Convolutional Neural Networks. In Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 1 (Lake Tahoe, Nevada) (NIPS '12). Curran Associates Inc., Red Hook, NY, USA, 1097--1105.

Digital Library

[40]

Sifis Lagouvardos, Julian Dolby, Neville Grech, Anastasios Antoniadis, and Yannis Smaragdakis. 2020. Static Analysis of Shape in TensorFlow Programs. In 34th European Conference on Object-Oriented Programming (ECOOP 2020) (Leibniz International Proceedings in Informatics (LIPIcs), Vol. 166), Robert Hirschfeld and Tobias Pape (Eds.). Schloss Dagstuhl-Leibniz-Zentrum für Informatik, Dagstuhl, Germany, 15:1--15:29.

[41]

Rasmus Munk Larsen and Tatiana Shpeisman. 2019. TensorFlow Graph Optimizations.

[42]

Nico Lehmann, Rose Kunkel, Jordan Brown, Jean Yang, Niki Vazou, Nadia Polikarpova, Deian Stefan, and Ranjit Jhala. 2021. STORM: Refinement Types for Secure Web Applications. In 15th USENIX Symposium on Operating Systems Design and Implementation (OSDI 21). USENIX Association, 441--459. https://www.usenix.org/conference/osdi21/presentation/lehmann

[43]

Wang Ling, Edward Grefenstette, Karl Moritz Hermann, Tomás Kociský, Andrew W. Senior, Fumin Wang, and Phil Blunsom. 2016. Latent Predictor Networks for Code Generation. CoRR abs/1603.06744 (2016). arXiv:1603.06744 http://arxiv.org/abs/1603.06744

[44]

Yu Liu, Cheng Chen, Ru Zhang, Tingting Qin, Xiang Ji, Haoxiang Lin, and Mao Yang. 2020. Enhancing the Interoperability between Deep Learning Frameworks by Model Conversion. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (Virtual Event, USA) (ESEC/FSE 2020). Association for Computing Machinery, New York, NY, USA, 1320--1330.

Digital Library

[45]

Yutaka Matsuno and Hiroyuki Sato. 2003. Flow Analytic Type System for Array Bound Checks. Electronic Notes in Theoretical Computer Science 78 (04 2003), 178--195.

[46]

Alexander Merritt. 2020. ONNX-Scala: Typeful, Functional Deep Learning / Dotty Meets an Open AI Standard (Open-Source Talk). In Proceedings of the 11th ACM SIGPLAN International Symposium on Scala (Virtual, USA) (SCALA 2020). Association for Computing Machinery, New York, NY, USA, 33.

Digital Library

[47]

Bertrand Meyer. 1992. Applying "Design by Contract". Computer 25, 10 (oct 1992), 40--51.

Digital Library

[48]

Microsoft. 2021. ND-series Virtual Machines. https://docs.microsoft.com/en-us/azure/virtual-machines/nd-series.

[49]

Robin Milner, Mads Tofte, and David Macqueen. 1997. The Definition of Standard ML. MIT Press, Cambridge, MA, USA.

Digital Library

[50]

Takayuki Muranushi and Richard A. Eisenberg. 2014. Experience Report: Type-Checking Polymorphic Units for Astrophysics Research in Haskell. In Proceedings of the 2014 ACM SIGPLAN Symposium on Haskell (Gothenburg, Sweden) (Haskell '14). Association for Computing Machinery, New York, NY, USA, 31--38.

Digital Library

[51]

David L. Olson and Dursun Delen. 2008. Advanced Data Mining Techniques (1st ed.). Springer Publishing Company, Incorporated.

[52]

oneDNN. 2021. Understanding Memory Formats. https://oneapi-src.github.io/oneDNN/v2.5/dev_guide_understanding_memory_formats.html.

[53]

ONNX. 2017. Open Neural Network Exchange. https://onnx.ai/.

[54]

Stack Overflow. 2017. Negative dimension size caused by subtracting 3 from 1 for "Conv2D". https://stackoverflow.com/questions/41651628/negative-dimension-size-caused-by-subtracting-3-from-1-for-conv2d.

[55]

Ran Pan. 2014. Tensor Transpose and Its Properties. CoRR abs/1411.1503 (2014). arXiv:1411.1503 http://arxiv.org/abs/1411.1503

[56]

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems 32, Vol. 32. Curran Associates, Inc., 8024--8035. https://proceedings.neurips.cc/paper/2019/file/bdbca288fee7f92f2bfa9f7012727740-Paper.pdf

[57]

Benjamin C. Pierce. 2002. Types and Programming Languages (1st ed.). The MIT Press.

Digital Library

[58]

Leonardo Piñeyro, Alberto Pardo, and Marcos Viera. 2021. Structure verification of deep neural networks at compilation time. Journal of Computer Languages 67 (2021), 101074.

[59]

PyTorch. 2019. Conv2d crashes with stride=0. https://github.com/pytorch/pytorch/issues/27598.

[60]

PyTorch. 2020. The topological sorting algorithm for the graph transformation subsystem. https://github.com/pytorch/pytorch/blob/v1.5.1/caffe2/core/nomnigraph/include/nomnigraph/Graph/TopoSort.h#L26.

[61]

PyTorch. 2020. The torch.nn.Conv2d API. https://pytorch.org/docs/1.5.1/nn.html#conv2d.

[62]

PyTorch. 2021. Torchvision. https://github.com/pytorch/vision.

[63]

PyTorch. 2021. Translation with a Sequence to Sequence Network and Attention. https://pytorch.org/tutorials/intermediate/seq2seq_translation_tutorial.html.

[64]

Stuart Russell and Peter Norvig. 2009. Artificial Intelligence: A Modern Approach (3rd ed.). Prentice Hall Press, USA.

Digital Library

[65]

Brennan Saeta, Denys Shabalin, Marc Rasi, Brad Larson, Xihui Wu, Parker Schuh, Michelle Casbon, Daniel Zheng, Saleem Abdulrasool, Aleksandr Efremov, Dave Abrahams, Chris Lattner, and Richard Wei. 2021. Swift for TensorFlow: A portable, flexible platform for deep learning. In Proceedings of Machine Learning and Systems, A. Smola, A. Dimakis, and I. Stoica (Eds.), Vol. 3. 240--254.

[66]

Sergio Guadarrama, Nathan Silberman. 2016. TensorFlow-Slim: A lightweight library for defining, training and evaluating complex models in TensorFlow. https://github.com/google-research/tf-slim.

[67]

Olha Shkaravska, Marko van Eekelen, and Ron van Kesteren. 2009. Polynomial Size Analysis of First-Order Shapely Functions. Logical Methods in Computer Science Volume 5, Issue 2 (May 2009).

[68]

João P. Marques Silva and Karem A. Sakallah. 1997. GRASP---a New Search Algorithm for Satisfiability. In Proceedings of the 1996 IEEE/ACM International Conference on Computer-Aided Design (San Jose, California, USA) (ICCAD '96). IEEE Computer Society, USA, 220--227.

[69]

Karen Simonyan and Andrew Zisserman. 2015. Very Deep Convolutional Networks for Large-Scale Image Recognition. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7--9, 2015, Conference Track Proceedings, Yoshua Bengio and Yann Le Cun (Eds.). http://arxiv.org/abs/1409.1556

[70]

Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to Sequence Learning with Neural Networks. In Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2 (Montreal, Canada) (NIPS '14). MIT Press, Cambridge, MA, USA, 3104--3112.

Digital Library

[71]

C. Szegedy, Wei Liu, Yangqing Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich. 2015. Going deeper with convolutions. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1--9.

[72]

TensorFlow. 2016. Large Strides for 1x1 Convolutions. https://github.com/tensorflow/tensorflow/issues/889.

[73]

TensorFlow. 2018. Primitive Neural Net (NN) Operations. https://github.com/tensorflow/docs/tree/r1.8/site/en/api_docs/python/tf/nn.

[74]

TensorFlow. 2019. The tf.layers.Conv2D API. https://github.com/tensorflow/docs/blob/r1.13/site/en/api_docs/python/tf/layers/Conv2D.md.

[75]

TensorFlow. 2019. The tf.nn.convolution API. https://github.com/tensorflow/docs/blob/r1.13/site/en/api_docs/python/tf/nn/convolution.md.

[76]

Takahisa Toda and Takehide Soh. 2016. Implementing Efficient All Solutions SAT Solvers. ACM J. Exp. Algorithmics 21, Article 1.12 (Nov. 2016), 44 pages.

Digital Library

[77]

Ben Trevett. 2021. PyTorch Seq2Seq. https://github.com/bentrevett/pytorch-seq2seq.

[78]

Sahil Verma and Zhendong Su. 2020. ShapeFlow: Dynamic Shape Interpreter for TensorFlow. CoRR abs/2011.13452 (2020). arXiv:2011.13452 https://arxiv.org/abs/2011.13452

[79]

WALA. 2021. The T. J. Watson Libraries for Analysis. https://github.com/wala/WALA.

[80]

Wikipedia. 2021. Cross-correlation --- Wikipedia, The Free Encyclopedia. http://en.wikipedia.org/w/index.php?title=Cross-correlation&oldid=1031522391.

[81]

Hongwei Xi and Frank Pfenning. 1998. Eliminating Array Bound Checking through Dependent Types. In Proceedings of the ACM SIGPLAN 1998 Conference on Programming Language Design and Implementation (Montreal, Quebec, Canada) (PLDI '98). Association for Computing Machinery, New York, NY, USA, 249--257.

Digital Library

[82]

Hongkun Yu, Chen Chen, Xianzhi Du, Yeqing Li, Abdullah Rashwan, Le Hou, Pengchong Jin, Fan Yang, Frederick Liu, Jaeyoun Kim, and Jing Li. 2020. TensorFlow Model Garden. https://github.com/tensorflow/models.

[83]

Matei Zaharia, Reynold S. Xin, Patrick Wendell, Tathagata Das, Michael Armbrust, Ankur Dave, Xiangrui Meng, Josh Rosen, Shivaram Venkataraman, Michael J. Franklin, Ali Ghodsi, Joseph Gonzalez, Scott Shenker, and Ion Stoica. 2016. Apache Spark: A Unified Engine for Big Data Processing. Commun. ACM 59, 11 (Oct. 2016), 56--65.

Digital Library

[84]

Jian Zhang, Xu Wang, Hongyu Zhang, Hailong Sun, Kaixuan Wang, and Xudong Liu. 2019. A Novel Neural Source Code Representation Based on Abstract Syntax Tree. In Proceedings of the 41st International Conference on Software Engineering (Montreal, Quebec, Canada) (ICSE '19). IEEE Press, 783--794.

Digital Library

[85]

Ru Zhang, Wencong Xiao, Hongyu Zhang, Yu Liu, Haoxiang Lin, and Mao Yang. 2020. An Empirical Study on Program Failures of Deep Learning Jobs. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering (Seoul, South Korea) (ICSE '20). Association for Computing Machinery, New York, NY, USA, 1159--1170.

Digital Library

[86]

Tianyi Zhang, Cuiyun Gao, Lei Ma, Michael Lyu, and Miryung Kim. 2019. An Empirical Study of Common Challenges in Developing Deep Learning Applications. In 2019 IEEE 30th International Symposium on Software Reliability Engineering (ISSRE). 104--115.

[87]

Yuhao Zhang, Yifan Chen, Shing-Chi Cheung, Yingfei Xiong, and Lu Zhang. 2018. An Empirical Study on TensorFlow Program Bugs. In Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis (Amsterdam, Netherlands) (ISSTA 2018). Association for Computing Machinery, New York, NY, USA, 129--140.

Digital Library

Cited By

Meijer WCombemale BWimmer MChechik MEgyed A(2024)Contract-based Validation of Conceptual Design Bugs for Engineering Complex Machine Learning SoftwareProceedings of the ACM/IEEE 27th International Conference on Model Driven Engineering Languages and Systems10.1145/3652620.3688201(155-161)Online publication date: 22-Sep-2024
https://dl.acm.org/doi/10.1145/3652620.3688201
Zheng DSen KRoychoudhury APaiva AAbreu RStorey MAniche MNagappan N(2024)Dynamic Inference of Likely Symbolic Tensor Shapes in Python Machine Learning ProgramsProceedings of the 46th International Conference on Software Engineering: Software Engineering in Practice10.1145/3639477.3639718(147-156)Online publication date: 14-Apr-2024
https://dl.acm.org/doi/10.1145/3639477.3639718
Gao YHe YLi XZhao BLin HLiang YZhong JZhang HWang JZeng YGui KTong JYang MRoychoudhury APaiva AAbreu RStorey M(2024)An Empirical Study on Low GPU Utilization of Deep Learning JobsProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639232(1-13)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3639232

Index Terms

Refty: refinement types for valid deep learning models
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation

Recommendations

A Hoare logic style refinement types formalisation
TyDe 2022: Proceedings of the 7th ACM SIGPLAN International Workshop on Type-Driven Development

Refinement types is a lightweight yet expressive tool for specifying and reasoning about programs. The connection between refinement types and Hoare logic has long been recognised but the discussion remains largely informal. In this paper, we present a ...
Contracts made manifest
POPL '10: Proceedings of the 37th annual ACM SIGPLAN-SIGACT symposium on Principles of programming languages

Since Findler and Felleisen introduced higher-order contracts , many variants have been proposed. Broadly, these fall into two groups: some follow Findler and Felleisen in using latent contracts, purely dynamic checks that are transparent to the type ...
Counterexample generation for program verification based on ownership refinement types
PEPM 2021: Proceedings of the 2021 ACM SIGPLAN Workshop on Partial Evaluation and Program Manipulation

Type-based program verification, which reduces program verification to type inference, has been used as a lightweight approach to automated program verification. Whilst it is often effective and faster than other methods such as model checking, the type-...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICSE '22: Proceedings of the 44th International Conference on Software Engineering

May 2022

2508 pages

ISBN:9781450392211

DOI:10.1145/3510003

General Chair:
Matthew B Dwyer
University of Virginia
,
Program Chairs:
Daniela Damian
University of Victoria, Canada
,
Andreas Zeller
CISPA, Germany

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

In-Cooperation

IEEE CS

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 July 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ICSE '22

Sponsor:

SIGSOFT

ICSE '22: 44th International Conference on Software Engineering

May 21 - 29, 2022

Pennsylvania, Pittsburgh

Acceptance Rates

Overall Acceptance Rate 276 of 1,856 submissions, 15%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
140
Total Downloads

Downloads (Last 12 months)23
Downloads (Last 6 weeks)4

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Meijer WCombemale BWimmer MChechik MEgyed A(2024)Contract-based Validation of Conceptual Design Bugs for Engineering Complex Machine Learning SoftwareProceedings of the ACM/IEEE 27th International Conference on Model Driven Engineering Languages and Systems10.1145/3652620.3688201(155-161)Online publication date: 22-Sep-2024
https://dl.acm.org/doi/10.1145/3652620.3688201
Zheng DSen KRoychoudhury APaiva AAbreu RStorey MAniche MNagappan N(2024)Dynamic Inference of Likely Symbolic Tensor Shapes in Python Machine Learning ProgramsProceedings of the 46th International Conference on Software Engineering: Software Engineering in Practice10.1145/3639477.3639718(147-156)Online publication date: 14-Apr-2024
https://dl.acm.org/doi/10.1145/3639477.3639718
Gao YHe YLi XZhao BLin HLiang YZhong JZhang HWang JZeng YGui KTong JYang MRoychoudhury APaiva AAbreu RStorey M(2024)An Empirical Study on Low GPU Utilization of Deep Learning JobsProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639232(1-13)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3639232

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten