Abstract
In the context of classification, robustness verification of a neural network is the problem which consists in determining if small changes of inputs lead to a change of their assigned classes. We investigate such a problem on binarized neural networks via an integer linear programming perspective. We namely present a constraint generation framework based on disjunctive programming and complete descriptions of polytopes related to outputs of neuron pairs. We also introduce an alternative relying on specific families of facet defining inequalities. Preliminary experiments assess the performance of the latter approach against recent single neuron convexification results.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Amir, G., Wu, H., Barrett, C., Katz, G.: An SMT-based approach for verifying binarized neural networks. In: Groote, J.F., Larsen, K.G. (eds.) TACAS 2021. LNCS, vol. 12652, pp. 203–222. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-72013-1_11
Anderson, R., Huchette, J., Ma, W., Tjandraatmadja, C., Vielma, J.P.: Strong mixed-integer programming formulations for trained neural networks. Math. Program. 183, 3–39 (2020)
Bal, H., et al.: A medium-scale distributed system for computer science research: infrastructure for the long term. Computer 49(5), 54–63 (2016)
Bunel, R., Turkaslan, I., Torr, P.H.S., Kohli, P., Mudigonda, P.K.: A unified view of piecewise linear neural network verification. In: Neural Information Processing Systems (2017)
Cheng, C., Nührenberg, G., Ruess, H.: Verification of binarized neural networks. CoRR abs/1710.03107 (2017). http://arxiv.org/abs/1710.03107
Christof, T., Löbel, A.: Porta - polyhedron representation transformation algorithm. https://porta.zib.de/
Deng, L.: The MNIST database of handwritten digit images for machine learning research. IEEE Signal Process. Mag. 29(6), 141–142 (2012)
Goldberg, Y.: A primer on neural network models for natural language processing. J. Artif. Intell. Res. 57, 345–420 (2016)
Han, S., Gómez, A.: Single-neuron convexifications for binarized neural networks. University of Southern California (2021). https://optimization-online.org/?p=17148
Hornik, K., Stinchcombe, M., White, H.: Multilayer feedforward networks are universal approximators. Neural Netw. 2(5), 359–366 (1989)
Hubara, I., Courbariaux, M., Soudry, D., El-Yaniv, R., Bengio, Y.: Binarized neural networks. In: Lee, D., Sugiyama, M., Luxburg, U., Guyon, I., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 29. Curran Associates, Inc. (2016)
Jia, K., Rinard, M.C.: Efficient exact verification of binarized neural networks. CoRR abs/2005.03597 (2020). https://arxiv.org/abs/2005.03597
Katz, G., Barrett, C., Dill, D.L., Julian, K., Kochenderfer, M.J.: Reluplex: a calculus for reasoning about deep neural networks. Formal Methods Syst. Des. 60, 87–116 (2022). https://doi.org/10.1007/s10703-021-00363-7
Khalil, E.B., Gupta, A., Dilkina, B.: Combinatorial attacks on binarized neural networks. In: International Conference on Learning Representations (ICLR) (2019)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Pereira, F., Burges, C., Bottou, L., Weinberger, K. (eds.) Advances in Neural Information Processing Systems, vol. 25. Curran Associates, Inc. (2012)
Lin, W., et al.: Robustness verification of classification deep neural networks via linear programming. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11410–11419 (2019)
Liu, C., Arnon, T., Lazarus, C., Strong, C., Barrett, C., Kochenderfer, M.J.: Algorithms for Verifying Deep Neural Networks (2021)
Lyu, B., Huchette, J.: Verifying binarized neural networks: convex relaxations, mixed-integer programming, and consistency. https://bochuanbob.github.io/BNN_MIP.pdf
Narodytska, N., Kasiviswanathan, S., Ryzhyk, L., Sagiv, M., Walsh, T.: Verifying properties of binarized deep neural networks. In: AAAI Conference on Artificial Intelligence (AAAI) (2018)
Narodytska, N., Zhang, H., Gupta, A., Walsh, T.: In search for a sat-friendly binarized neural network architecture. In: International Conference on Learning Representations (ICLR) (2020)
Szegedy, C., et al.: Intriguing properties of neural networks. In: International Conference on Learning Representations (ICLR) (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Lubczyk, D., Neto, J. (2024). Neuron Pairs in Binarized Neural Networks Robustness Verification via Integer Linear Programming. In: Basu, A., Mahjoub, A.R., Salazar González, J.J. (eds) Combinatorial Optimization. ISCO 2024. Lecture Notes in Computer Science, vol 14594. Springer, Cham. https://doi.org/10.1007/978-3-031-60924-4_23
Download citation
DOI: https://doi.org/10.1007/978-3-031-60924-4_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-60923-7
Online ISBN: 978-3-031-60924-4
eBook Packages: Computer ScienceComputer Science (R0)