Authors:
Adiel Ashrov
and
Guy Katz
Affiliation:
The Hebrew University of Jerusalem, Jerusalem, Israel
Keyword(s):
Scenario-Based Modeling, Behavioral Programming, Machine Learning, Deep Neural Networks, Software Engineering, Reactive Systems.
Abstract:
Deep neural networks (DNNs) have become a crucial instrument in the software development toolkit, due to
their ability to efficiently solve complex problems. Nevertheless, DNNs are highly opaque, and can behave
in an unexpected manner when they encounter unfamiliar input. One promising approach for addressing this
challenge is by extending DNN-based systems with hand-crafted override rules, which override the DNN’s
output when certain conditions are met. Here, we advocate crafting such override rules using the well-studied
scenario-based modeling paradigm, which produces rules that are simple, extensible, and powerful enough to
ensure the safety of the DNN, while also rendering the system more translucent. We report on two extensive
case studies, which demonstrate the feasibility of the approach; and through them, propose an extension to
scenario-based modeling, which facilitates its integration with DNN components. We regard this work as a
step towards creating safer and mo
re reliable DNN-based systems and models.
(More)