Abstract
Interdicting the flow of illegal goods (such as drugs and ivory) is a major security concern for many countries. The massive scale of these networks, however, forces defenders to make judicious use of their limited resources. While existing solutions model this problem as a Network Security Game (NSG), they do not consider humans’ bounded rationality. Previous human behavior modeling works in Security Games, however, make use of large training datasets that are unrealistic in real-world situations; the ability to effectively test many models is constrained by the time-consuming and complex nature of field deployments. In addition, there is an implicit assumption in these works that a model’s prediction accuracy strongly correlates with the performance of its corresponding defender strategy (referred to as predictive reliability). If the assumption of predictive reliability does not hold, then this could lead to substantial losses for the defender. In the following paper, we (1) first demonstrate that predictive reliability is indeed strong for previous Stackelberg Security Game experiments. We also run our own set of human subject experiments in such a way that models are restricted to learning on dataset sizes representative of real-world constraints. In the analysis on that data, we demonstrate that (2) predictive reliability is extremely weak for NSGs. Following that discovery, however, we identify (3) key factors that influence predictive reliability results: the training set’s exposed attack surface and graph structure.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
In other words, to simulate real-world scenarios, we do not assume the presence of very large amounts of data, but nonetheless, there is a sufficient amount of NSG data included in our study to at least see a stable prediction made by our different behavior models.
- 2.
The algorithm to generate a Maximin strategy can be found in [10].
- 3.
Details on the binary search algorithm can be found in Yang et al.’s original PASAQ formulation [19].
- 4.
For a more detailed discussion of human subject experiment design considerations, such as steps taken to reduce sources of bias, please see the online appendix at: http://teamcore.usc.edu/people/benjamin/Ford15_GameSecAppendix.pdf.
References
Abbasi, Y.D., Short, M., Sinha, A., Sintov, N., Zhang, C., Tambe, M.: Human adversaries in opportunistic crime security games: evaluating competing bounded rationality models. In: 3rd Conference on Advances in Cognitive Systems (2015)
Bell, M.G.H., Kanturska, U., Schmöcker, J.D., Fonzone, A.: Attacker-defender models and road network vulnerability. Philos. Trans. Roy. Soc. A: Math. Phys. Eng. Sci. 366(1872), 1893–1906 (2008)
Camerer, C.: Behavioral Game Theory: Experiments in Strategic Interaction. Princeton University Press, Princeton (2003)
Correa, J.R., Harks, T., Kreuzen, V.J.C., Matuschke, J.: Fare evasion in transit networks. In: CoRR (2014)
Cui, J., John, R.S.: Empirical comparisons of descriptive multi-objective adversary models in stackelberg security games. In: Poovendran, R., Saad, W. (eds.) GameSec 2014. LNCS, vol. 8840, pp. 309–318. Springer, Heidelberg (2014)
Eppstein, D., Goodrich, M.T.: Studying (non-planar) road networks through an algorithmic lens. In: Proceedings of the 16th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, p. 16. ACM (2008)
Fave, F.M.D., Jiang, A.X., Yin, Z., Zhang, C., Tambe, M., Kraus, S., Sullivan, J.: Game-theoretic security patrolling with dynamic execution uncertainty and a case study on a real transit system. J. Artif. Intell. Res. 50, 321–367 (2014)
Gutfraind, A., Hagberg, A., Pan, F.: Optimal interdiction of unreactive markovian evaders. In: Hooker, J.N., van Hoeve, W.-J. (eds.) CPAIOR 2009. LNCS, vol. 5547, pp. 102–116. Springer, Heidelberg (2009)
Jain, M., Conitzer, V., Tambe, M.: Security scheduling for real-world networks. In: AAMAS (2013)
Jain, M., Korzhyk, D., Vanek, O., Conitzer, V., Pechoucek, M., Tambe, M.: A double oracle algorithm for zero-sum security games on graphs. In: AAMAS (2011)
Kar, D., Fang, F., Fave, F.D., Sintov, N., Tambe, M.: “A game of thrones”: when human behavior models compete in repeated stackelberg security games. In: AAMAS (2015)
Manadhata, P., Wing, J.M.: Measuring a system’s attack surface. Technical report, DTIC Document (2004)
Morton, D.P., Pan, F., Saeger, K.J.: Models for nuclear smuggling interdiction. IIE Trans. 39(1), 3–14 (2007)
Nguyen, T.H., Yang, R., Azaria, A., Kraus, S., Tambe, M.: Analyzing the effectiveness of adversary modeling in security games. In: AAAI (2013)
Shieh, E., An, B., Yang, R., Tambe, M., Baldwin, C., DiRenzo, J., Maule, B., Meyer, G.: Protect: a deployed game theoretic system to protect the ports of the united states. In: AAMAS (2012)
Tambe, M.: Security and Game Theory: Algorithms, Deployed Systems. Lessons Learned. Cambridge University Press, New York (2011)
Tsai, J., Yin, Z., Kwak, J.y., Kempe, D., Kiekintveld, C., Tambe, M.: Urban security: Game-theoretic resource allocation in networked physical domains. In: AAAI (2010)
Yang, R., Fang, F., Jiang, A.X., Rajagopal, K., Tambe, M., Maheswaran, R.: Modeling human bounded rationality to improve defender strategies in network security games. In: HAIDM Workshop at AAMAS (2012)
Yang, R., Ordonez, F., Tambe, M.: Computing optimal strategy against quantal response in security games. In: AAMAS (2012)
Acknowledgments
This research was supported by MURI Grant W911NF-11-1-0332 and by CREATE under grant number 2010-ST-061-RE0001.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Ford, B., Nguyen, T., Tambe, M., Sintov, N., Fave, F.D. (2015). Beware the Soothsayer: From Attack Prediction Accuracy to Predictive Reliability in Security Games. In: Khouzani, M., Panaousis, E., Theodorakopoulos, G. (eds) Decision and Game Theory for Security. GameSec 2015. Lecture Notes in Computer Science(), vol 9406. Springer, Cham. https://doi.org/10.1007/978-3-319-25594-1_3
Download citation
DOI: https://doi.org/10.1007/978-3-319-25594-1_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-25593-4
Online ISBN: 978-3-319-25594-1
eBook Packages: Computer ScienceComputer Science (R0)