Skip to main content
SpringerLink
Log in

Combining Online Learning and Equilibrium Computation in Security Games

  • Conference paper
  • First Online:
Decision and Game Theory for Security (GameSec 2015)

Part of the book series: Lecture Notes in Computer Science ((LNSC,volume 9406))

Included in the following conference series:

Abstract

Game-theoretic analysis has emerged as an important method for making resource allocation decisions in both infrastructure protection and cyber security domains. However, static equilibrium models defined based on inputs from domain experts have weaknesses; they can be inaccurate, and they do not adapt over time as the situation (and adversary) evolves. In cases where there are frequent interactions with an attacker, using learning to adapt to an adversary revealed behavior may lead to better solutions in the long run. However, learning approaches need a lot of data, may perform poorly at the start, and may not be able to take advantage of expert analysis. We explore ways to combine equilibrium analysis with online learning methods with the goal of gaining the advantages of both approaches. We present several hybrid methods that combine these techniques in different ways, and empirically evaluated the performance of these methods in a game that models a border patrolling scenario.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    This is sometimes described as the problem of estimating the total flow of traffic, rather than just the known or observed flow based on detections and apprehensions.

References

  1. 2012–2016 border patrol strategic plan. U.S. Customs and Border Protection (2012)

    Google Scholar 

  2. An, B., Brown, M., Vorobeychik, Y., Tambe, M.: Security games with surveillance cost and optimal timing of attack execution. In: AAMAS, pp. 223–230 (2013)

    Google Scholar 

  3. An, B., Kiekintveld, C., Shieh, E., Singh, S., Tambe, M., Vorobeychik, Y.: Security games with limited surveillance. In: AAAI, pp. 1241–1248 (2012)

    Google Scholar 

  4. Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multi-armed bandit problem. Mach. Learn. 47, 235–256 (2002)

    Article  MATH  Google Scholar 

  5. Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The non-stochastic multi-armed bandit problem. SIAM J. Comput. 32(1), 48–77 (2001)

    Article  MathSciNet  Google Scholar 

  6. Balcan, M.-F., Blum, A., Haghtalab, N., Procaccia, A.D.: Commitment without regrets: online learning in stackelberg security games. In: ACM Conference on Economics and Computation (EC-2015), pp. 61–78 (2015)

    Google Scholar 

  7. Bard, N., Johanson, M., Burch, N., Bowling, M.: Online implicit agent modelling. In: AAMAS, pp. 255–262 (2013)

    Google Scholar 

  8. Bard, N., Nicholas, D., Szepesvari, C., Bowling, M.: Decision-theoretic clustering of strategies. In: AAMAS, pp. 17–25 (2015)

    Google Scholar 

  9. Blum, A., Nika, H., Procaccia, F.D.: Lazy defenders are almost optimal against diligent attackers. In: AAAI, pp. 573–579 (2014)

    Google Scholar 

  10. Combes, R., Lelarge, M., Proutiere, A., Talebi, M.S.: Stochastic and adversarial combinatorial bandits (2015). arXiv:1502.03475

  11. Cowling, P.I., Powley, E.J., Whitehouse, D.: Information set monte carlo tree search. IEEE Trans. Comput. Intell. AI Games 4, 120–143 (2012)

    Article  Google Scholar 

  12. Fudenberg, D., Levine, D.K.: The Theory of Learning in Games. The MIT Press, Cambridge (1998)

    MATH  Google Scholar 

  13. Garivier, A., Moulines, E.: On upper-confidence bound policies for non-stationary bandit problems. In: ALT, pp. 174–188 (2011)

    Google Scholar 

  14. Kiekintveld, C., Jain, M., Tsai, J., Pita, J., Ordonez, F., Tambe, M.: Computing optimal randomized resource allocations for massive security games. In: AAMAS, pp. 689–696 (2009)

    Google Scholar 

  15. Kiekintveld, C., Kreinovich, V.: Efficient approximation for security games with interval uncertainty. In: AAAI, pp. 42–45 (2012)

    Google Scholar 

  16. Kiekintveld, C., Marecki, J., Tambe, M.: Approximation methods for infinite Bayesian Stackelberg games: modeling distributional payoff uncertainty. In: AAMAS, pp. 1005–1012 (2011)

    Google Scholar 

  17. Klima, R., Kiekintveld, C., Lisy, V.: Online learning methods for border patrol resource allocation. In: GAMESEC, pp. 340–349 (2014)

    Google Scholar 

  18. Nguyen, T.H., Jiang, A., Tambe, M.: Stop the compartmentalization: unified robust algorithms for handling uncertainties in security games. In: AAMAS, pp. 317–324 (2014)

    Google Scholar 

  19. Pita, J., Jain, M., Ordonez, F., Portway, C., Tambe, M., Western, C., Paruchuri, P., Kraus, S.: ARMOR security for los angeles international airport. In: AAAI, pp. 1884–1885 (2008)

    Google Scholar 

  20. Pita, J., Jain, M., Ordonez, F., Tambe, M., Kraus, S.: Robust solutions to stackelberg games: addressing bounded rationality and limited observations in human cognition. Artif. Intell. J. 174(15), 1142–1171 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  21. Pita, J., John, R., Maheswaran, R., Tambe, M., Kraus, S.: A robust approach to addressing human adversaries in security games. In: European Conference on Artificial Intelligence (ECAI), pp. 660–665 (2012)

    Google Scholar 

  22. Shieh, E., An, B., Yang, R., Tambe, M., Baldwin, C., Direnzo, J., Meyer, G., Baldwin, C.W., Maule, B.J., Meyer, G.R.: PROTECT : a deployed game theoretic system to protect the ports of the United States. In: AAMAS, pp. 13–20 (2012)

    Google Scholar 

  23. Tambe, M.: Security and Game Theory: Algorithms, Deployed Systems, Lessons Learned. Cambridge University Press, Cambridge (2011)

    Book  Google Scholar 

  24. Tsai, J., Rathi, S., Kiekintveld, C., Ordóñez, F., Tambe, M.: IRIS - a tools for strategic security allocation in transportation networks. In: AAMAS, pp. 37–44 (2009)

    Google Scholar 

  25. Tsai, J., Yin, Z., Kwak, J.-Y., Kempe, D., Kiekintveld, C., Tambe, M.: Urban security: game-theoretic resource allocation in networked physical domains. In: AAAI, pp. 881–886 (2010)

    Google Scholar 

  26. Yang, R., Ford, B., Tambe, M., Lemieux, A.: Adaptive resource allocation for wildlife protection against illegal poachers. In: AAMAS, pp. 453–460 (2014)

    Google Scholar 

  27. Yang, R., Kiekintvled, C., Ordonez, F., Tambe, M., John, R.: Improving resource allocation strategies against human adversaries in security games: an extended study. Artif. Intell. J. (AIJ) 195, 440–469 (2013)

    Article  MATH  Google Scholar 

  28. Yin, Z., Jain, M., Tambe, M., Ordonez, F.: Risk-averse strategies for security games with execution and observational uncertainty. In: AAAI, pp. 758–763 (2011)

    Google Scholar 

  29. Yin, Z., Korzhyk, D., Kiekintveld, C., Conitzer, V., Tambe, M.: Stackelberg vs. nash in security games: interchangeability, equivalence, and uniqueness. In: AAMAS, pp. 1139–1146 (2010)

    Google Scholar 

  30. Zhang, C., Sinha, A., Tambe, M.: Keeping pace with criminals: designing patrol allocation against adaptive opportunistic criminals. In: AAMAS, pp. 1351–1359 (2015)

    Google Scholar 

Download references

Acknowledgements

This research was supported by the Office of Naval Research Global (grant no. N62909-13-1-N256).

Author information

Authors and Affiliations

  1. Department of Computer Science, FEE, Czech Technical University in Prague, Prague, Czech Republic

    Richard Klíma & Viliam Lisý

  2. Department of Computing Science, University of Alberta, Edmonton, Canada

    Viliam Lisý

  3. Computer Science Department, University of Texas at El Paso, El Paso, USA

    Christopher Kiekintveld

  4. Department of Computer Science, University of Liverpool, Liverpool, UK

    Richard Klíma

Authors
  1. Richard Klíma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Viliam Lisý
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Christopher Kiekintveld
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Richard Klíma .

Editor information

Editors and Affiliations

  1. Queen Mary University of London, London, United Kingdom

    MHR Khouzani

  2. University of Brighton, Brighton, United Kingdom

    Emmanouil Panaousis

  3. Cardiff University, Cardiff, United Kingdom

    George Theodorakopoulos

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Klíma, R., Lisý, V., Kiekintveld, C. (2015). Combining Online Learning and Equilibrium Computation in Security Games. In: Khouzani, M., Panaousis, E., Theodorakopoulos, G. (eds) Decision and Game Theory for Security. GameSec 2015. Lecture Notes in Computer Science(), vol 9406. Springer, Cham. https://doi.org/10.1007/978-3-319-25594-1_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-25594-1_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-25593-4

  • Online ISBN: 978-3-319-25594-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation