Skip to main content
SpringerLink
Log in

Building Conversational Agents for Military Training: Towards a Virtual Wingman

  • Conference paper
  • First Online:
Artificial Intelligence in HCI (HCII 2021)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 12797))

Included in the following conference series:

  • 2609 Accesses

Abstract

In military training simulations there is an increasing need for synthetic agents as team members that are capable of replacing human role players. Human-agent interaction using spoken natural language is an important capability for such agents, especially when communication skills are part of training. Interaction technologies such as speech-to-text, text-to-speech, natural language understanding and dialogue management are maturing, though their use in human-in-the-loop simulations is scarce. In this paper we address design challenges for building conversational agents in the domain of military training. In this domain, agents often have to adhere to specific communication standards, protocols and use of jargon. We propose a data-driven design method to tailor conversational agents to a particular application domain, while minimizing human authoring. We demonstrate this method within an existing military training simulation capability and show that agents are able to effectively build shared situational awareness and coordinated their activities in a shared task environment.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    DialogFlow was only used for intent classification, as its features for dialogue management are now handled by the system’s dialogue manager.

References

  1. Dompke, U.: Computer Generated Forces - Background, Definition and Basic Technologies. TNO, The Hague (2003)

    Google Scholar 

  2. Hancock, P., Vincenzi, D., Wise, J., Mouloua, M.: Human Factors in Simulation and Training. CRC Press (2008)

    Google Scholar 

  3. Ball, J., et al.: The synthetic teammate project. Comput. Math. Organ. Theory 16(3), 271–299 (2010)

    Article  Google Scholar 

  4. Andersson, J., Svensson, J.: Speech acts, communication problems, and fighter pilot team performance. Ergonomics 49(12–13), 1226–1237 (2006)

    Google Scholar 

  5. Noyes, J.M., Haas, E.: Military applications: human factors aspects of speech-based systems. In: Chen, F., Jokinen, K. (eds.) Speech Technology, pp. 251–270. Springer, New York (2010). https://doi.org/10.1007/978-0-387-73819-2_13

    Chapter  Google Scholar 

  6. Jokinen, K.: Spoken language dialogue models. In: Chen, F., Jokinen, K. (eds.) Speech Technology: Theory and Applications, pp. 33–60. Springer, New York (2010) https://doi.org/10.1007/978-0-387-73819-2_3

  7. Harms, J.G., Kucherbaev, P., Bozzon, A., Houben, G.J.: Approaches for dialog management in conversational agents. IEEE Internet Comput. 23, 13–22 (2019)

    Article  Google Scholar 

  8. Perez-Martin, D., Pascual-Nieto, I.: Conversational Agents and Natural Language Interaction: Techniques and Effective Practices, Information Science Reference - Imprint of. IGI Publishing (2011)

    Google Scholar 

  9. Kerry, A., Ellis, R., Bull, S.: Conversational agents in E-Learning. In: Allen, T., Ellis, R., Petridis, M. (eds.) Applications and Innovations in Intelligent Systems XVI, pp. 169–182. Springer, London (2009) https://doi.org/10.1007/978-1-84882-215-3_13

  10. Klopfenstein, L., Delpriori, S., Malatini, S, Bogliolo, A.: The rise of bots: a survey of conversational interfaces, patterns, and paradigms. In: Proceedings of the 2017 Conference on Designing Interactive Systems (2017)

    Google Scholar 

  11. Gatt, A., Krahmer, E.: Survey of the state of the art in natural language generation: core tasks, applications and evaluations. J. Artif. Intell. Res. 61, 65–170 (2018)

    Article  MathSciNet  Google Scholar 

  12. Graesser, A.C., Wiemer-Hastings, K., Wiemer-Hastings, P., Kreuz, R.: AutoTutor: a simulation of a human tutor. Cogn. Syst. Res. 1(1), 35–51 (1999)

    Article  Google Scholar 

  13. Talbot, T.R.A.: Virtual human standardized patients. In: Rizzo, A., Bouchard, S. (eds.) Virtual Reality for Psychological and Neurocognitive Interventions, pp. 387–405. Springer, New York (2019) https://doi.org/10.1007/978-1-4939-9482-3_17

  14. Hart, J., Gratch, J., Marsella, S.: How virtual reality training can win friends and influence people. In: Human Factors in Defence, vol. 21, no. 1, pp. 235-249. Ashgate, Farnham (2013)

    Google Scholar 

  15. van Oijen, J., van Doesburg, W., Dignum, F.: Goal-based communication using BDI agents as virtual humans in training: an ontology driven dialogue system. In: Dignum, F. (ed.) AGS 2010. LNCS (LNAI), vol. 6525, pp. 38–52. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-18181-8_3

    Chapter  Google Scholar 

  16. Graesser, A.C., Li, H., Forsyth, C.: Learning by communicating in natural language with conversational agents. Curr. Dir. Psychol. Sci. 23(5), 374–380 (2014)

    Article  Google Scholar 

  17. Traum, D., Rickel, J.: Embodied agents for multi-party dialogue in immersive virtual worlds. In: Proceedings of the First International Joint Conference on Autonomous Agents and Multiagent Systems: Part 2 (2002)

    Google Scholar 

  18. Kenny, P., et al.: Building interactive virtual humans for training environments. In: Proceedings of the Interservice/Industry Training, Simulation and Education Conference (I/ITSEC) (2007)

    Google Scholar 

  19. Chen, H., Liu, X., Yin, D., Tang, J.: A survey on dialogue systems: recent advances and new frontiers. ACM SIGKDD Explor. Newsl 19(2), 25–35 (2017)

    Article  Google Scholar 

  20. Grice, H.: Logic and conversation. In: Studies in Syntax and Semantics III: Speech Acts, pp. 41--58 (1975)

    Google Scholar 

  21. Traum, D., Swartout, W., Gratch, J., Marsella, S.: A virtual human dialogue model for non-team interaction. In: Dybkjær, L., Minker, W. (eds.) Recent Trends in Discourse and Dialogue. Text, Speech and Language Technology, vol. 39. Springer, Dordrecht (2008) https://doi.org/10.1007/978-1-4020-6821-8_3

  22. AFTTP 3–1: Operational Brevity Words, Definitions, and Counter Air Communication Standards (2001)

    Google Scholar 

  23. Hadzic, M., Wongthongtham, P., Dillon, T., Chang, E.: Ontology-Based Multi-Agent System. Springer, New York (2009). https://doi.org/10.1007/978-3-642-01904-3

    Book  Google Scholar 

  24. Braun, D., Hernandez, M., Matthes, F., Langen, M.: Evaluating natural language understanding services for conversational question answering systems. In: Proceedings of the 18th Annual {SIG}dial Meeting on Discourse and Dialogue, Saarbrücken (2017)

    Google Scholar 

  25. Walker, M., et al.: DARPA communicator dialog travel planning systems. In: 7th European Conference on Speech Communication and Technology, Aalborg, Denmark (2001)

    Google Scholar 

  26. Möller, S.: Assessment and evaluation of speech-based interactive systems: from manual annotation to automatic usability evaluation. In: Speech Technology: Theory and Applications, Boston, MA, pp. 301–322S (2010

    Google Scholar 

  27. Affolter, K., Stockinger, K., Bernstein, A.: A comparative survey of recent natural language interfaces for databases. VLDB J. 28(5), 793–819 (2019). https://doi.org/10.1007/s00778-019-00567-8

    Article  Google Scholar 

  28. Han, Y.J., Park, S.B., Park, S.Y.: A natural language interface concordant with a knowledge base. Comput. Intell. Neurosci. 2016, 1–15 (2016). https://doi.org/10.1155/2016/9174683

    Article  Google Scholar 

  29. Amershi, S., et al.: Guidelines for human-AI interaction. In: Proceedings of the 2019 HCI Conference on Human Factors in Computing Systems, Glasgow (2019)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Royal Netherlands Aerospace Centre, Amsterdam, the Netherlands

    Joost van Oijen & Olivier Claessen

Authors
  1. Joost van Oijen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Olivier Claessen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Joost van Oijen .

Editor information

Editors and Affiliations

  1. Siemens Corporation, Princeton, NJ, USA

    Helmut Degen

  2. Foundation for Research and Technology – Hellas (FORTH), Heraklion, Greece

    Stavroula Ntoa

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

van Oijen, J., Claessen, O. (2021). Building Conversational Agents for Military Training: Towards a Virtual Wingman. In: Degen, H., Ntoa, S. (eds) Artificial Intelligence in HCI. HCII 2021. Lecture Notes in Computer Science(), vol 12797. Springer, Cham. https://doi.org/10.1007/978-3-030-77772-2_34

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-77772-2_34

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-77771-5

  • Online ISBN: 978-3-030-77772-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation