Skip to main content
SpringerLink
Log in

Deep AI military staff: cooperative battlefield situation awareness for commander’s decision making

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

There are many studies adopting artificial intelligence (AI) to develop core technologies for the future army but they are still at the level of basic research. It is expected that military power will be negatively affected by aging and declining population. In addition, as more than 500,000 agents will be dispatched to monitor combat scenes, the data sensed by each agent should be managed simultaneously recognize and evaluate the situation on the battlefield in real time. Despite increased complexity in the battlefield, current command system entirely rely on the experience and expertise of individual commanders, which severely restricts defense capabilities. Therefore, AI based military staff needs to be developed to identify potential threats that commanders are likely to miss, to develop smart command systems, and to provide data-driven rationale for commander’s decisions. In this paper, we propose a deep AI military staff to support commander decision-making. Our proposed model is composed of four key parts: multi-agent based manned-unmanned collaboration architecture (MACA), robust tactical map fusion technology in poor environments (RTMF), hypergraph based representation learning (HRL) and space-time multi layer model for battlefields recognition (STBR). We design an architecture and generate dataset for training the core network. Simulation results are provided to demonstrate the performance of Deep AI military staff.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

Data availability

All data generated or analysed during this study are included in this published article. For more information on datasets, please see sections. 9.1.2 and 9.3.1.

References

  1. Weiss J (2020) The army’s ultimate heads-up display: The ivas. https://sofrep.com/news/the-armys-ultimateheads-up-display-the-ivas/

  2. DARPA: DARPA demonstrates “Competition” tool at combatant command (2020). https://www.darpa.mil/news-events/2020-03-19a

  3. DARPA: Active interpretation of disparate alternatives (AIDA) (2017). https://www.darpa.mil/newsevents/

  4. Kirillov A, He K, Girshick R, Rother C, Dollár P (2019) Panoptic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9404–9413

  5. Lin T-Y, Dollár P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2117–2125

  6. de Geus D, Meletis P, Dubbelman G (2020) Fast panoptic segmentation network. IEEE Robot Autom Lett 5(2):1742–1749

    Article  Google Scholar 

  7. Kirillov A, Girshick R, He K, Dollár P (2019) Panoptic feature pyramid networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6399–6408

  8. Li Y, Chen X, Zhu Z, Xie L, Huang G, Du D, Wang X (2019) Attention-guided unified network for panoptic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7026–7035

  9. Xiong Y, Liao R, Zhao H, Hu R, Bai M, Yumer E, Urtasun R (2019) Upsnet: A unified panoptic segmentation network. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8818–8826

  10. He K, Gkioxari G, Dollár P, Girshick R (2017) Mask r-cnn. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2961–2969

  11. Gao N, Shan Y, Wang Y, Zhao X, Yu Y, Yang M, Huang K (2019) Ssap: Single-shot instance segmentation with affinity pyramid. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 642–651

  12. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C-Y, Berg AC (2016) Ssd: Single shot multibox detector. In: European Conference on Computer Vision, pp. 21–37. Springer

  13. Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: Unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788

  14. Barnes C, Shechtman E, Finkelstein A, Goldman DB (2009) Patchmatch: a randomized correspondence algorithm for structural image editing. ACM Trans Gr 28(3):24

    Article  Google Scholar 

  15. Bertalmio M, Sapiro G, Caselles V, Ballester C (2000) Image inpainting. In: Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, pp. 417–424

  16. Satoshi I, Edgar S-S, Hiroshi I (2017) Globally and locally consistent image completion. ACM Trans Gr 36(4):3073659

    Google Scholar 

  17. Liu G, Reda FA, Shih KJ, Wang T-C, Tao A, Catanzaro B (2018) Image inpainting for irregular holes using partial convolutions. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 85–100

  18. Pathak D, Krahenbuhl P, Donahue J, Darrell T, Efros AA (2016) Context encoders: Feature learning by inpainting. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2536–2544

  19. Perozzi B, Al-Rfou R, Skiena S (2014) Deepwalk: Online learning of social representations. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 701–710

  20. Grover A, Leskovec J (2016) node2vec: Scalable feature learning for networks. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 855–864

  21. Dong Y, Chawla NV, Swami A (2017) metapath2vec: Scalable representation learning for heterogeneous networks. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 135–144

  22. Veličković P, Cucurull G, Casanova A, Romero A, Lió P, Bengio Y (2018) Graph Attention networks

  23. Ying R, You J, Morris C, Ren X, Hamilton WL, Leskovec J (2018) Hierarchical graph representation learning with differentiable pooling. http://arxiv.org/abs/1806.08804

  24. Zhang M, Cui Z, Neumann M, Chen Y (2018) An end-to-end deep learning architecture for graph classification. In: Thirty-Second AAAI Conference on Artificial Intelligence

  25. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778

  26. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 25:1097–1105

    Google Scholar 

  27. Kim C, Li F, Ciptadi A, Rehg JM (2015) Multiple hypothesis tracking revisited. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4696–4704

  28. Girshick R (2015) Fast r-cnn. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1440–1448

  29. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440

  30. Bolya D, Zhou C, Xiao F, Lee YJ (2019) Yolact: Real-time instance segmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9157–9166

  31. Zheng C, Cham T-J, Cai J (2019) Pluralistic image completion. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1438–1447

  32. Ji R, Chen F, Cao L, Gao Y (2018) Cross-modality microblog sentiment prediction via bi-layer multimodal hypergraph learning. IEEE Trans Multim 21(4):1062–1075

    Article  Google Scholar 

  33. Feng Y, You H, Zhang Z, Ji R, Gao Y (2019) Hypergraph neural networks. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 3558–3565

  34. Bai S, Zhang F, Torr PH (2021) Hypergraph convolution and hypergraph attention. Pattern Recognit 110:107637

    Article  Google Scholar 

  35. Pohlen T, Hermans A, Mathias M, Leibe B (2017) Full-resolution residual networks for semantic segmentation in street scenes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4151–4160

  36. Krishna R, Zhu Y, Groth O, Johnson J, Hata K, Kravitz J, Chen S, Kalantidis Y, Li L-J, Shamma DA, Bernstein M, Fei-Fei L (2016) Visual genome: Connecting language and vision using crowdsourced dense image annotations. https://arxiv.org/abs/1602.07332

  37. Duan K, Bai S, Xie L, Qi H, Huang Q, Tian Q (2019) Centernet: Keypoint triplets for object detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6569–6578

  38. Mikolov T, Chen K, Corrado G, Dean J (2013) Efficient estimation of word representations in vector space. http://arxiv.org/abs/1301.3781

  39. Wu Y, He K (2018) Group normalization. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 3–19

  40. Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. http://arxiv.org/abs/1412.6980

  41. Cordts M, Omran M, Ramos S, Rehfeld T, Enzweiler M, Benenson R, Franke U, Roth S, Schiele B (2016) The cityscapes dataset for semantic urban scene understanding. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3213–3223

  42. Hariharan B, Arbeláez P, Girshick R, Malik J (2014) Simultaneous detection and segmentation. In: European Conference on Computer Vision, pp. 297–312. Springer

  43. Yang T-J, Collins MD, Zhu Y, Hwang J-J, Liu T, Zhang X, Sze V, Papandreou G, Chen L-C (2019) Deeperlab: Single-shot image parser. http://arxiv.org/abs/1902.05093

Download references

Acknowledgements

This work was supported by Electronics and Telecommunications Research Institute (ETRI) grant funded by the Korean government (21YR1610, Research on Spatio-temporal Multi-Layer Battlefield Situation Awareness for AI Military Staff) and Korea Research Institute for Defense Technology planning and advancement (KRIT) grant funded by the Korea government DAPA(Defense Acquisition Program Administration) (No. 20-107-C00-008-02, Control technology for collective operation of military ultra-small ground robots).

Author information

Authors and Affiliations

  1. Electronics and Telecommunications Research Institute (ETRI), Daejeon, Republic of Korea

    Chang-Eun Lee & Jaeuk Baek

  2. Gwangju Institute of Science and Technology (GIST), Gwangju, Republic of Korea

    Jeany Son

  3. Konkuk University, Seoul, Republic of Korea

    Young-Guk Ha

Authors
  1. Chang-Eun Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jaeuk Baek
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jeany Son
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Young-Guk Ha
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Young-Guk Ha.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lee, CE., Baek, J., Son, J. et al. Deep AI military staff: cooperative battlefield situation awareness for commander’s decision making. J Supercomput 79, 6040–6069 (2023). https://doi.org/10.1007/s11227-022-04882-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-022-04882-w

Keywords

Search

Navigation