Abstract
The Dynamic Data Driven Applications System (DDDAS) paradigm incorporates forward estimation with inverse modeling, augmented with contextual information. For cooperative infrared (IR) and radio-frequency (RF) based automatic target detection and recognition (ATR) systems, advantages of multimodal sensing and machine learning (ML) enhance real-time object detection and geolocation from an unmanned aerial vehicle (UAV). Using an RF subsystem, including the linear frequency modulated continuous wave (LFMCW) ranging radar and the smart antenna, line-of-sight (LOS) and non-line-of-sight (NLOS) friendly objects are detected and located. The IR subsystem detects and locates all human objects in a LOS scenario providing safety alerts to humans entering hazardous locations. By applying a ML-based object detection algorithm, i.e., the YOLO detector, which was specifically trained with IR images, the subsystem could detect humans that are 100 m away. Additionally, the DDDAS-inspired multimodal IR and RF (MIRRF) system discriminates LOS friendly and non-friendly objects. The whole MIRRF sensor system meets the size, weight, power, and cost (SWaP-C) requirement of being installed on the UAVs. Results of ground testing integrated with an all-terrain robot, the MIRRF sensor system demonstrated the capability of fast detection of humans, discrimination of friendly and non-friendly objects, and continuously tracked and geo-located the objects of interest.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Perera, F., Al-Naji, A, Law, Y., Chahl, J.: Human detection and motion analysis from a quadrotor UAV. IOP Conf. Ser.: Mater. Sci. Eng., 405, 012003, (2018)
Rudol, P., Doherty, P.: Human body detection and geolocalization for uav search and rescue missions using color and thermal imagery. In: Aerospace Conference, IEEE, pp. 1–8, (2008)
Andriluka, M., et al.: Vision based victim detection from unmanned aerial vehicles. Intelligent Robots and Systems (IROS), IEEE/RSJ International Conference on, pp. 1740–1747 (2010)
Oreifej, O., Mehran, R., Shah, M.: Human identity recognition in aerial images. Computer Vision and Pattern Recognition (CVPR). In: IEEE Conference on, pp 709–716, (2010)
Yeh, M., Chiu, H., Wang, J.: Fast medium-scale multiperson identification in aerial videos. Multimedia Tools Appl. 75, 16117–16133 (2016)
Monajjemi, M., Bruce, J., Sadat, S., Wawerla, J., Vaughan, R.: UAV, do you see me? Establishing mutual attention between an uninstrumented human and an outdoor UAV in flight. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3614–3620, (2015)
Wang, C., Bochkovskiy, A., Liao, H.: Scaled-YOLOv4: scaling cross stage partial network. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 13029–13038, (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Cheng, P., Lin, X., Zhang, Y., Blasch, E., Chen, G. (2024). Multimodal IR and RF Based Sensor System for Real-Time Human Target Detection, Identification, and Geolocation. In: Blasch, E., Darema, F., Aved, A. (eds) Dynamic Data Driven Applications Systems. DDDAS 2022. Lecture Notes in Computer Science, vol 13984. Springer, Cham. https://doi.org/10.1007/978-3-031-52670-1_20
Download citation
DOI: https://doi.org/10.1007/978-3-031-52670-1_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-52669-5
Online ISBN: 978-3-031-52670-1
eBook Packages: Computer ScienceComputer Science (R0)