Abstract
Potential obstacles on landing sites are becoming an increasing problem for aircraft landings, especially during challenging visual conditions for the pilot in command, and during fully autonomous landing operations. A novel vision based autonomous detection scheme has been proposed to allow the detection of obstacles on the runway. Using Detectron2 as the detection network, accurate detection and segmentation of runways and obstacles have been achieved all with over 95% accuracy. Network training has been done using transfer learning and over 1500 custom image dataset, achieving 73% average precision. Several modules have been developed for live data acquisition, pre-processing the data, which is then processed through the trained network. Data is then post-processed to be passed on to the custom decision-making algorithm, which outputs a decision based on obstacle location and its states. The complete network has been tested in a virtual, as well as real life environment, achieving promising results.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Kang, J., et al.: Safety and workload assessment of lost C2 link on Seoul-jeju route. J. Aerosp. Inf. Syst. 16(4), 120–131 (2019). https://doi.org/10.2514/1.I010670
Ion, A., Mitici, M.A., Delft, T.U., Lissone, M., Zarouchas, D.: Impact of ‘Lost C2 Link’ on Key ATM Performance Indicators in a Mixed RPAS-Manned Aircraft Operational Environment. http://repository.tudelft.nl/
Jbara, K.A.: A robust vision-based runway detection and tracking algorithm for automatic UAV landing (2015)
Wenkel, S., Alhazmi, K., Liiv, T., Alrshoud, S., Simon, M.: Confidence score: the forgotten dimension of object detection performance evaluation. Sensors 21(13), 4350 (2021). https://doi.org/10.3390/s21134350
Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: towards real-time object detection with region proposal networks. https://github.com/
Lin, T.-Y., Dollár, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection (2017)
Ackermann, M., Iren, D., Wesselmecking, S., Shetty, D., Krupp, U.: Automated segmentation of martensite-austenite islands in bainitic steel. Mater. Charact. 191, 112091 (2022). https://doi.org/10.1016/j.matchar.2022.112091
Paszke, A., et al.: PyTorch: an imperative style, high-performance deep learning library (2019)
Wu, Y., Kirillov, A., Massa, F., Lo, W.-Y., Girshick, R.: Detectron2 (2020)
Roboflow. https://roboflow.com/. Accessed 17 Aug 2022
Agrawal, S.: How to split data into three sets (Train, Validation and Test) and why? (2021). https://towardsdatascience.com/how-to-split-data-into-three-sets-train-validation-and-test-and-why-e50d22d3e54c. Accessed 17 Aug 2022
Eidos AI. Training on Detectron2 with a validation set, and plot loss on it to avoid overfitting (2020). https://eidos-ai.medium.com/training-on-detectron2-with-a-validation-set-and-plot-loss-on-it-to-avoid-overfitting-6449418fbf4e. Accessed 23 Aug 2022
Facebook AI Research. Detectron2 documentation - evaluation. https://detectron2.readthedocs.io/en/latest/tutorials/evaluation.html. Accessed 23 Aug 2022
Draelos, R.: The complete guide to AUC and average precision: simulations and visualizations (2020). https://glassboxmedicine.com/2020/07/14/the-complete-guide-to-auc-and-average-precision-simulations-and-visualizations/#:~:text=Average%20precision%20ranges%20from%20the,frequency%20of%20positives%20%3D%200.5. Accessed 21 Aug 2022
Google Developers - Machine Learning. Descending into ML: Training and Loss. Google Developers (2022). https://developers.google.com/machine-learning/crash-course/descending-into-ml/training-and-loss. Accessed 21 Aug 2022
Acknowledgements
I would like to thank Dr Yang Xing, Professor Hyo-Sang Shin from Cranfield University and Mr Timothy Bleakley from General Atomics Aeronautical Systems for their involvement and constant support provided throughout this research project. Special thanks to Dr Oscar Julian Gonzalez Villarreal, to always serve as someone to whom I could freely talk to and share very interesting conversations with during my time at Cranfield University, but also as a source of immense and unlimited ideas, solutions, and support. A quick mention to Dr Argyrios Zolotas as not only a great lecturer but also as the head of the course, always keen to help and advise students such as myself. Last but not least, it is important to mention my sheer appreciation towards my family. I am grateful beyond limits to my parents, Kamal and Reena as well as my sister, Tajmin who have supported me on all my decision and have always been there for me. Providing support, advice and for always believing in me my whole life, but specially these past 5 years I have spent away from home to pursue my passion in the field of Aerospace Engineering. Muchas Gracias por Todo.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Tajmilur Khemlani, T.R., Xing, Y., Shin, HS. (2023). Development and Feasibility Study of an Autonomous Obstacle Detection System for Landing Operations. In: Jo, J., et al. Robot Intelligence Technology and Applications 7. RiTA 2022. Lecture Notes in Networks and Systems, vol 642. Springer, Cham. https://doi.org/10.1007/978-3-031-26889-2_17
Download citation
DOI: https://doi.org/10.1007/978-3-031-26889-2_17
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-26888-5
Online ISBN: 978-3-031-26889-2
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)