Skip to main content

Advertisement

Springer Nature Link
Log in

On the performance evaluation of object classification models in low altitude aerial data

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

This paper compares the classification performance of machine learning classifiers vs. deep learning-based handcrafted models and various pretrained deep networks. The proposed study performs a comprehensive analysis of object classification techniques implemented on low-altitude UAV datasets using various machine and deep learning models. Multiple UAV object classification is performed through widely deployed machine learning-based classifiers such as K nearest neighbor, decision trees, naïve Bayes, random forest, a deep handcrafted model based on convolutional layers, and pretrained deep models. The best result obtained using random forest classifiers on the UAV dataset is 90%. The handcrafted deep model's accuracy score suggests the efficacy of deep models over machine learning-based classifiers in low-altitude aerial images. This model attains 92.48% accuracy, which is a significant improvement over machine learning-based classifiers. Thereafter, we analyze several pretrained deep learning models, such as VGG-D, InceptionV3, DenseNet, Inception-ResNetV4, and Xception. The experimental assessment demonstrates nearly 100% accuracy values using pretrained VGG16- and VGG19-based deep networks. This paper provides a compilation of machine learning-based classifiers and pretrained deep learning models and a comprehensive classification report for the respective performance measures.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Availability of data and material

Applicable on request.

Code availability

Applicable on request.

References

  1. Mohanan MG, Salgoankar A (2018) A survey of robotic motion planning in dynamic environments. J Robot Auton Syst 100:171–185

    Article  Google Scholar 

  2. Fernandes D, Silva A, Névoa R, Simões C, Gonzalez D, Guevara M, Novais P, Monteiro J, Melo-Pinto P (2021) Point-cloud based 3D object detection and classification methods for self-driving applications: a survey and taxonomy. Inf Fusion 68:161–191

    Article  Google Scholar 

  3. Tzelepi M, Tefas A (2017) Human crowd detection for drone flight safety using convolutional neural networks. In: 25th European Signal Processing Conference (EUSIPCO), pp 743–747

  4. Li L, Du B, Wang Y, Qin L, Tan H (2020) Estimation of missing values in heterogeneous traffic data: Application of multimodal deep learning model. Knowl Based Syst 194:105592

    Article  Google Scholar 

  5. Du Terrail JO, Jurie F (2017) On the use of deep neural networks for the detection of small vehicles in ortho-images. In: IEEE International Conference on Image Processing (ICIP), pp 4212–4216

  6. Huang T, Wang S, Sharma A (2020) Highway crash detection and risk estimation using deep learning. Accid Anal Prev 135:105392

    Article  Google Scholar 

  7. Ma J, Li W, Jia C, Zhang C, Zhang Y (2020) Risk prediction for ship encounter situation awareness using long short-term memory based deep learning on intership behaviors. J Adv Trans 2020:8897700. https://doi.org/10.1155/2020/8897700

    Article  Google Scholar 

  8. Ren S, Choi TM, Lee KM, Lin L (2020) Intelligent service capacity allocation for cross-border-E-commerce related third-party-forwarding logistics operations: a deep learning approach. Transport Res Part E Logist Transport Rev 134:101834

    Article  Google Scholar 

  9. Seema S, Goutham S, Vasudev S, Putane RR (2020) Deep learning models for analysis of traffic and crowd management from surveillance videos. In: Progress in computing, analytics and networking. Springer, Singapore, pp 83–93

  10. Tang W, Yang Q, Xiong K, Yan W (2020) Deep learning based automatic defect identification of photovoltaic module using electroluminescence images. Sol Energy 201:453–460

    Article  Google Scholar 

  11. Gaszczak A, Breckon TP, Han J (2011) Real-time people and vehicle detection from UAV imagery. In: Intelligent robots and computer vision XXVIII: algorithms and techniques, vol 7878, p 78780B

  12. Singh A, Patil D, Omkar SN (2018) Eye in the sky: real-time drone surveillance system (DSS) for violent individuals identification using scatternet hybrid deep learning network. http://arxiv.org/abs/1806.00746

  13. Wang J, Guo W, Pan T, Yu H, Duan L, Yang W (2018) Bottle detection in the wild using low-altitude unmanned aerial vehicles. In: 2018 21st International Conference on Information Fusion (FUSION), pp 439–444

  14. Varghese A, Gubbi J, Sharma H, Balamuralidhar P (2017) Power infrastructure monitoring and damage detection using drone captured images. In: 2017 International Joint Conference on Neural Networks (IJCNN), pp 1681–1687

  15. Amarasinghe A, Suduwella C, Elvitigala C, Niroshan L, Amaraweera RJ, Gunawardana K, Kumarasinghe P, De Zoysa K, Keppetiyagama C (2017) A machine learning approach for identifying mosquito breeding sites via drone images. In: Proceedings of the 15th ACM Conference on Embedded Network Sensor Systems, p 68

  16. Li Z, Shi W, Lu P, Yan L, Wang Q, Miao Z (2016) Landslide mapping from aerial photographs using change detection-based Markov random field. J Remote Sens Environ 187:76–90

    Article  Google Scholar 

  17. Quinlan JR (1986) Induction of decision trees. Mach Learn 1(1):81–106

    Article  Google Scholar 

  18. Breiman L (2001) Random forests. Mach Learn 45(1):5–32

    Article  Google Scholar 

  19. Kohavi R (1996) Scaling up the accuracy of naive-bayes classifiers: a decision-tree hybrid. Kdd 96:202–207

    Google Scholar 

  20. Li X (2007) Conference 9247: High-Performance Computing in Remote Sensing, Remote Sens. Secur. Def. Technol., p 188

  21. Xu Y, Yu G, Wang Y, Wu X, Ma Y (2016) A hybrid vehicle detection method based on viola-jones and HOG+ SVM from UAV images. Sensors 16(8):1325

    Article  Google Scholar 

  22. Sugimura D, Fujimura T, Hamamoto T (2016) Enhanced cascading classifier using multi-scale HOG for pedestrian detection from aerial images. Int J Pattern Recognit Artif Intell 30(03):1655009

    Article  MathSciNet  Google Scholar 

  23. Mizuno K, Terachi Y, Takagi K, Izumi S, Kawaguchi H, Yoshimoto M (2012) Architectural study of HOG feature extraction processor for real-time object detection. In: 2012 IEEE workshop on signal processing systems. IEEE, pp 197–202

  24. Neumann J, Schnörr C, Steidl G (2005) Combined SVM-based feature selection and classification. Mach Learn 61(1–3):129–150

    Article  Google Scholar 

  25. Teutsch M, Krüger W, Beyerer J (2014) Evaluation of object segmentation to improve moving vehicle detection in aerial videos. In: 11th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pp 265–270

  26. Moranduzzo T, Melgani F (2014) Automatic car counting method for unmanned aerial vehicle images. IEEE Transactions on Geoscience Remote Sensing 52(3):1635–1647

    Article  Google Scholar 

  27. Xu B, Xu X, Own C-M (2017) On the feature detection of nonconforming objects with automated drone surveillance. In: Proceedings of the 3rd International Conference on Communication and Information Processing, pp 484–489

  28. Fernández-Delgado M, Cernadas E, Barro S, Amorim D (2014) Do we need hundreds of classifiers to solve real world classification problems? J Mach Learn Res 15(1):3133–3181

    MathSciNet  MATH  Google Scholar 

  29. Lindeberg T (2012) Scale invariant feature transform, pp 10491

  30. Zhao B, Feng J, Wu X, Yan S (2017) A survey on deep learning-based fine-grained object classification and semantic segmentation. Int J Autom Comput 14:119–135

    Article  Google Scholar 

  31. Baykara HC, Biyik E, Gül G, Onural D, Öztürk AS, Yildiz I (2017) Real-time detection, tracking and classification of multiple moving objects in UAV videos. In IEEE 29th International Conference on Tools with Artificial Intelligence (ICTAI), pp 945–950

  32. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105

  33. Liang Y, Monteiro ST, Saber ES (2016) Transfer learning for high resolution aerial image classification. In: Applied imagery pattern recognition workshop (AIPR). IEEE, pp 1–8

  34. Wang SH, Zhou Q, Yang M, Zhang YD (2021) ADVIAN: Alzheimer’s disease VGG-inspired attention network based on convolutional block attention module and multiple way data augmentation. Front Aging Neurosci 13:313

    Google Scholar 

  35. Wang SH, Fernandes S, Zhu Z, Zhang YD (2021) AVNC: attention-based VGG-style network for COVID-19 diagnosis by CBAM. IEEE Sensors J

  36. Boonpook W, Tan Y, Xu B (2021) Deep learning-based multi-feature semantic segmentation in building extraction from images of UAV photogrammetry. Int J Remote Sens 42(1):1–19

    Article  Google Scholar 

  37. Kuchár D, Schreiber P (2021) Comparison of UAV landing site classifications with deep neural networks. In: Computer Science On-line Conference. Springer, Cham, pp 55–63

  38. Tian G, Liu J, Zhao H, Yang W (2022) Small object detection via dual inspection mechanism for UAV visual images. Appl Intell 52(4):4244–4257

    Article  Google Scholar 

  39. Bouguettaya A, Zarzour H, Kechida A, Taberkit AM (2021) Vehicle detection from UAV imagery with deep learning: a review. IEEE Transactions on Neural Networks and Learning Systems

  40. Kumar P, Ashtekar S, Jayakrishna SS, Bharath KP, Vanathi PT, Kumar MR (2021) Classification of mango leaves infected by fungal disease anthracnose using deep learning. In: 5th International conference on computing methodologies and communication (ICCMC). IEEE, pp 1723–1729

  41. Treneska S, Stojkoska BR (2021) Wildfire detection from UAV collected images using transfer learning

  42. Sommer L, Nie K, Schumann A, Schuchert T, Beyerer J (2017) Semantic labeling for improved vehicle detection in aerial imagery. In: 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pp 1–6

  43. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. http://arxiv.org/abs/1409.1556

  44. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1–9

  45. Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. http://arxiv.org/abs/1502.03167

  46. Ramachandran P, Zoph B, Le QV (2017) Searching for activation functions. http://arxiv.org/abs/1710.05941

  47. Szegedy C, Ioffe S, Vanhoucke V, Alemi AA (2017) Inception-v4, inception-resnet and the impact of residual connections on learning. In: Thirty-First AAAI Conference on Artificial Intelligence

  48. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  49. Iandola F, Moskewicz M, Karayev S, Girshick R, Darrell T, Keutzer K (2014) Densenet: implementing efficient convnet descriptor pyramids. http://arxiv.org/abs/ 1404.1869

  50. Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V (2011) Scikit-learn: machine learning in Python. J Mach Learn Res 12:2825–2830

    MathSciNet  MATH  Google Scholar 

  51. Bradski G, Kaehler A (2008) Learning OpenCV: computer vision with the OpenCV library. O'Reilly Media, Inc.

  52. Gulli A, Pal S (2017) Deep learning with Keras. Packt Publishing Ltd

  53. Abadi M, Barham P, Chen J, Chen Z, Davis A, Dean J, Devin M, Ghemawat S, Irving G, Isard M, Kudlur M (2016) Tensorflow: a system for large-scale machine learning. OSDI 16:265–283

    Google Scholar 

  54. Chollet F (2017) Xception: deep learning with depthwise separable convolutions. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

  55. Kirk D (2015) NVIDIA CUDA software and GPU parallel computing architecture. In: ISMM, vol 7

  56. Hsieh M-R, Lin Y-L, Hsu WH (2017) Drone-based object counting by spatially regularized regional proposal network. In: The IEEE International Conference on Computer Vision (ICCV), vol 1

  57. Barekatain M, Martí M, Shih HF, Murray S, Nakayama K, Matsuo Y, Prendinger H (2017) Okutama-action: an aerial view video dataset for concurrent human action detection. In: 1st Joint BMTT-PETS Workshop on Tracking and Surveillance, CVPR, pp 1–8

  58. Razakarivony S, Jurie F (2016) Vehicle detection in aerial imagery: a small target detection benchmark. J Vis Commun Image Represent 34:187–203

    Article  Google Scholar 

  59. Yoshihashi R, Kawakami R, Iida M, Naemura T (2015) Construction of a bird image dataset for ecological investigations. In: 2015 IEEE International Conference on Image Processing (ICIP), pp 4248–4252

  60. Caglayan A, Guclu O, Can AB (2013) A plant recognition approach using shape and color features in leaf images. In: International Conference on Image Analysis and Processing. Springer, Berlin, pp 161–170

  61. Kim BK, Kang HS, Park SO (2016) Drone classification using convolutional neural networks with merged Doppler images. IEEE Geosci Remote Sens Lett 14(1):38–42

    Article  Google Scholar 

  62. Yalcin, H. and Razavi, S (2016) Plant classification using convolutional neural networks. In: 2016 Fifth International Conference on Agro-Geoinformatics. IEEE, pp 1–5

  63. Yousefi E, Baleghi Y, Sakhaei SM (2017) Rotation invariant wavelet descriptors, a new set of features to enhance plant leaves classification. Comput Electron Agric 140:70–76

    Article  Google Scholar 

  64. Liu H, Qu F, Liu Y, Zhao W, Chen Y (2018) A drone detection with aircraft classification based on a camera array. IOP Conf Ser Mater Sci Eng 322(5):052005

    Article  Google Scholar 

  65. Kaya A, Keceli AS, Catal C, Yalic HY, Temucin H, Tekinerdogan B (2019) Analysis of transfer learning for deep neural network based plant classification models. Comput Electron Agric 158:20–29

    Article  Google Scholar 

  66. Yang W, Xu W, Wu C, Zhu B, Chen P, Zhang L, Lan Y (2021) Cotton hail disaster classification based on drone multispectral images at the flowering and boll stage. Comput Electron Agric 180:105866

    Article  Google Scholar 

Download references

Acknowledgements

We acknowledge DIC, Panjab University Chandigarh for funding a workstation that enabled us to perform experiments in the form of NVIDIA TITAN XP GPUs. This research work has been done under UGC NET SRF scholarship, New Delhi, India.

Funding

The authors did not receive support from any organization for the submitted work.

Author information

Authors and Affiliations

  1. UIET Panjab University, Chandigarh, India

    Payal Mittal & Akashdeep Sharma

  2. Thapar Institute of Engineering and Technology, Patiala, India

    Raman Singh

  3. School of Computer Science and Engineering, Vellore Institute of Technology, Vellore, India

    Arun Kumar Sangaiah

  4. Department of Industrial Engineering and Management, National Yunlin University of Science and Technology, Douliu, Taiwan

    Arun Kumar Sangaiah

Authors
  1. Payal Mittal
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Akashdeep Sharma
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Raman Singh
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Arun Kumar Sangaiah
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Arun Kumar Sangaiah.

Ethics declarations

Conflict of interest

The authors have no conflicts of interest to declare that are relevant to the content of this article.

Ethics approval

This paper does not contain any studies with human or animal subjects and all authors declare that they have no conflict of interest.

Consent to participate

All authors declare that they have the consent to participate.

Consent for publication

All authors declare that they have consent for publication.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mittal, P., Sharma, A., Singh, R. et al. On the performance evaluation of object classification models in low altitude aerial data. J Supercomput 78, 14548–14570 (2022). https://doi.org/10.1007/s11227-022-04469-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-022-04469-5

Keywords