Skip to main content
Springer Nature Link
Log in

Automatic detection of oil palm fruits from UAV images using an improved YOLO model

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Manual harvesting of loose fruits in the oil palm plantation is both time consuming and physically laborious. Automatic harvesting system is an alternative solution for precision agriculture which requires accurate visual information of the targets. Current state-of-the-art one-stage object detection method provides excellent detection accuracy; however, it is computationally intensive and impractical for embedded system. This paper proposed an improved YOLO model to detect oil palm loose fruits from unmanned aerial vehicle images. In order to improve the robustness of the detection system, the images are augmented by brightness, rotation, and blurring to simulate the actual natural environment. The proposed improved YOLO model adopted several improvements; densely connected neural network for better feature reuse, swish activation function, multi-layer detection to enhance detection on small targets and prior box optimization to obtain accurate bounding box information. The experimental results show that the proposed model achieves outstanding average precision of 99.76% with detection time of 34.06 ms. In addition, the proposed model is also light in weight size and requires less training time which is significant in reducing the hardware costs. The results exhibit the superiority of the proposed improved YOLO model over several existing state-of-the-art detection models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. MPOC: Malaysian Palm Oil Council, http://www.mpoc.org.my, accessed 15 September 2020

  2. Idrees, A.: Malaysia Palm Oil Industry, http://www.mpoc.org.my/Malaysian_Palm_Oil_Industry.aspx, accessed 15 September 2020

  3. Zhao, Y., Gong, L., Huang, Y., Liu, C.: A review of key techniques of vision-based control for harvesting robot. Comput. Electron. Agric. 127, 311–323 (2016)

    Article  Google Scholar 

  4. Mairon, R., Edan, Y.: Computer vision for fruit harvesting robots—state of the art and challenges ahead. Int. J. Comput. Vis. Robot. 3, 4–34 (2012)

    Article  Google Scholar 

  5. Yamamoto, K., Guo, W., Yoshioka, Y., Ninomiya, S.: On plant detection of intact tomato fruits using image analysis and machine learning methods. Sensors 14(7), 12191–12206 (2014)

    Article  Google Scholar 

  6. Maldonado, W., Barbosa, J.C.: Automatic green fruit counting in orange trees using digital images. Comput. Electron. Agric. 127, 572–581 (2016)

    Article  Google Scholar 

  7. Qureshi, W.S., Payne, A., Walsh, K.B., Linker, R., Cohen, O., Dailey, M.N.: Machine vision for counting fruit on mango tree canopies. Precis. Agric. 18, 224–244 (2016)

    Article  Google Scholar 

  8. Hamza, R., Chtourou, M.: Design of fuzzy inference system for apple ripeness estimation using gradient method. IET Image Process. 14, 561–569 (2020)

    Article  Google Scholar 

  9. Liu, L., Ouyang, W., Wang, X., Fieguth, P., Chen, J., Liu, X., Pietikäinen, M.: Deep learning for generic object detection: A survey. Int. J. Comput. Vis. 128, 261–318 (2020)

    Article  Google Scholar 

  10. Zhao, Z.Q., Zheng, P., Xu, S.T., Wu, X.: Object detection with deep learning: a review. IEEE Trans. Neural Netw. Learn. Syst. 30, 3212–3232 (2019)

    Article  Google Scholar 

  11. Jiao, L., Zhang, F., Liu, F., Yang, S., Li, L., Feng, Z., Qu, R.: A survey of deep learning-based object detection. IEEE Access. 7, 128837–128868 (2019)

    Article  Google Scholar 

  12. Girshick, R., Donahue, J., Darrell, T., Malik, J., Berkeley, U.C.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 580–587 (2014)

  13. Girshick, R.: Fast R-CNN. In: IEEE International Conference on Computer Vision Fast, pp. 1440–1448 (2015)

  14. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN : Towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 36, 1–14 (2017)

    Google Scholar 

  15. He, K., Gkioxari, G., Dollar, P., Girshick, R.: Mask R-CNN. In: Proceedings of the IEEE international conference on computer vision, pp. 2980–2988 (2017)

  16. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C., Berg, A.C.: SSD : single shot multibox detector. In: European Conference on Computer Vision, pp. 21–37 (2016)

  17. Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once : unified, real-time object detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788 (2016)

  18. Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: YOLO9000: Better, faster, stronger. In: IEEE conference on Computer Vision and Pattern Recognition, pp. 6517–6525 (2017)

  19. Redmon, J., Farhadi, A.: YOLOv3 : An incremental improvement. In: IEEE Conference on Computer Vision and Pattern Recognition (2018)

  20. Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollar, P.: Focal Loss for Dense Object Detection. In: IEEE transactions on pattern analysis and machine intelligence. pp. 318–327 (2020)

  21. Tan, M., Pang, R., Le, Q. V.: EfficientDet: Scalable and efficient object detection. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp. 10778–10787 (2020)

  22. Li, Y., Han, Z., Xu, H., Liu, L., Li, X., Zhang, K.: YOLOv3-lite: a lightweight crack detection network for aircraft structure based on depthwise separable convolutions. Appl. Sci. 9, 3781 (2019)

    Article  Google Scholar 

  23. Park, S.E., Eem, S.H., Jeon, H.: Concrete crack detection and quantification using deep learning and structured light. Constr. Build. Mater. 252, 119096 (2020)

    Article  Google Scholar 

  24. Xi, P., Guan, H., Shu, C., Borgeat, L., Goubran, R.: An integrated approach for medical abnormality detection using deep patch convolutional neural networks. Vis. Comput. 36, 1869–1882 (2020)

    Article  Google Scholar 

  25. Ozturk, T., Talo, M., Yildirim, E.A., Baloglu, U.B., Yildirim, O., Rajendra Acharya, U.: Automated detection of COVID-19 cases using deep neural networks with X-ray images. Comput. Biol. Med. 121, 103792 (2020)

    Article  Google Scholar 

  26. Villamizar, M., Sanfeliu, A., Moreno-Noguer, F.: Online learning and detection of faces with low human supervision. Vis. Comput. 35, 349–370 (2019)

    Article  Google Scholar 

  27. Chen, W., Huang, H., Peng, S., Zhou, C., Zhang, C.: YOLO-face: a real-time face detector. Vis. Comput. 37, 1–9 (2020)

    Google Scholar 

  28. Min, W., Li, X., Wang, Q., Zeng, Q., Liao, Y.: New approach to vehicle license plate location based on new model YOLO-L and plate pre-identification. IET Image Process. 13, 1041–1049 (2019)

    Article  Google Scholar 

  29. Hendry, R.C.: Automatic license plate recognition via sliding-window darknet-YOLO deep learning. Image Vis. Comput. 87, 47–56 (2019)

    Article  Google Scholar 

  30. Lee, E., Kim, D.: Accurate traffic light detection using deep neural network with focal regression loss. Image Vis. Comput. 87, 24–36 (2019)

    Article  Google Scholar 

  31. Chen, S.W., Shivakumar, S.S., Dcunha, S., Das, J., Okon, E., Qu, C., Taylor, C.J., Kumar, V.: Counting apples and oranges with deep learning : A data driven approach. IEEE Robot. Autom. Lett. 2, 781–788 (2017)

    Article  Google Scholar 

  32. Dyrmann, M., Jørgensen, R.N., Midtiby, H.S.: RoboWeedSupport - Detection of weed locations in leaf occluded cereal crops using a fully convolutional neural network. Adv. Anim. Precis. Agric. 8, 842–847 (2017)

    Google Scholar 

  33. Dias, P.A., Tabb, A., Medeiros, H.: Apple flower detection using deep convolutional networks. Comput. Ind. 99, 17–28 (2018)

    Article  Google Scholar 

  34. Sa, I., Ge, Z., Dayoub, F., Upcroft, B., Perez, T., McCool, C.: Deepfruits: a fruit detection system using deep neural networks. Sensors 16(8), 122 (2016)

    Article  Google Scholar 

  35. Madeleine, S., Bargoti, S., Underwood, J.: Image based mango fruit detection, localisation and yield estimation using multiple view geometry. Sensors 16(11), 1915 (2016)

    Article  Google Scholar 

  36. Chen, Y., Lee, W.S., Gan, H., Peres, N., Fraisse, C., Zhang, Y., He, Y.: Strawberry yield prediction based on a deep neural network using high-resolution aerial orthoimages. Remote Sens 11, 1–21 (2019)

    Google Scholar 

  37. Gené-Mola, J., Vilaplana, V., Rosell-Polo, J.R., Morros, J.R., Ruiz-Hidalgo, J., Gregorio, E.: Multi-modal deep learning for Fuji apple detection using RGB-D cameras and their radiometric capabilities. Comput. Electron. Agric. 162, 689–698 (2019)

    Article  Google Scholar 

  38. Koirala, A., Walsh, K.B., Wang, Z., McCarthy, C.: Deep learning for real-time fruit detection and orchard fruit load estimation: benchmarking of MangoYOLO. Precis. Agric. 20, 1107–1135 (2019)

    Article  Google Scholar 

  39. Liu, J., Wang, X.: Tomato diseases and pests detection based on improved Yolo V3 convolutional neural network. Front. Plant Sci. 11, 1–12 (2020)

    Article  Google Scholar 

  40. Tian, Y., Yang, G., Wang, Z., Wang, H., Li, E., Liang, Z.: Apple detection during different growth stages in orchards using the improved YOLO-V3 model. Comput. Electron. Agric. 157, 417–426 (2019)

    Article  Google Scholar 

  41. Tian, Y., Yang, G., Wang, Z., Li, E., Liang, Z.: Detection of apple lesions in orchards based on deep learning methods of CycleGAN and YOLOV3-Dense. J. Sensors. 2019, 1–13 (2019)

    Google Scholar 

  42. Liu, G., Nouaze, J.C., Mbouembe, P.L.T., Kim, J.H.: YOLO-tomato: a robust algorithm for tomato detection based on YOLOv3. Sensors 20(7), 2145 (2020). https://doi.org/10.3390/s20072145

    Article  Google Scholar 

  43. Huang, G., Liu, Z., Maaten, L. van der, Weinberger, K.Q.: Densely connected convolutional networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)

  44. Ramachandran, P., Zoph, B., Le, Q. V.: Swish: a self-gated activation function, In: Neural and Evolutionary Computing. pp. 1–12 (2017). arXiv:1710.05941

  45. Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft COCO: common objects in context. In: Computer Vision—ECCV 2014. Lecture Notes in Computer Science, pp. 740–755 (2014)

  46. Han, S., Pool, J., Tran, J., Dally, W.J.: Learning both weights and connections for efficient neural networks. In: Advances in Neural Information in Processing Systems. pp. 1135–1143 (2015). arXiv:1506.02626v3

  47. Zhu, P., Wen, L., Du, D., Bian, X., Hu, Q., Ling, H.: Vision meets drones: past, present and future. In: Computer Vision and Pattern Recognition, pp. 1–20 (2020). arXiv:2001.06303

  48. Zhao, H., Zhou, Y., Zhang, L., Peng, Y., Hu, X., Peng, H., Cai, X.: Mixed YOLOv3-LITE: a lightweight real-time object detection method. Sensors 20(7), 1861 (2020)

    Article  Google Scholar 

  49. Zhang, P., Zhong, Y., Li, X.: SlimYOLOv3: narrower, faster and better for real-time UAV applications. In: 2019 International Conference on Computer Vision Workshop, pp. 37–45 (2019)

Download references

Acknowledgements

The research funding was provided by RU Grant—Faculty Programme by Faculty of Engineering, University of Malaya with Project No. GPF042A-2019 and Industry-driven Innovation Grant (IDIG) with Project No.: PPSI-2020-CLUSTER-SD01.

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, Faculty of Engineering, Universiti Malaya, 50603, Kuala Lumpur, Malaysia

    Mohamad Haniff Junos, Anis Salwa Mohd Khairuddin & Mahidzal Dahari

  2. Advanced Technologies and Robotics, Sime Darby Technology Centre Sdn Bhd, 43400, Serdang, Selangor, Malaysia

    Subbiah Thannirmalai

Authors
  1. Mohamad Haniff Junos
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Anis Salwa Mohd Khairuddin
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Subbiah Thannirmalai
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Mahidzal Dahari
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Anis Salwa Mohd Khairuddin.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Junos, M.H., Mohd Khairuddin, A.S., Thannirmalai, S. et al. Automatic detection of oil palm fruits from UAV images using an improved YOLO model. Vis Comput 38, 2341–2355 (2022). https://doi.org/10.1007/s00371-021-02116-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-021-02116-3

Keywords