Skip to main content
Springer Nature Link
Log in

Development of a Computer Vision System for an Optical Sorting Robot

  • Conference paper
  • First Online:
Interactive Collaborative Robotics (ICR 2024)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14898))

Included in the following conference series:

  • 349 Accesses

Abstract

The use of sorting robots at some stages of agricultural production seems quite promising. In addition to the high-tech design, the most important element of such robots may be a pattern recognition system, which in turn, in addition to optical devices, includes an intelligent decision support system. This paper presents the development of a computer vision system for recognizing suitable samples of grain crops for an optical sorting robot. The aim of the work is to create a reliable and efficient algorithm that can accurately determine the presence and characteristics of damage in images of wheat, oats and peas. Modern machine learning methods, such as convolutional neural networks, were implemented to train the recognition model. The development process included collecting and preparing training data, selecting and setting up the neural network architecture, as well as testing and optimizing the algorithm. A comparison of computer vision libraries YOLO, FASTER R-CNN, VISSL, OpenCV was carried out. The resulting system demonstrated high accuracy in recognizing defects and morphological features of seeds in test images, with an accuracy of up to 87%. The developed system can be used in optical sorting robots and various mechatronic applications related to automation of agricultural processes, product quality analysis, robotic phenotyping devices, as well as in seed quality control systems and in intelligent control systems for agricultural production processes in crop production.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. GOST 12036-85 is an interstate standard. https://fsvps.gov.ru/files/gost-12036-85-mezhgosudarstvennyj-standart-s/. Accessed 30 May 2024

  2. GOST 12037–81 is an interstate standard. https://fsvps.gov.ru/files/gost-12037-81-mezhgosudarstvennyj-standart-s/. Accessed 30 May 2024

  3. Rutts, R., Kolmakov, Y.: Varietal composition is the basis for stabilizing the production of high-quality agricultural products in the Omsk Region. Achiev. Sci. Technol. Agro-Ind. Complex 11, 38–39 (2010)

    Google Scholar 

  4. Korchagina, I., Yushkevich, L.: Wheat Varieties in Intensive Agriculture of the Omsk Irtysh Region. Omsk Federal State Budgetary Educational Institution «Omsk ASC», Omsk (2023)

    Google Scholar 

  5. Khramtsov, I., et al.: The System of Adaptive Agriculture of the Omsk Region. Publishing house of Maksheeva E.A., Omsk (2020)

    Google Scholar 

  6. Arkhipov, M., Danilova, T., Pavlyushin, V., Sinitsyna, S., Pasynkova, E., Tyukalov, Y.: Ways and possibilities of phytosanitary optimization of agroecosystems in Northwest Region of Russia. Plant Prot. News 2(92), 5–14 (2017)

    Google Scholar 

  7. GOST 12043-88 is an interstate standard. https://fsvps.gov.ru/files/gost-12043-88-gosudarstvennyj-standart-sojuz/. Accessed 30 May 2024

  8. Grain Cleaning System, Construction of Granaries. https://baitekmachinery.ru/grainclean/. Accessed 30 May 2024

  9. Kovrikov, I.: Technological Equipment of Grain Storage, Processing and Processing Enterprises. «GOU OGU», Orenburg (2009)

    Google Scholar 

  10. «GRAIN SORTING DEVICE» RU2495728 C1. https://searchplatform.rospatent.gov.ru/doc/RU2495728C1_20131020?q=&from=search_simple&hash=1314158884. Accessed 30 May 2024

  11. «SEED SORTING DEVICE» RU2687509 C1. https://searchplatform.rospatent.gov.ru/doc/RU2687509C1_20190514?q=&from=search_simple&hash=-469636176. Accessed 30 May 2024

  12. Grain Sorting. https://www.buhlergroup.com/global/ru/process-technologies/Optical-Sorting/Grain-sorting.html. Accessed 30 May 2024

  13. MEYER Sorting Equipment. https://meyer-corp.ru/. Accessed 30 May 2024

  14. Catalog of Photo Separators and Sorting Solutions. https://csort.ru/photoseparator/. Accessed 30 May 2024

  15. Contreras, L., Savage, J., Ortuno-Chanelo, S., Negrete, M., Sakamaki, A., Okada, H.: Fail it till you make it: error expectation in complex-plan execution for service robots. In: Ronzhin, A., Sadigov, A., Meshcheryakov, R. (eds.) Interactive Collaborative Robotics. ICR 2023. LNCS, vol. 14214, pp. 36–46. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-43111-1_4

  16. Zhou, H., Ou, J., Meng, P., Tong, J., Ye, H., Li, Z.: Reasearch on kiwi fruit flower recognition for efficient pollination based on an improved YOLOv5 algorithm. Horticulturae 9(3) (2023)

    Google Scholar 

  17. Li, H., et al: Rapid assessment of ready-to-eat Xuxiang kiwifruit quality based on Chroma recognition and GC-MS analysis. LWT 182 (2023)

    Google Scholar 

  18. Ortuno-Chanelo, S., Contreras, L., Savage, J., Okada, H.: Keep it simple: understanding natural language commands for general-purpose service robots. In: 2024 IEEE/SICE International Symposium on System Integration (SII), pp. 1320–1325. IEEE (2024)

    Google Scholar 

  19. Kolpashchikov, D., Gerget, O., Meshcheryakov, R.: Robotics in Healthcare. In: Lim, C.P., Chen, Y.W., Vaidya, A., Mahorkar, C., Jain, L.C. (eds.) Handbook of Artificial Intelligence in Healthcare. Intelligent Systems Reference Library, LNCS, vol. 212, pp. 281–306. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-83620-7_12

  20. Wu, N., Liu, F., Meng, F, Li M, Zhang, C., He, Y.: Rapid and accurate varieties classification of different crop seeds under sample-limited condition based on hyperspectral imaging and deep transfer learning. Front. Bioeng. Biotechnol. 9 (2021)

    Google Scholar 

  21. Chen, H., Qiao, H., Feng, Q., Xu, L., Lin, Q., Cai, K.: Rapid detection of pomelo fruit quality using near-infrared hyperspectral imaging combined with chemometric methods. Front. Bioeng. Biotechnol. 8 (2021)

    Google Scholar 

  22. Allegra, D., Battiato, S., Ortis, A., Urso, S., Polosa, R.: A review on food recognition technology for health applications. Health Psychol. Res. 8(3), 172–187 (2020)

    Article  Google Scholar 

  23. Jin, H., Li, Y., Qi, J., Feng, J., Tian, D., Mu, W.: GrapeGAN: unsupervised image enhancement for improved grape leaf disease recognition. Comput. Electron. Agric. 198 (2022)

    Google Scholar 

  24. Saleem, R., Shah, J., Sharif, M., Yasmin, M., Yong, H., Cha, J.: Mango leaf disease recognition and classification using novel segmentation and vein pattern technique. Appl. Sci. 11(24) (2021)

    Google Scholar 

  25. Kuznetsova, A., Maleva, T., Soloviev, V.: Using YOLOv3 algorithm with pre-and post-processing for apple detection in fruit-harvesting robot. Agronomy 10(7) (2020)

    Google Scholar 

  26. Krakhmalev, O., et al.: Robotic complex for harvesting apple crops. Robotics 11(4) (2022)

    Google Scholar 

  27. Kamyshova, G., et al.: Artificial neural networks and computer vision’s-based phytoindication systems for variable rate irrigation improving. IEEE Access 10, 8577–8589 (2022)

    Article  Google Scholar 

  28. Osipov, A., et al.: Deep learning method for recognition and classification of images from video recorders in difficult weather conditions. Sustainability 14(4) (2022)

    Google Scholar 

  29. Karpov, O., et al.: Detecting epileptic seizures using machine learning and interpretable features of human EEG. Eur. Phys. J. Spec. Top. 232(5), 673–682 (2023)

    Article  Google Scholar 

  30. Kuc, A., Korchagin, S., Maksimenko, V., Shusharina, N., Hramov, A.: Combining statistical analysis and machine learning for EEG scalp topograms classification. Front. Syst. Neurosci. 15 (2021)

    Google Scholar 

  31. Ronzhin, A., Dudakov, M., Dudakova, D.: Conceptual and set-theoretical models of the functioning and application of system solutions for bottom sediment sampling. Vestnik Yuzhno-Ural'skogo Gosudarstvennogo Universiteta. Seriya «Matematika. Mekhanika. Fizika» 15(1), 43–54 (2023)

    Google Scholar 

  32. Wu, N., Liu, F., Meng, F., Li, M., Zhang, C., He, Y.: Rapid and accurate varieties classification of different crop seeds under sample-limited condition based on hyperspectral imaging and deep transfer learning. Front. Bioeng. Biotechnol. 9 (2021)

    Google Scholar 

  33. Meshcheryakov, R., Rusakov, K., Tevyashov, G., Myshkin, A.: Detection and characterization of caviar using a neural network algorithm. In: Ronzhin, A., Kostyaev, A. (eds.) Agriculture Digitalization and Organic Production. ADOP 2023. Smart Innovation, Systems and Technologies, LNCS, vol. 362, pp. 383–395. Springer, Singapore (2023). https://doi.org/10.1007/978-981-99-4165-0_35

  34. Galin, R., Meshcheryakov, R., Mamchenko, M.: Simple task allocation algorithm in a collaborative robotic system. In: Ronzhin, A., Pshikhopov, V. (eds.) Frontiers in Robotics and Electromechanics. Smart Innovation, Systems and Technologies, LNCS, vol. 329, pp. 433–447. Springer, Singapore (2023). https://doi.org/10.1007/978-981-19-7685-8_28

  35. Durum Wheat Dataset. https://www.kaggle.com/datasets/muratkokludataset/durum-wheat-dataset. Accessed 30 May 2024

  36. Andriyanov, N., Dementiev, V., Tashlinskiy, A.: Optimization of the computer vision system for the detection of moving objects. In: Rousseau, J.J., Kapralos, B. (eds.) Pattern Recognition, Computer Vision, and Image Processing. ICPR 2022 International Workshops and Challenges. ICPR 2022. LNCS, vol. 13644, pp. 424–431. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-37742-6_32

  37. Vasiliev, N., et al.: Development of the intelligent object detection system on the road for self-driving cars in low visibility conditions. In: Klimov, V.V., Kelley, D.J. (eds.) Biologically Inspired Cognitive Architectures 2021. BICA 2021. Studies in Computational Intelligence, LNCS, vol. 1032, pp. 576–584. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-96993-6_64

  38. Ronzhin, A., Khalilov, E., Lazukin, A., Savelyev, A., Ma, Z., Wang, M.: Simulation of methods of control the dynamics of cyanobacterial blooming using air and surface robotics. Trans. Kola Sci. Centre RAS. Ser. Eng. Sci. 14(7), 86–91 (2023)

    Google Scholar 

  39. What is YOLOv7? A Complete Guide. https://blog.roboflow.com/yolov7-breakdown/. Accessed 30 May 2024

Download references

Acknowledgments

The article was carried out with the financial support of the Ministry of Science and Higher Education of the Russian Federation, Agreement No. 075-15-2024-542.

Author information

Authors and Affiliations

  1. State University of Management, 99 Ryazansky Av., Moscow, 109542, Russia

    Otari Didmanidze, Maria Karelina, Vladimir Filatov, Dmitriy Rybakov, Nikita Andriyanov, Sergey Korchagin, Yuliya Kafiyatullina & Denis Serdechnyy

Authors
  1. Otari Didmanidze
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Maria Karelina
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Vladimir Filatov
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dmitriy Rybakov
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Nikita Andriyanov
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Sergey Korchagin
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Yuliya Kafiyatullina
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Denis Serdechnyy
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Denis Serdechnyy .

Editor information

Editors and Affiliations

  1. St. Petersburg Federal Research Center of the Russian Academy of Sciences, Saint Petersburg, Russia

    Andrey Ronzhin

  2. National Autonomous University of Mexico, Mexico City, Mexico

    Jesus Savage

  3. V.A. Trapeznikov Institute of Control Sciences of the Russian Academy of Sciences, Moscow, Russia

    Roman Meshcheryakov

Ethics declarations

The authors have no competing interests to declare that are relevant to the content of this article.

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Didmanidze, O. et al. (2024). Development of a Computer Vision System for an Optical Sorting Robot. In: Ronzhin, A., Savage, J., Meshcheryakov, R. (eds) Interactive Collaborative Robotics. ICR 2024. Lecture Notes in Computer Science(), vol 14898. Springer, Cham. https://doi.org/10.1007/978-3-031-71360-6_16

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-71360-6_16

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-71359-0

  • Online ISBN: 978-3-031-71360-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics