Skip to main content
SpringerLink
Log in

Sound-Based Anomalies Detection in Agricultural Robotics Application

  • Conference paper
  • First Online:
Progress in Artificial Intelligence (EPIA 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14116))

Included in the following conference series:

  • 426 Accesses

Abstract

Agricultural robots are exposed to adverse conditions reducing the components’ lifetime. To reduce the number of inspection, repair and maintenance activities, we propose using audio-based systems to diagnose and detect anomalies in these robots. Audio-based systems are non-destructive/intrusive solutions. Besides, it provides a significant amount of data to diagnose problems and for a wiser scheduler for preventive activities. So, in this work, we installed two microphones in an agricultural robot with a mowing tool. Real audio data was collected with the robotic mowing tool operating in several conditions and stages. Besides, a Sound-based Anomalies Detector (SAD) is proposed and tested with this dataset. The SAD considers a short-time Fourier transform (STFT) computation stage connected to a Support Vector Machine (SVM) classifier. The results with the collected dataset showed an F1 score between 95% and 100% in detecting anomalies in a mowing robot operation.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Baltazar, A.: Dataset for Sound-based Anomalies Detection in Agricultural Robotics Application (Oct2022) https://doi.org/10.5281/zenodo.7194547, https://doi.org/10.5281/zenodo.7194547

  2. Bayram, B., Duman, T.B., Ince, G.: Real time detection of acoustic anomalies in industrial processes using sequential autoencoders. Expert Syst. 38. https://doi.org/10.1111/exsy.12564

  3. Becker, P., Roth, C., Roennau, A., Dillmann, R.: Acoustic anomaly detection in additive manufacturing with long short-term memory neural networks (2020). https://doi.org/10.1109/ICIEA49774.2020.9102002

  4. Chachada, S., Kuo, C.C.J.: Environmental sound recognition: a survey. In: 2013 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, pp. 1–9 (2013). https://doi.org/10.1109/APSIPA.2013.6694338

  5. Gribonval, R.: Linear Time-Frequency Analysis I: Fourier-Type Representations, pp. 61–91 (2010). https://doi.org/10.1002/9780470611203.ch3

  6. Henze, D., Gorishti, K., Bruegge, B., Simen, J.P.: Audioforesight: a process model for audio predictive maintenance in industrial environments. In: 2019 18th IEEE International Conference On Machine Learning And Applications (ICMLA), pp. 352–357 (2019). https://doi.org/10.1109/ICMLA.2019.00066

  7. Huang, Z., Shiigi, T., Tsay, L.W.J., Nakanishi, H., Suzuki, T., Ogawa, Y., Naoshi, K.: A sound-based positioning system with centimeter accuracy for mobile robots in a greenhouse using frequency shift compensation. Comput. Electron. Agric. 187, 106235 (2021) https://doi.org/10.1016/j.compag.2021.106235, https://www.sciencedirect.com/science/article/pii/S0168169921002520

  8. Oppenheim, A.V., Schafer, R.W.: Discrete-Time Signal Processing, 3rd edn. Prentice Hall Press, USA (2009)

    Google Scholar 

  9. Pandiyan, V., Prost, J., Vorlaufer, G., Varga, M., Wasmer, K.: Identification of abnormal tribological regimes using a microphone and semi-supervised machine-learning algorithm 10, 583–596 (2021). https://doi.org/10.1007/s40544-021-0518-0

  10. Park, D., Kim, H., Kemp, C.C.: Multimodal anomaly detection for assistive robots. Auton. Robots 43(3), 611–629 (2019). https://doi.org/10.1007/s10514-018-9733-6, https://doi.org/10.1007/s10514-018-9733-6

  11. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., Duchesnay, E.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)

    MathSciNet  Google Scholar 

  12. Rocha, F., Garcia, G., Pereira, R., Faria, H., Silva, T., Andrade, R., Barbosa, E., Santos, A., da Cruz, E., Andrade, W., Serrantola, W., Moura, L., Azpúrua, H., Franca, A., Pessin, G., Freitas, G., Costa, R., Lizarralde, F.: Rosi: A robotic system for harsh outdoor industrial inspection—system design and applications. J. Intell. Robot. Syst. 103 (2021). https://doi.org/10.1007/s10846-021-01459-2

  13. Scipy: scipy.signal.stft - SciPy v1.9.1 Manual, https://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.stft.html. Accessed 03 Oct 2021

  14. Seo, C.B., Lee, G., Lee, Y., Seo, S.H.: Echo-guard: acoustic-based anomaly detection system for smart manufacturing environments. In: Kim, H. (ed.) Information Security Applications, pp. 64–75. Springer International Publishing, Cham (2021)

    Chapter  Google Scholar 

  15. Tanuska, P., Spendla, L., Kebisek, M., Ďuriš, R., Strémy, M.: Smart anomaly detection and prediction for assembly process maintenance in compliance with industry 4.0. Sensors 21, 2376 (2021). https://doi.org/10.3390/s21072376

  16. Vincent, P., Larochelle, H., Bengio, Y., Manzagol, P.A.: Extracting and Composing Robust Features with Denoising Autoencoders, pp. 1096–1103. ICML ’08, Association for Computing Machinery, New York, NY, USA (2008). https://doi.org/10.1145/1390156.1390294, https://doi.org/10.1145/1390156.1390294

  17. Widodo, S., Shiigi, T., Hayashi, N., Kikuchi, H., Yanagida, K., Nakatsuchi, Y., Ogawa, Y., Kondo, N.: Moving object localization using sound-based positioning system with doppler shift compensation. Robotics 2(2), 36–53 (2013) https://doi.org/10.3390/robotics2020036, https://www.mdpi.com/2218-6581/2/2/36

  18. Wüstrich, L., Schröder, L., Pahl, M.O.: Cyber-physical anomaly detection for ICS. In: 2021 IFIP/IEEE International Symposium on Integrated Network Management (IM), pp. 950–955 (2021)

    Google Scholar 

  19. Yoo, Y., Lee, C.Y., Zhang, B.T.: Multimodal anomaly detection based on deep auto-encoder for object slip perception of mobile manipulation robots. In: 2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 11443–11449 (2021). https://doi.org/10.1109/ICRA48506.2021.9561586

  20. Yun, H., Kim, H., Jeong, Y., Jun, M.: Autoencoder-based anomaly detection of industrial robot arm using stethoscope based internal sound sensor. J. Intell. Manuf. 1–18 (2021). https://doi.org/10.1007/s10845-021-01862-4

Download references

Acknowledgment

André Rogrigues Baltazar thanks the FCT-Foundation for Science and Technology, Portugal for the Ph.D. Grant 2021.04859.BD. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 101000554. Disclaimer: The sole responsibility for the content on this publication lies with the authors. It does not necessarily reflect the opinion of the European Research Executive Agency (REA) or the European Commission (EC). The REA or the EC are not responsible for any use that may be made of the information contained therein.

Author information

Authors and Affiliations

  1. CRIIS-Centre for Robotics in Industry and Intelligent Systems, INESC TEC-Institute for Systems and Computer Engineering, Technology and Science, 4200-465, Porto, Portugal

    André Rodrigues Baltazar, Filipe Neves dos Santos, António Paulo Moreira & José Boaventura Cunha

  2. ECT-School of Science and Technology, UTAD-University of Trás-os-Montes and Alto Douro, 5000-801, Vila Real, Portugal

    André Rodrigues Baltazar, Salviano Pinto Soares & José Boaventura Cunha

  3. FEUP-Faculty of Engineering, University of Porto, 4200-465, Porto, Portugal

    António Paulo Moreira

  4. IEETA—Institute of Electronics and Informatics Engineering of Aveiro, UA Campus, 3810-193, Aveiro, Portugal

    Salviano Pinto Soares

Authors
  1. André Rodrigues Baltazar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Filipe Neves dos Santos
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Salviano Pinto Soares
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. António Paulo Moreira
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. José Boaventura Cunha
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to André Rodrigues Baltazar .

Editor information

Editors and Affiliations

  1. Lucy Family Institute for Data and Society, Notre Dame, IN, USA

    Nuno Moniz

  2. GECAD, Polytechnic of Porto, Porto, Portugal

    Zita Vale

  3. GRIA—LIACC, University of Azores, Ponta Delgada, Portugal

    José Cascalho

  4. CISUC, University of Coimbra, Coimbra, Portugal

    Catarina Silva

  5. IEETA, University of Aveiro, Aveiro, Portugal

    Raquel Sebastião

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Baltazar, A.R., dos Santos, F.N., Soares, S.P., Moreira, A.P., Cunha, J.B. (2023). Sound-Based Anomalies Detection in Agricultural Robotics Application. In: Moniz, N., Vale, Z., Cascalho, J., Silva, C., Sebastião, R. (eds) Progress in Artificial Intelligence. EPIA 2023. Lecture Notes in Computer Science(), vol 14116. Springer, Cham. https://doi.org/10.1007/978-3-031-49011-8_27

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-49011-8_27

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-49010-1

  • Online ISBN: 978-3-031-49011-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation