Skip to main content
SpringerLink
Log in

Road pothole detection from smartphone sensor data using improved LSTM

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Road abnormalities can be caused by man-made and natural disasters that affect the safety of drivers and damage vehicles. Therefore, several automatic road monitoring approaches have been proposed to monitor the road surface and detect road abnormalities like potholes. However, low accuracy in detecting the pothole in low-light conditions is taken as the main problem in this work. To address this issue, we presented an Improved Long Short Term Memory model (ILSTM) that combines a three-layer deep LSTM with a One-Dimensional Local Binary Pattern (1D-LBP) layer to detect the presence of potholes during low-light conditions and extract features such as the number of neighbouring samples and pixel values. The proposed strategy collects the pothole data from accelerometer and gyroscope sensor data using a smart phone. This model is used to classify the sensor data and label it either as a pothole or normal. Besides, this makes classification possible and extracts the location of the pothole. Evaluation results demonstrate that the proposed ILSTM approach is also robust to low lighting conditions with a detection accuracy of 99% and requires less execution time in classifying potholes and non-pothole regions on the pothole dataset collected with the help of an accelerometer and a gyroscope.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Data availability

All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by Prabhat Singh, Abhay Bansal, Ahmed E. Kamal, Sunil Kumar. The first draft of the manuscript was written by Prabhat Singh and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Conceptualization: Prabhat Singh; Methodology: Prabhat Singh, Abhay Bansal; Formal analysis and investigation: Ahmed E. Kamal, Prabhat Singh; Writing - original draft preparation: Prabhat Singh, Sunil Kumar; Writing –review and editing: Abhay Bansal; Supervision: Sunil Kumar.

References

  1. Agebure MA, Oyetunji EO, Baagyere EY (2022) A three-tier road condition classification system using a spiking neural network model. J King Saud Univ-Comput Inf Sci 34(5):1718–1729

    Google Scholar 

  2. Allouch A, Koubâa A, Abbes T, Ammar A (2017) Roadsense: smartphone application to estimate road conditions using accelerometer and gyroscope. IEEE Sensors J 17(13):4231–4238

    Article  ADS  Google Scholar 

  3. Ameur S, Khalifa AB, Bouhlel MS (2020) A novel hybrid bidirectional unidirectional LSTM network for dynamic hand gesture recognition with leap motion. Entertain Comput 35:100373

    Article  Google Scholar 

  4. Anaissi A, Khoa NLD, Rakotoarivelo T, Alamdari MM, Wang Y (2019) Smart pothole detection system using vehicle-mounted sensors and machine learning. J Civ Struct Heal Monit 9(1):91–102

    Article  Google Scholar 

  5. Anand S, Gupta S, Darbari V, Kohli S (2018) Crack-pot: autonomous road crack and pothole detection. In: 2018 digital image computing: techniques and applications (DICTA), pp 1–6

    Google Scholar 

  6. Anandhalli M, Tanuja A, Baligar VP, Baligar P (2022) Indian pothole detection based on CNN and anchor-based deep learning method. Int J Inf Technol 14(7):3343–3353

  7. Baek JW, Chung K (2020) Pothole classification model using edge detection in road image. Appl Sci 10(19):6662

    Article  CAS  Google Scholar 

  8. Bansal K, Mittal K, Ahuja G, Singh A, Gill SS (2020) DeepBus: machine learning based real time pothole detection system for smart transportation using IoT. Internet Technol Lett 3(3):e156

    Article  Google Scholar 

  9. Bhamare L, Mitra N, Varade G, Mehta H (2021) Study of types of road abnormalities and techniques used for their detection. In: 2021 7th international conference on electrical, electronics and information engineering (ICEEIE). IEEE, pp 472–477

    Chapter  Google Scholar 

  10. Bhatt U, Mani S, Xi E, Kolter JZ (2017) Intelligent pothole detection and road condition assessment, arXiv preprint arXiv:1710,02595

  11. Cao MT, Tran QV, Nguyen NM, Chang KT (2020) Survey on performance of deep learning models for detecting road damages using multiple dashcam image resources. Adv Eng Inform 46:101182

    Article  Google Scholar 

  12. Carlos MR, Aragón ME, González LC, Escalante HJ, Martínez F (2018) Evaluation of detection approaches for road anomalies based on accelerometer readings addressing who’s who. IEEE Trans Intell Transp Syst 19(10):3334–3343

    Article  Google Scholar 

  13. Casas-Avellaneda DA, López-Parra JF (2016) Detection and localization of potholes in roadways using smartphones. Dyna. 83(195):156–162

    Article  Google Scholar 

  14. Celaya-Padilla JM, Galván-Tejada CE, López-Monteagudo FE, Alonso-González O, Moreno-Báez A, Martínez-Torteya A, Gamboa-Rosales H (2018) Speed bump detection using accelerometric features: a genetic algorithm approach. Sensors 18(2):443

    Article  PubMed  PubMed Central  ADS  Google Scholar 

  15. Chen H, Yao M, Gu Q (2020) Pothole detection using location-aware convolutional neural networks. Int J Mach Learn Cybern 11(4):899–911

    Article  Google Scholar 

  16. Dey MR, Satapathy U, Bhanse P, Mohanta BK, Jena D (2019) MagTrack: detecting road surface condition using smartphone sensors and machine learning. In: TENCON 2019-2019 IEEE region 10 conference (TENCON), pp 2485–2489

    Chapter  Google Scholar 

  17. Dhiman A, Klette R (2019) Pothole detection using computer vision and learning. IEEE Trans Intell Transp Syst 21(8):3536–3550

    Article  Google Scholar 

  18. Du R, Qiu G, Gao K, Hu L, Liu L (2020) Abnormal road surface recognition based on smartphone acceleration sensor. Sensors 20(2):451

    Article  PubMed  PubMed Central  ADS  Google Scholar 

  19. Fan R, Ozgunalp U, Hosking B, Liu M, Pitas I (2019) Pothole detection based on disparity transformation and road surface modeling. IEEE Trans Image Process 29:897–908

    Article  MathSciNet  ADS  Google Scholar 

  20. Fox A, Kumar BV, Chen J, Bai F (2015) Crowdsourcing undersampled vehicular sensor data for pothole detection. In: 2015 12th annual IEEE international conference on sensing, communication, and networking (SECON). IEEE, pp 515–523

    Chapter  Google Scholar 

  21. Gupta S, Sharma P, Sharma D, Gupta V, Sambyal N (2020) Detection and localization of potholes in thermal images using deep neural networks. Multimed Tools Appl 79(35):26265–26284

    Article  Google Scholar 

  22. Hoang ND, Nguyen QL (2019) A novel method for asphalt pavement crack classification based on image processing and machine learning. Eng Comput 35(2):487–498

    Article  Google Scholar 

  23. Kavitha D, Ravikumar S (2021) Designing an IoT based autonomous vehicle meant for detecting speed bumps and lanes on roads. J Ambient Intell Humaniz Comput 12(7):7417–7426

    Article  Google Scholar 

  24. Kaya Y, Uyar M, Tekin R, Yıldırım S (2014) 1D-local binary pattern based feature extraction for classification of epileptic EEG signals. Appl Math Comput 243:209–219

    Article  MathSciNet  Google Scholar 

  25. Le VNT, Apopei B, Alameh K (2019) Effective plant discrimination based on the combination of local binary pattern operators and multiclass support vector machine methods. Inf Process Agric 6(1):116–131

    Google Scholar 

  26. Lekshmipathy J, Velayudhan S, Mathew S (2021) Effect of combining algorithms in smartphone based pothole detection. Int J Pavement Res Technol 14(1):63–72

    Article  Google Scholar 

  27. Mednis A, Strazdins G, Zviedris R, Kanonirs G, Selavo L (2011) Real time pothole detection using android smartphones with accelerometers. In: 2011 international conference on distributed computing in sensor systems and workshops (DCOSS). IEEE, pp 1–6

    Google Scholar 

  28. Motta G, Sacco D, Ma T, You L, Liu K (2015) Personal mobility service system in urban areas: the IRMA project. In: 2015 IEEE symposium on service-oriented system engineering. IEEE, pp 88–97

    Chapter  Google Scholar 

  29. Sagheer A, Kotb M (2019) Time series forecasting of petroleum production using deep LSTM recurrent networks. Neurocomputing 323:203–213

    Article  Google Scholar 

  30. Sathya R, Saleena B (2022) A framework for designing unsupervised pothole detection by integrating feature extraction using deep recurrent neural network. Wirel Pers Commun 126(2):1241–1271

    Article  Google Scholar 

  31. Singh P, Bansal A, Kumar S (2020) Performance analysis of various information platforms for recognizing the quality of Indian roads. In: 2020 10th international conference on cloud computing data science engineering (confluence). IEEE, pp 63–76

    Chapter  Google Scholar 

  32. Tan HX, Aung NN, Tian J, Chua MCH, Yang YO (2019) Time series classification using a modified LSTM approach from accelerometer-based data: a comparative study for gait cycle detection. Gait Posture 74:128–134

    Article  PubMed  Google Scholar 

  33. Tian Y, Zhang K, Li J, Lin X, Yang B (2018) LSTM-based traffic flow prediction with missing data. Neurocomputing 318:297–305

    Article  Google Scholar 

  34. Varona B, Monteserin A, Teyseyre A (2020) A deep learning approach to automatic road surface monitoring and pothole detection. Pers Ubiquit Comput 24(4):519–534

    Article  Google Scholar 

  35. Wang L, Liu R (2020) Human activity recognition based on wearable sensor using hierarchical deep LSTM networks. Circuits Syst Signal Process 39(2):837–856

    Article  Google Scholar 

  36. Wang H, Huo N, Li J, Wang K, Wang Z (2018) A road quality detection method based on the mahalanobis-taguchi system. IEEE Access 6:29078–29087

    Article  Google Scholar 

  37. Wu H, Yao L, Xu Z, Li Y, Ao X, Chen Q, Meng B (2019) Road pothole extraction and safety evaluation by integration of point cloud and images derived from mobile mapping sensors. Adv Eng Inform 42:100936

    Article  Google Scholar 

  38. Wu C, Wang Z, Hu S, Lepine J, Na X, Ainalis D, Stettler M (2020) An automated machine-learning approach for road pothole detection using smartphone sensor data. Sensors 20(19):5564

    Article  PubMed  PubMed Central  ADS  Google Scholar 

  39. Yun HS, Kim TH, Park TH (2019) Speed-bump detection for autonomous vehicles by Lidar and camera. J Electr Eng Technol 14(5):2155–2162

    Article  Google Scholar 

  40. Zantalis F, Koulouras G, Karabetsos S, Kandris D (2019) A review of machine learning and IoT in smart transportation. Future Internet 11(4):94

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Amity School of Engineering and Technology, Noida, Uttar Pradesh, 201301, India

    Prabhat Singh, Abhay Bansal & Sunil Kumar

  2. Department of Electrical and Computer Engineering at Iowa State University, Ames, IA, 50011, USA

    Ahmed E. Kamal

Authors
  1. Prabhat Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ahmed E. Kamal
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Abhay Bansal
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sunil Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All the authors have participated in writing the manuscript and have revised the final version. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Prabhat Singh.

Ethics declarations

This article does not contain any studies with human participants and/or animals performed by any of the authors.

Consent to participate

There is no informed consent for this study.

Consent for publication

Not Applicable.

Conflict of interest

Authors declares that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Singh, P., Kamal, A.E., Bansal, A. et al. Road pothole detection from smartphone sensor data using improved LSTM. Multimed Tools Appl 83, 26009–26030 (2024). https://doi.org/10.1007/s11042-023-16177-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-16177-0

Keywords

Search

Navigation