Skip to main content

Advertisement

Springer Nature Link
Log in

A novel approach for end-to-end navigation for real mobile robots using a deep hybrid model

  • Original Research Paper
  • Published:
Intelligent Service Robotics Aims and scope Submit manuscript

Abstract

The ability to use a single camera for indoor navigation of a real mobile robot can help develop a complete navigation system, eliminating the need to use a traditional navigation system. The latter involves many subsystems such as perception for capturing the scene and extracting the necessary information, path planning to determine the optimal trajectory from the origin to the destination, and obstacle avoidance. The complexity of this system makes it highly sensitive to variations in the environment, making it inappropriate for real-time use, particularly for mobile robots with limited computational capacity. In this paper, we present an end-to-end navigation system for indoor navigation based on convolutional neural networks (CNNs). We developed four distinct deep hybrid model (DHMs) approaches using four pretrained convolutional networks (Visual Geometry Group 19 (VGG19), Residual Network 50 (ResNet50), Alex Krizhevsky’s network (AlexNet), and Efficient Network B0 (EfficientNetB0)) as feature extractors, combined with a random forest (RF) machine learning classifier. Validation metrics (accuracy, precision, and F1 score) were used to evaluate the performance of the models. During the test phase, we introduced three new and never-before-seen environments. This study revealed that the combination of VGG19 and random forest models produced excellent results, achieving an accuracy of 97.63%. This model was also able to navigate efficiently in most environments, generating a smoother path than other models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+
from $39.99 /Month
  • Starting from 10 chapters or articles per month
  • Access and download chapters and articles from more than 300k books and 2,500 journals
  • Cancel anytime
View plans

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21

Similar content being viewed by others

Explore related subjects

Discover the latest articles and news from researchers in related subjects, suggested using machine learning.

Data availability

The dataset without augmentation is available at: Mendeley Data, and the dataset with augmentation is available at: Mendeley Data.

References

  1. Sevastopoulos C, Konstantopoulos S (2022) A survey of traversability estimation for mobile robots. IEEE Access 10:96331–96347. https://doi.org/10.1109/ACCESS.2022.3202545

    Article  MATH  Google Scholar 

  2. Chen T (2023) Investigating the mental health of university students during the covid-19 pandemic in a UK university: a machine learning approach using feature permutation importance. Brain Inform 10:27. https://doi.org/10.1186/s40708-023-00205-8

    Article  Google Scholar 

  3. Rathod T, Jadav NK, Tanwar S, Polkowski Z, Yamsani N, Sharma R, Alqahtani F, Gafar A (2023) Ai and blockchain-based secure data dissemination architecture for IoT-enabled critical infrastructure. Sensors 23:8928. https://doi.org/10.3390/s23218928

    Article  Google Scholar 

  4. Ni J, Wu L, Fan X, Yang SX (2016) Bioinspired intelligent algorithm and its applications for mobile robot control: a survey. Comput Intell Neurosci 2016:3810903. https://doi.org/10.1155/2016/3810903

    Article  MATH  Google Scholar 

  5. Soltani AR, Tawfik H, Goulermas JY, Fernando T (2002) Path planning in construction sites: Performance evaluation of the dijkstra, a, and ga search algorithms. Adv Eng Inform 16:291–303. https://doi.org/10.1016/S1474-0346(03)00018-1

    Article  Google Scholar 

  6. Weigl M, Siemia̧tkowska B, Sikorski CA, Borkowski A (1993) Grid-based mapping for autonomous mobile robot. Robot Auton Syst 11:13–21. https://doi.org/10.1016/0921-8890(93)90004-V

    Article  MATH  Google Scholar 

  7. Waga A, Chaymaa L, Benhlima S, Ali B (2021) Fuzzy logic obstacle avoidance by a nao robot in unknown environment. In: 2021 Fifth international conference on intelligent computing in data sciences (ICDS), pp 1–7. https://doi.org/10.1109/ICDS53782.2021.9626718

  8. Lamini C, Benhlima S, Elbekri A (2018) Genetic algorithm based approach for autonomous mobile robot path planning. Proced Comput Sci 127:180–189. https://doi.org/10.1016/j.procs.2018.01.113

    Article  Google Scholar 

  9. Waga A, Benhouria Y, Ba-Ichou A, Benhlima S, Ali B, Abdouni J (2023) A new method for mobile robots to learn an optimal policy from an expert using deep imitation learning. https://doi.org/10.1007/978-3-031-29857-8_87

  10. Vale A, Lucas JM, Ribeiro MI (2004) Feature extraction and selection for mobile robot navigation in unstructured environments. IFAC Proc Vol 37(8):102–107. https://doi.org/10.1016/S1474-6670(17)31958-4

    Article  MATH  Google Scholar 

  11. Zhang S, Xie L, Adams MD (2006) Feature extraction for outdoor mobile robot navigation based on a modified gauss-newton optimization approach. Robot Auton Syst 54(4):277–287. https://doi.org/10.1016/j.robot.2005.11.008

    Article  MATH  Google Scholar 

  12. Ali MAH, Mailah M, Yussof WAB, Hamedon ZB, Yussof ZB, Majeed APP (2016) Sensors fusion based online mapping and features extraction of mobile robot in the road following and roundabout. IOP Conf Ser Mater Sci Eng 114(1):012135. https://doi.org/10.1088/1757-899X/114/1/012135

    Article  MATH  Google Scholar 

  13. Bonin-Font F, Ortiz A, Oliver G (2008) Visual navigation for mobile robots: a survey. J Intell Rob Syst 53:263–296. https://doi.org/10.1007/s10846-008-9235-4

    Article  MATH  Google Scholar 

  14. Temeltas H, Kayak D (2008) Slam for robot navigation. IEEE Aerosp Electron Syst Mag 23(12):16–19. https://doi.org/10.1109/MAES.2008.4694832

    Article  MATH  Google Scholar 

  15. Liu R, Zhang J, Chen S, Yang T, Arth C (2021) Accurate real-time visual slam combining building models and gps for mobile robot. J Real-Time Image Proc 18:419–429. https://doi.org/10.1007/s11554-020-00989-6

    Article  MATH  Google Scholar 

  16. Feng K, Gao Q, Wang X, Jiang Y (2023) Indoor high-precision 3d-slam algorithm incorporating planar element features. In: Yan L, Duan H, Deng Y (eds.) Advances in guidance, navigation and control. Springer, Singapore, pp 3025–3036 https://doi.org/10.1007/978-981-19-6613-2_293

  17. Sabry ES, Elagooz SS, El-Samie FEA, El-Bahnasawy NA, El-Banby GM, Ramadan RA (2023) Evaluation of feature extraction methods for different types of images. J Opt 52:716–741. https://doi.org/10.1007/s12596-022-01024-6

    Article  MATH  Google Scholar 

  18. Li Y (2020) Multimodal visual image processing of mobile robot in unstructured environment based on semi-supervised multimodal deep network. J Ambient Intell Humaniz Comput 11:6349–6359. https://doi.org/10.1007/s12652-020-02037-4

    Article  MATH  Google Scholar 

  19. Leonard JJ, Durrant-Whyte HF, Cox IJ (1992) Dynamic map building for an autonomous mobile robot. Int J Robot Res 11(4):286–298. https://doi.org/10.1177/027836499201100402

    Article  MATH  Google Scholar 

  20. Bhavya Sree B, Yashwanth Bharadwaj V, Neelima N (2021) An inter-comparative survey on state-of-the-art detectors—r-cnn, yolo, and ssd. In: Reddy ANR, Marla D, Favorskaya MN, Satapathy SC (eds.) Intelligent Manufacturing and Energy Sustainability, Springer, Singapore, pp 475–483 https://doi.org/10.1007/978-981-33-4443-3_46

  21. Kim Y-H, Jang J-I, Yun S (2018) End-to-end deep learning for autonomous navigation of mobile robot. In: 2018 IEEE international conference on consumer electronics (ICCE), pp 1–6. https://doi.org/10.1109/ICCE.2018.8326229

  22. Ran T, Yuan L, Zhang J (2021) Scene perception based visual navigation of mobile robot in indoor environment. ISA Trans 109:389–400. https://doi.org/10.1016/j.isatra.2020.10.023

    Article  MATH  Google Scholar 

  23. Vijitkunsawat W, Chantngarm P (2020) Comparison of machine learning algorithm’s on self-driving car navigation using nvidia jetson nano. In: 2020 17th international conference on electrical engineering/electronics, computer, telecommunications and information technology (ECTI-CON), pp 201–204 https://doi.org/10.1109/ECTI-CON49241.2020.9158311

  24. Carballo A, Seiya S, Lambert J, Darweesh H, Narksri P, Morales Y, Akai N, Takeuchi E, Takeda K (2018) End-to-end autonomous mobile robot navigation with model-based system support. J Robot Mechatron 30:563–583. https://doi.org/10.20965/jrm.2018.p0563

    Article  Google Scholar 

  25. Febbo R, Flood B, Halloy J, Lau P, Wong K, Ayala A (2020) Autonomous vehicle control using a deep neural network and jetson nano. In: Practice and experience in advanced research computing. PEARC ’20, Association for Computing Machinery, New York, pp 333–338. https://doi.org/10.1145/3311790.3396669

  26. Liu C, Zheng B, Wang C, Zhao Y, Fu S, Li H (2017) Cnn-based vision model for obstacle avoidance of mobile robot. MATEC Web Conf 139:00007. https://doi.org/10.1051/matecconf/201713900007

    Article  MATH  Google Scholar 

  27. Xu J, Liu Q, Guo H, Kageza A, AlQarni S, Wu S (2018) Shared multi-task imitation learning for indoor self-navigation. In: CoRR arXiv:1808.04503. https://doi.org/10.48550/arXiv.1808.045037

  28. Xu J, Zhu S, Guo H, Wu S (2017) Avoidance of manual labeling in robotic autonomous navigation through multi-sensory semi-supervised learning. In: CoRR arXiv:1709.07911. https://doi.org/10.48550/arXiv.1709.07911

  29. Xu S, Rai R (2024) Vision-based autonomous navigation stack for tractors operating in peach orchards. Comput Electron Agric 217:108558. https://doi.org/10.1016/j.compag.2023.108558

    Article  Google Scholar 

  30. de Silva R, Cielniak G, Gao J (2024) Vision based crop row navigation under varying field conditions in arable fields. Comput Electron Agric 217:108581. https://doi.org/10.1016/j.compag.2023.108581

    Article  Google Scholar 

  31. Zhou K, Guo C, Zhang H, Yang B (2023) Optimal graph transformer viterbi knowledge inference network for more successful visual navigation. Adv Eng Inform 55:101889. https://doi.org/10.1016/j.aei.2023.101889

    Article  Google Scholar 

  32. Machkour Z, Ortiz-Arroyo D, Durdevic P (2023) Monocular based navigation system for autonomous ground robots using multiple deep learning models. Int J Comput Intell Syst 16(1):79. https://doi.org/10.1007/s44196-023-00250-5

    Article  MATH  Google Scholar 

  33. Rao J, Bian H, Xu X, Chen J (2023) Autonomous visual navigation system based on a single camera for floor-sweeping robot. Appl Sci 13(3):1562. https://doi.org/10.3390/app13031562

    Article  MATH  Google Scholar 

  34. Abbasi Moshaei AR, Mohammadi Moghaddam M, Dehghan Neistanak VA (2020) Analytical model of hand phalanges desired trajectory for rehabilitation and design a sliding mode controller based on this model. Modares Mech Eng 20(1):129–137

    Google Scholar 

  35. Aly AAI, Abbasimoshaei A, Kern TA (2022) Developing a vr training environment for fingers rehabilitation. In: 13th international conference on human haptic sensing and touch enabled computer applications, EuroHaptics pp. 331–333

  36. Sharma B (2024) Robotic dog for navigation of a rehabilitation wheelchair robot in a highly constrained environment. PLoS ONE 19(9):0310024

    Article  Google Scholar 

  37. Hunt J (2023) Sockets in Python. Springer, Cham, pp 557–569 https://doi.org/10.1007/978-3-031-40336-1_49

  38. Kenlon S (2019) Getting Started with the Raspberry Pi. https://doi.org/10.1007/978-1-4842-4170-7_1

  39. Tian C, Fei L, Zheng W, Xu Y, Zuo W, Lin C-W (2020) Deep learning on image denoising: An overview. Neural Netw 131:251–275. https://doi.org/10.1016/j.neunet.2020.07.025

    Article  MATH  Google Scholar 

  40. Kandel I, Castelli M, Manzoni L (2022) Brightness as an augmentation technique for image classification. Emerg Sci J 6:881–892. https://doi.org/10.28991/ESJ-2022-06-04-015

    Article  MATH  Google Scholar 

  41. Nanni L, Paci M, Brahnam S, Lumini A (2021) Comparison of different image data augmentation approaches. J Imaging 7(12):254. https://doi.org/10.3390/jimaging7120254

    Article  MATH  Google Scholar 

  42. Liu Y, Pu H, Sun D-W (2021) Efficient extraction of deep image features using convolutional neural network (cnn) for applications in detecting and analysing complex food matrices. Trends Food Sci Technol 113:193–204. https://doi.org/10.1016/j.tifs.2021.04.042

    Article  MATH  Google Scholar 

  43. Lichy A, Bader O, Dubin R, Dvir A, Hajaj C (2023) When a rf beats a cnn and gru, together-a comparison of deep learning and classical machine learning approaches for encrypted malware traffic classification. Comput Secur 124:103000. https://doi.org/10.1016/j.cose.2022.103000

    Article  Google Scholar 

  44. Qamar S, Öberg R, Malyshev D, Andersson M (2023) A hybrid CNN-random forest algorithm for bacterial spore segmentation and classification in tem images. Sci Rep 13(1):18758. https://doi.org/10.1038/s41598-023-44212-5

    Article  MATH  Google Scholar 

  45. Khozeimeh F, Sharifrazi D, Izadi NH, Joloudari JH, Shoeibi A, Alizadehsani R, Tartibi M, Hussain S, Sani ZA, Khodatars M, Sadeghi D, Khosravi A, Nahavandi S, Tan R-S, Acharya UR, Islam SMS (2022) Rf-cnn-f: random forest with convolutional neural network features for coronary artery disease diagnosis based on cardiac magnetic resonance. Sci Rep 12(1):11178. https://doi.org/10.1038/s41598-022-15374-5

    Article  MATH  Google Scholar 

  46. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444. https://doi.org/10.1038/nature14539

    Article  MATH  Google Scholar 

  47. Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556 [cs.CV]. https://doi.org/10.48550/arXiv.1409.1556

  48. He K, Zhang X, Ren S, Sun J (2015) Deep residual learning for image recognition. In: CoRR arXiv:1512.03385. https://doi.org/10.48550/arXiv.1512.03385

  49. Krizhevsky A, Sutskever I, Hinton GE (2017) Imagenet classification with deep convolutional neural networks. Commun ACM 60:84–90. https://doi.org/10.1145/3065386

    Article  MATH  Google Scholar 

  50. Tan M, Le QV (2019) Efficientnet: rethinking model scaling for convolutional neural networks. In: CoRR arXiv:1905.11946. https://doi.org/10.48550/arXiv.1905.11946

  51. Breiman L (2001) Random forests. Mach Learn 45:5–32. https://doi.org/10.1023/A:1010933404324

    Article  MATH  Google Scholar 

  52. Jena B, Saxena S, Nayak GK, Saba L, Sharma N, Suri JS (2021) Artificial intelligence-based hybrid deep learning models for image classification: the first narrative review. Comput Biol Med 137:104803. https://doi.org/10.1016/j.compbiomed.2021.104803

    Article  Google Scholar 

  53. Vujovic Z (2021) Classification model evaluation metrics. Int J Adv Comput Sci Appl 12:599–606. https://doi.org/10.14569/IJACSA.2021.0120670

    Article  MATH  Google Scholar 

  54. Dourado CMJM, da Silva SPP, da Nóbrega RVM, Barros ACS, Sangaiah AK, Rebouças Filho PP, de Albuquerque VHC (2019) A new approach for mobile robot localization based on an online IoT system. Futur Gener Comput Syst 100:859–881. https://doi.org/10.1016/j.future.2019.05.074

    Article  Google Scholar 

  55. Zhang Q, Tsoi N, Choi B, Tan J, Chiang H-TL, Vázquez M (2023) Towards inferring users’ impressions of robot performance in navigation scenarios

  56. Anđelić N, Baressi Šegota S, Glučina M, Lorencin I (2023) Classification of wall following robot movements using genetic programming symbolic classifier. Machines 11(1):105. https://doi.org/10.3390/machines11010105

  57. Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, Killeen T, Lin Z, Gimelshein N, Antiga L, Desmaison A, Köpf A, Yang EZ, DeVito Z, Raison M, Tejani A, Chilamkurthy S, Steiner B, Fang L, Bai J, Chintala S (2019) Pytorch: an imperative style, high-performance deep learning library. In: CoRR arXiv:1912.01703

  58. Tan C, Sun F, Kong T, Zhang W, Yang C, Liu C (2018) A survey on deep transfer learning. In: Kůrková V, Manolopoulos Y, Hammer B, Iliadis L, Maglogiannis I (eds.) Artificial Neural Networks and Machine Learning – ICANN 2018, Springer, Cham, pp 270–279 https://doi.org/10.1007/978-3-030-01424-7_27

  59. Zhu Q, He Z, Zhang T, Cui W (2020) Improving classification performance of softmax loss function based on scalable batch-normalization. Appl Sci 10(8):2950. https://doi.org/10.3390/app10082950

    Article  MATH  Google Scholar 

  60. Zhang M, Li W, Du Q (2018) Diverse region-based CNN for hyperspectral image classification. IEEE Trans Image Process 27(6):2623–2634. https://doi.org/10.1109/TIP.2018.2809606

    Article  MathSciNet  MATH  Google Scholar 

  61. Alabassy B, Safar M, El-Kharashi MW (2020) A high-accuracy implementation for softmax layer in deep neural networks. In: 2020 15th design and technology of integrated systems in nanoscale era (DTIS), pp 1–6. https://doi.org/10.1109/DTIS48698.2020.9081313

Download references

Author information

Author notes

  1. Said Benhlima, Ali Bekri, Jawad Abdouni have contributed equally to this work.

Authors and Affiliations

  1. Computer science, Moulay Ismail University, 50000, Zitoun, Meknes, Morocco

    Abderrahim Waga, Said Benhlima & Ali Bekri

  2. National School of Applied Sciences of Kenitra, Ibn Tofail University, Kenitra, 14000, Kenitra, Morocco

    Jawad Abdouni

Authors
  1. Abderrahim Waga
    View author publications

    Search author on:PubMed Google Scholar

  2. Said Benhlima
    View author publications

    Search author on:PubMed Google Scholar

  3. Ali Bekri
    View author publications

    Search author on:PubMed Google Scholar

  4. Jawad Abdouni
    View author publications

    Search author on:PubMed Google Scholar

Corresponding author

Correspondence to Abderrahim Waga.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Waga, A., Benhlima, S., Bekri, A. et al. A novel approach for end-to-end navigation for real mobile robots using a deep hybrid model. Intel Serv Robotics 18, 75–95 (2025). https://doi.org/10.1007/s11370-024-00569-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11370-024-00569-8

Keywords