Skip to main content

Advertisement

Springer Nature Link
Log in

TAAWUN: a Decision Fusion and Feature Specific Road Detection Approach for Connected Autonomous Vehicles

  • Published:
Mobile Networks and Applications Aims and scope Submit manuscript

Abstract

Road transportation is among the global grand challenges affecting human lives, health, society, and economy, caused due to road accidents, traffic congestion, and other transportation deficiencies. Autonomous vehicles (AVs) are set to address major transportation challenges including safety, efficiency, reliability, sustainability, and personalization. The foremost challenge for AVs is to perceive their environments in real-time with the highest possible certainty. Relatedly, connected vehicles (CVs) have been another major driver of innovation in transportation. In this paper, we bring autonomous and connected vehicles together and propose TAAWUN, a novel approach based on the fusion of data from multiple vehicles. The aim herein is to share the information between multiple vehicles about their environments, enhance the information available to the vehicles, and make better decisions regarding the perception of their environments. TAWUN shares, among the vehicles, visual data acquired from cameras installed on individual vehicles, as well as the perceived information about the driving environments. The environment is perceived using deep learning, random forest (RF), and C5.0 classifiers. A key aspect of the TAAWUN approach is that it uses problem specific feature sets to enhance the prediction accuracy in challenging environments such as problematic shadows, extreme sunlight, and mirage. TAAWUN has been evaluated using multiple metrics, accuracy, sensitivity, specificity, and area-under-the-curve (AUC). It performs consistently better than the base schemes. Directions for future work to extend the tool are provided. This is the first work where visual information and decision fusion are used in CAVs to enhance environment perception for autonomous driving.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Mehmood R, Bhaduri B, Katib I, Chlamtac I (eds) (2018) Smart societies, infrastructure, technologies and applications, volume 224 of lecture notes of the institute for computer sciences, social informatics and telecommunications engineering (LNICST). Springer International Publishing, Cham

  2. Mehmood R, See S, Katib I, Chlamtac I (eds) (2019) Smart infrastructure and applications: foundations for smarter cities and societies. EAI/Springer innovations in communication and computing. Springer International Publishing, Cham

  3. Schlingensiepen J, Mehmood R, Nemtanu FC, Niculescu M (2014) Increasing sustainability of road transport in european cities and metropolitan areas by facilitating autonomic road transport systems (arts). In: Sustainable automotive technologies 2013. Springer, pp 201–210

  4. Schlingensiepen J, Nemtanu F, Mehmood R, McCluskey L (2016) Autonomic transport management systems-enabler for smart cities, personalized medicine, participation and industry grid/industry 4.0. In: Intelligent transportation systems–problems and perspectives. Springer, pp 3–35

  5. Schlingensiepen J, Mehmood R, Nemtanu FC (2015) Framework for an autonomic transport system in smart cities. Cybernetics and Information Technologies 15(5):50–62

    Article  Google Scholar 

  6. Mehmood R, Meriton R, Graham G, Hennelly P, Kumar M (2017) Exploring the influence of big data on city transport operations: a markovian approach. Int J Oper Prod Manag 37(1):75–104

    Article  Google Scholar 

  7. Litman T (2018) Autonomous vehicle implementation predictions: implications for transport planning. Victoria Transport Policy Institute, pp 1–34

  8. Garsten E (2018) Sharp growth in autonomous car market value predicted but may be stalled by Rise in consumer fear. Forbes

  9. Trigueiros P, Ribeiro F, Reis LP (2012) A comparison of machine learning algorithms applied to hand gesture recognition. In: 7th Iberian conference on information systems and technologies (CISTI 2012)

  10. Greenough J (2016) The connected car report: forecasts, competing technologies, and leading manufacturers. Business Insider jun 2016

  11. Visiongain (2017) Top 20 connected car companies 2017 PR Newswire

  12. Hatcher WG, Yu W (2018) A survey of deep learning platforms, applications and emerging research trends. IEEE Access 6:24411–24432

    Article  Google Scholar 

  13. Sanberg WP, Dubbleman G, De With PHN (2017) Free-Space detection with self-supervised and online trained fully convolutional networks. Cornell university library

  14. Zhou S, Gong J, Xiong G, Chen H, Iagnemma K (2010) Road detection using support vector machine based on online learning and evaluation. In: 2010 IEEE intelligent vehicles symposium, pp 256–261

  15. Liu X, Deng Z, Yang G (2017) Drivable road detection based on dilated FPN with feature aggregation. In: 2017 international conference on tools with artificial intelligence, pp 1128–1134

  16. Liu X, Deng Z (2017) Segmentation of drivable road using deep fully convolutional residual network with pyramid pooling. Cogn Comput 10:272–281

    Article  Google Scholar 

  17. Laddha A, Kocamaz MK, Navarro-serment LE, Hebert M (2016) Map-supervised road detection. In: 2016 IEEE intelligent vehicles symposium (IV), (Iv), pp 0–5

  18. Zhang Z, Liu Q, Wang Y (2017) Road extraction by deep residual U-Net. IEEE Geosci Remote Sens Lett 18:1–5

  19. Yu N, Lu H, Kagemoto K, Hirano N, Yang S, Serikawa S (2017) Recognition of road in bad weather using deep learning. In: Proceedings of the 5th IIAE international conference on industrial application engineering, pp 1–4

  20. Chen Z, Chen Z (2017) RBNet: a deep neural network for unified road and road boundary detection. In: Liu D, Xie S, Li Y, Zhao D, el al (eds) Neural information processing. ICONIP 2017. Lecture notes in computer science, pp 10634:677–687

  21. Siegel JE, Erb DC, Sarma SE (2017) A survey of the connected vehicle landscape–architectures, enabling technologies applications, and development areas. IEEE Trans Intell Transp Syst 19:1–16

  22. Kumru M, Debada E, Makarem L, Gillet D (2017) Mobility-on-demand scenarios relying on lightweight autonomous and connected vehicles for large pedestrian areas and intermodal hubs. In: 2017 2nd IEEE international conference on intelligent transportation engineering, ICITE 2017, pp 178–183

  23. Datta SK, Da Costa RPF, Harri J, Bonnet C (2016) Integrating connected vehicles in internet of things ecosystems challenges and solutions. In: WoWMom 2016 - 17th international symposium on a world of wireless mobile and multimedia networks

  24. Gora P, Rub I (2016) Traffic models for self-driving connected cars. 6th Transport Research Arena 14:2207–2216

    Google Scholar 

  25. Strategy & PWC (2017) Connected and Autonomous Vehicles: Revolutionising mobility in society. International Automotive Summit, pp 24

  26. Mehmood R, Alam F, Albogami NN, Katib I, Albeshri A, Altowaijri SM (2017) UTiLearn: A personalised ubiquitous teaching and learning system for smart societies. IEEE Access 5:2615–2635

    Article  Google Scholar 

  27. Muhammed T, Mehmood R, Albeshri A, Katib I (2018) Ubehealth: a personalized ubiquitous cloud and edge-enabled networked healthcare system for smart cities. IEEE Access 6:32258–32285

    Article  Google Scholar 

  28. Büscher M, Coulton P, Efstratiou C, Gellersen H, Hemment D, Mehmood R, Sangiorgi D (2009) Intelligent mobility systems: some socio-technical challenges and opportunities. In: International conference on communications infrastructure. systems and applications in Europe. Springer, pp 140–152

  29. Arfat Y, Aqib M, Mehmood R, Albeshri A, Katib I, Albogami N, Alzahrani A (2017) Enabling smarter societies through mobile big data fogs and clouds. Prog Comput Sci 109:1128–1133

    Article  Google Scholar 

  30. Alsolami B, Mehmood R, Albeshri A (2020) Hybrid statistical and machine learning methods for road traffic prediction: a review and tutorial. In: Mehmood R, See S, Katib I, Chlamtac I (eds) Smart infrastructure and applications: foundations for smarter cities and societies. Springer, Cham, pp 115–133

  31. Mehmood R, Nekovee M (2007) Vehicular ad hoc and grid networks: discussion, design and evaluation. In: Proceedings of the 14th world congress on intelligent transport systems (ITS), held Beijing, October 2007

  32. Gillani S, Shahzad F, Qayyum A, Mehmood R (2013) A survey on security in vehicular ad hoc networks. In: International workshop on communication technologies for vehicles. Springer, pp 59–74

  33. Alvi A, Greaves D, Mehmood R (2010) Intra-vehicular verification and control: a two-pronged approach. In: 2010 7th international symposium on communication systems, networks & digital signal processing (CSNDSP 2010). IEEE, pp 401–405

  34. Nabi Z, Alvi A, Mehmood R (2011) Towards standardization of in-car sensors. In: International workshop on communication technologies for vehicles. Springer, pp 216–223

  35. Alazawi Z, Abdljabar MB, Altowaijri S, Vegni AM, Mehmood R (2012) Icdms: an intelligent cloud based disaster management system for vehicular networks. In: International workshop on communication technologies for vehicles. Springer, pp 40–56

  36. Alam F, Mehmood R, Katib I (2017) D2TFRS: an object recognition method for autonomous vehicles based on rgb and spatial values of pixels. In: Smart societies, infrastructure, technologies and applications. SCITA 2017. Lecture notes of the institute for computer sciences, social informatics and telecommunications engineering (LNICST), vol 224. Springer, pp 155–168

  37. Alam F, Mehmood R, Katib I (2020) Comparison of decision trees and deep learning for object classification in autonomous driving. Springer International Publishing, Cham, pp 135–158

    Google Scholar 

  38. Aqib M, Mehmood R, Alzahrani A, Katib I (2019) A smart disaster management system for future cities using deep learning, GPUs, and in-memory computing. In: Mehmood R, See S, Katib I, Chlamtac I (eds) Smart infrastructure and applications: foundations for smarter cities and societies. https://doi.org/10.1007/978-3-030-13705-2_7. Springer

  39. Aqib M, Mehmood R, Alzahrani A, Katib I (2019) In-memory deep learning computations on GPUs for prediction of road traffic incidents using big data fusion. In: Mehmood R, See S, Katib I, Chlamtac I (eds) Smart infrastructure and applications: foundations for smarter cities and societies. Springer, DOI https://doi.org/10.1007/978-3-030-13705-2_4, (to appear in print)

  40. Aqib M, Mehmood R, Alzahrani A, Katib I, Albeshri A (2018) A deep learning model to predict vehicles occupancy on freeways for traffic management. International Journal of Computer Science and Network Security (IJCSNS) 18(12):246–254

    Google Scholar 

  41. Aqib M, Mehmood R, Albeshri A, Alzahrani A (2017) Disaster management in smart cities by forecasting traffic plan using deep learning and GPUs. In: Mehmood R, Bhaduri B, Katib I, Chlamtac I (eds) International conference on smart cities, infrastructure, technologies and applications (SCITA 2017); lecture notes of the institute for computer sciences, social-informatics and telecommunications engineering, LNICST, vol 224. Springer, pp 139–154

  42. Aqib M, Mehmood R, Alzahrani A, Katib I, Albeshri A, Altowaijri SM (2019) Rapid transit systems: smarter urban planning using big data, in-memory computing, deep learning, and GPUs. Sustainability 11 (10):2736

    Article  Google Scholar 

  43. Aqib M, Mehmood R, Alzahrani A, Katib I, Albeshri A, Altowaijri SM (2019) Smarter traffic prediction using big data, in-memory computing, deep learning and GPUs. Sensors 19(9):2206

    Article  Google Scholar 

  44. Alazawi Z, Altowaijri S, Mehmood R, Abdljabar MB (2011) Intelligent disaster management system based on cloud-enabled vehicular networks. In: 2011 11th international conference on ITS telecommunications (ITST). IEEE, pp 361–368

  45. Alazawi Z, Alani O, Abdljabar MB, Altowaijri S, Mehmood R (2014) A smart disaster management system for future cities. In: Proceedings of the ACM international workshop on wireless and mobile technologies for smart cities. ACM, p 2014

  46. Alazawi Z, Alani O, Abdljabar MB, Mehmood R (2014) An intelligent disaster management system based evacuation strategies. In: 2014 9th international symposium on communication systems, networks & digital signal processing (CSNDSP). IEEE, pp 673–678

  47. Mehmood R, Graham G (2015) Big data logistics: a health-care transport capacity sharing model. In: Procedia computer science, vol 64. Elsevier, pp 1107–1114

  48. Arfat Y, Mehmood R, Albeshri A (2017) Parallel shortest path graph computations of united states road network data on apache spark. In: International conference on smart cities, infrastructure, technologies and applications. Springer, pp 323–336

  49. Arfat Y, Suma S, Mehmood R, Albeshri A (2020) Parallel shortest path big data graph computations of US road network using apache spark: survey, architecture, and evaluation. In: Mehmood R, See S, Katib I, Chlamtac I (eds) Smart infrastructure and applications: foundations for smarter cities and societies. Springer, Cham, pp 185–214

  50. Mehmood R, Lu JA (2011) Computational markovian analysis of large systems. J Manuf Technol Manag 22(6):804–817

    Article  Google Scholar 

  51. Suma S, Mehmood R, Albugami N, Katib I, Albeshri A (2017) Enabling next generation logistics and planning for smarter societies. Prog Comput Sci 109:1122–1127

    Article  Google Scholar 

  52. Suma S, Mehmood R, Albeshri A (2017) Automatic event detection in smart cities using big data analytics. In: Smart societies, infrastructure, technologies and applications. SCITA lecture notes of the institute for computer sciences, social informatics and telecommunications engineering (LNICST), vol 224. Springer, pp 111–122

  53. Alomari E, Mehmood R (2018) Analysis of tweets in arabic language for detection of road traffic conditions. In: lecture notes of the institute for computer sciences, social-informatics and telecommunications engineering, LNICST, vol 224. Springer, Cham, pp 98–110

  54. Suma S, Mehmood R, Albeshri A (2020) Automatic detection and validation of smart city events using HPC and apache spark platforms. In: Mehmood R, See S, Katib I, Chlamtac I (eds) Smart infrastructure and applications: foundations for smarter cities and societies. Springer, Cham, pp 55–78

  55. Graham G, Mehmood R, Coles E (2015) Exploring future cityscapes through urban logistics prototyping: a technical viewpoint. Supply Chain Management: An International Journal 20(3):341–352

    Article  Google Scholar 

  56. Alam F, Mehmood R, Katib I, Albeshri A (2016) Analysis of eight data mining algorithms for smarter internet of things (IoT). In: International workshop on data mining in IoT systems (DaMIS 2016) analysis, pp 437–442

  57. Alam F, Thayananthan V, Katib I (2016) Analysis of round-robin load-balancing algorithm with adaptive and predictive approaches. In: 11th international conference on control (CONTROL)

  58. Esteva A, Robicquet A, Ramsundar B, Kuleshov V, Depristo M, Chou K, Cui C, Corrado G, Thrun S, Dean J (2019) A guide to deep learning in healthcare. Nat Med 25(January):24–29

  59. Muhammed T, Mehmood R, Albeshri A, Katib I (2018) UbeHealth: a personalized ubiquitous cloud and edge-enabled networked healthcare system for smart cities. IEEE Access 6:32258–32285

    Article  Google Scholar 

  60. Baldi P (2018) Deep learning in biomedical data science. Annual review of biomedical data science

  61. Raissi M (2018) Deep hidden physics models: deep learning of nonlinear partial differential equations. Cornell university library

  62. Wang J, Ma Y, Zhang L, Gao RX, Wu D (2018) Deep learning for smart manufacturing methods and applications. J Manuf Syst 48:1–13

  63. Stolee J, Wang Y (2013) A survey of machine learning techniques for road detection. University of Toronto

  64. Sousa N, Natividade-Jesus E, Almeida A (2017) Dawn of autonomous vehicles: review and challenges ahead. Munic Eng 171(ME1): 3–14

  65. Hillel AB, Lerner R, Levi D, Raz G (2014) Recent progress in road and lane detection: a survey. Mach Vis Appl 25(3):727–745

    Article  Google Scholar 

  66. Wu BF, Lin C, Chen C, Lin C, Chen C (2015) Real-time lane, and vehicle detection. real-time lane and vehicle detection based on a single camera model. Int J Comput Appl 32(2):149–159

    Google Scholar 

  67. Yuying Z, Xiaodong G, Uanyuan W (2010) A model-oriented road detection approach using fuzzy SVM. J Electron 27(6):795–800

  68. Xiao L, Dai B, Liu D, Zhao D, Wu T (2016) Monocular road detection using structured random forest. Int J Adv Robot Syst 13:1–13

    Google Scholar 

  69. Bedawi SM, Kamel MS (2015) Road detection in urban areas using random forest tree-based ensemble classification. In: Kamel M, Campilho A (eds) Image analysis and recognition. ICIAR 2015. Lecture notes in computer science, 9164

  70. Alam F, Mehmood R, Katib I, Nasser N (2017) Data fusion and IoT for smart ubiquitous environments: a survey. IEEE Access 3536(c):1–24

    Google Scholar 

  71. Castanedo F (2013) A review of data fusion techniques. The ScientificWorld Journal, 2013

  72. Mangai U, Samanta S, Das S, Chowdhury P (2010) A survey of decision fusion and feature fusion strategies for pattern classification. IETE Tech Rev 27(4):293

    Article  Google Scholar 

  73. Kokar M, Tomasik J, Weyman J (2001) Data vs. decision fusion in the category theory framework. In: Proceedings of FUSION 2001 - 4th international conference on information fusion

  74. Zhu M (2013) When is the majority-vote classifier beneficial? pp 1–17

  75. Ruta D, Gabrys B (2005) Classifier selection for majority voting. Information Fusion 6(1):63–81

    Article  Google Scholar 

  76. James G (1998) Majority vote classifiers: theory and applications. PhD thesis, Stanford University

  77. Kuncheva LI, Whitaker CJ, Shipp CA, Duin RPW (2003) Limits on the majority vote accuracy in classifier fusion. Pattern Anal Applic 6(1):22–31

    Article  MathSciNet  Google Scholar 

  78. Zhang Y, Zhang H, Cai J, Yang B (2014) A weighted voting classifier based on differential evolution. Abstr Appl Anal 2014:6

  79. Kim H, Kim H, Moon H, Ahn H (2011) A weight-adjusted voting algorithm for ensemble of classifiers. Journal of the Korean Statistical Society 40:437–449

    Article  MathSciNet  Google Scholar 

  80. Kuncheva LI, Rodríguez JJ (2014) A weighted voting framework for classifiers ensembles. Knowl Inf Syst 38(2):259–275

    Article  Google Scholar 

  81. Limmer M, Forster J, Baudach D, Schüle F, May CV (2016) Robust deep-learning-based road-prediction for augmented reality navigation systems. Cornell university library

  82. Han X, Wang H, Lu J, Zhao C (2017) Road detection based on the fusion of Lidar and image data. Int J Adv Robot Syst, (200), pp 1–10

  83. Xiao L, Wang R, Dai B, Fang Y, Liu D, Wu T (2017) Hybrid conditional random field based camera-LIDAR fusion for road detection. Inf Sci 0:1–16

    Google Scholar 

  84. Tsai L-W, Hsieh J-W, Chuang C-H, Fan K-C (2008) Lane detection using directional random walks. In: Intelligent vehicles symposium, 2008 IEEE

  85. Li Q, Zheng N, Cheng H (2004) Springrobot: a prototype autonomous vehicle and its algorithms for lane detection. IEEE Trans Intell Transp Syst 5(4):300–308

  86. Shu Y, Tan Z (2004) Vision based lane detection in autonomous vehicle. In: Fifth world congress on intelligent control and automation

  87. Geiger A, Lenz P, Stiller C, Urtasun R (2013) Vision meets robotics: the KITTI dataset. Int J Robot Res 32(11):1231–1237

    Article  Google Scholar 

  88. Quinlan RJ (1993) C4.5: programs for machine learning. Morgan Kaufmann Publishers Inc, San Francisco

    Google Scholar 

  89. Ho TK (1995) Random decision forest. In: Proceedings of 3rd international conference on document analysis and recognition

  90. Svozil D, Kvasnicka V, Pospichal J (1997) Introduction to multi-layer feed-forward neural networks. Chemometr Intell Lab Syst 39(1):43–62

    Article  Google Scholar 

  91. Candel A, LeDell E, Parmar V, Arora A (2018) Deep learning with H2O. H2O.ai Inc, (June)

  92. Usman S, Mehmood R, Katib I (2018) Big data and HPC convergence: The cutting edge and outlook. In: International conference on smart cities, infrastructure, technologies and applications (SCITA 2017); lecture notes of the institute for computer sciences, social-informatics and telecommunications engineering, LNICST, vol 224. Springer, Cham, pp 11–26

  93. Beygelzimer A, Kakade S, Langford J (2006) Cover trees for nearest neighbor. In: Proceedings of the 23rd international conference on Machine learning - ICML ’06, p 7

  94. Sokolova M, Lapalme G (2009) A systematic analysis of performance measures for classification tasks. Inf Process Manag 45:10

  95. Wickham H (2016) ggplot2: elegant graphics for data analysis. Springer, New York

    Book  Google Scholar 

  96. Kassambara A (2018) ggpubr: ‘ggplot2’ based publication ready plots. R package version 0.2

  97. Wilke CO (2018) cowplot: streamlined plot theme and plot annotations for ‘ggplot2’. R package version 0.9.3

Download references

Acknowledgements

This project was funded by the Deanship of Scientific Research (DSR) at King Abdulaziz University, Jeddah, under grant number RG-11-611-40. The authors, therefore, acknowledge with thanks DSR for technical and financial support. The experiments performed in this paper were executed on the Aziz supercomputer being managed by the HPC Center at the King Abdul-Aziz University.

Author information

Authors and Affiliations

  1. Department of Computer Science, Faculty of Computing and Information Technology, King Abdulaziz University, Jeddah, 21589, Kingdom of Saudi Arabia

    Furqan Alam, Iyad Katib & Aiiad Albeshri

  2. High Performance Computing Center, King Abdulaziz University, Jeddah, 21589, Kingdom of Saudi Arabia

    Rashid Mehmood

  3. Faculty of Computing and Information Technology, Northern Border University, Rafha, 91911, Kingdom of Saudi Arabia

    Saleh M. Altowaijri

Authors
  1. Furqan Alam
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Rashid Mehmood
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Iyad Katib
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Saleh M. Altowaijri
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Aiiad Albeshri
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Furqan Alam.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Alam, F., Mehmood, R., Katib, I. et al. TAAWUN: a Decision Fusion and Feature Specific Road Detection Approach for Connected Autonomous Vehicles. Mobile Netw Appl 28, 636–652 (2023). https://doi.org/10.1007/s11036-019-01319-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11036-019-01319-2

Keywords