Skip to main content

Advertisement

SpringerLink
Log in

Multi-sensor information fusion for efficient smart transport vehicle tracking and positioning based on deep learning technique

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

A smart transportation system relies on connected environments and cloud systems for ease of operation and assisted routing. Smart vehicles understand the environment through multi-sensor, network, and pervasive systems for gaining useful information. The problem arises with the absence of useful information in explicit scenarios where heterogeneous information becomes mandatory. This article aims to improve transportation support's effectiveness using a discrete behavior information fusion (DBIF) based on the deep learning technique by considering the contradiction in information availability. This proposed technique observes the vehicles' behavior and response to the scene throughout the route displacement. The deep learning model achieves greater accuracy in target detection and classification. The learning output is the independent fusion of behavior (response output) and information (sensed). This sensed information is useful in categorizing further deviations and stipulations for the progressive displacements. The stipulated information and its deviations are recurrently categorized using support vector machine learning. The information provides accurate positioning and tracking of smart vehicles by reducing approximation errors and complexity. The simulation results illustrate the proposed technique's efficiency by improving the accuracy of 92.078% and fusion rate of 0.9741 and reducing error of 0.0662, complexity of 0.0717, and fusion time of 0.9938 compared to existing methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Fascista A, Ciccarese G, Coluccia A, Ricci G (2018) Angle of arrival-based cooperative positioning for smart vehicles. IEEE Trans Intell Transp Syst 19(9):2880–2892

    Google Scholar 

  2. Zhang Y, Zhang G, Fierro R, Yang Y (2018) Force-driven traffic simulation for a future connected autonomous vehicle-enabled smart transportation system. IEEE Trans Intell Transp Syst 19(7):2221–2233

    Google Scholar 

  3. El-Said M, Mansour S, Bhuse V (2018) DSRC based sensor-pooling protocol for connected vehicles in future smart cities. Procedia Comput Sci 140:70–78

    Google Scholar 

  4. An C, Wu C (2019) Traffic big data assisted V2X communications toward smart transportation. Wireless Netw 26(3):1601–1610

    Google Scholar 

  5. Mei Q, Gül M, Shirzad-Ghaleroudkhani N (2020) Towards smart cities: crowdsensing-based monitoring of transportation infrastructure using in-traffic vehicles. J Civ Struct Heal Monit 10(4):653–665

    Google Scholar 

  6. Geissler F, Unnervik A, Paulitsch M (2020) A plausibility-based fault detection method for high-level fusion perception systems. IEEE Open J Intell Transp Syst 1:176–186

    Google Scholar 

  7. Hu F, Wu G (2020) Distributed error correction of EKF algorithm in multi-sensor fusion localization model. IEEE Access 8:93211–93218

    Google Scholar 

  8. Zhang W, Qi Y, Zhou Z, Biancardo SA, Wang Y (2018) Method of speed data fusion based on Bayesian combination algorithm and high-order multi-variable Markov model. IET Intel Transp Syst 12(10):1312–1321

    Google Scholar 

  9. Lau BPL, Marakkalage SH, Zhou Y, Hassan NU, Yuen C, Zhang M, Tan U-X (2019) A survey of data fusion in smart city applications. Inform Fusion 52:357–374

    Google Scholar 

  10. Engelbert A, Mirwani R (2019) Vehicle-in-the-loop with sensor fusion testing for efficient ADAS and AD tests. ATZelectronics Worldwide 14(12):84–89

    Google Scholar 

  11. Zhang J, Xiao W, Coifman B, Mills JP (2020) Vehicle tracking and speed estimation from roadside lidar. IEEE J Sel Topics Appl Earth Observ Rem Sens 13:5597–5608

    Google Scholar 

  12. Wang X, Zhang J, Tian X, Gan X, Guan Y, Wang X (2018) Crowdsensing-based consensus incident report for road traffic acquisition. IEEE Trans Intell Transp Syst 19(8):2536–2547

    Google Scholar 

  13. Manfreda A, Ljubi K, Groznik A (2019) Autonomous vehicles in the smart city era: An empirical study of adoption factors important for millennials, International Journal of Information Management, p. 102050

  14. Espinoza C, Munizaga M, Bustos B, Trépanier M (2018) Assessing the public transport travel behavior consistency from smart card data. Transp Res Procedia 32:44–53

    Google Scholar 

  15. Haider SW, Zhuang G, Ali S (2019) Identifying and bridging the attitude-behavior gap in sustainable transportation adoption. J Ambient Intell Humaniz Comput 10(9):3723–3738

    Google Scholar 

  16. Li X, Chen W, Chan C, Li B, Song X (2019) Multi-sensor fusion methodology for enhanced land vehicle positioning. Inform Fusion 46:51–62

    Google Scholar 

  17. Sun Y, Guan L, Chang Z, Li C, Gao Y (2019) Design of a low-cost indoor navigation system for food delivery robot based on multi-sensor information fusion. Sensors 19(22):4980

    Google Scholar 

  18. Meng T, Jing X, Yan Z, Pedrycz W (2020) A survey on machine learning for data fusion. Inform Fusion 57:115–129

    Google Scholar 

  19. Shahian Jahromi B, Tulabandhula T, Cetin S (2019) Real-time hybrid multi-sensor fusion framework for perception in autonomous vehicles. Sensors 19(20):4357

    Google Scholar 

  20. Nweke HF, Teh YW, Mujtaba G, Al-Garadi MA (2019) Data fusion and multiple classifier systems for human activity detection and health monitoring: Review and open research directions. Inform Fusion 46:147–170

    Google Scholar 

  21. Bai J, Li S, Wang J, Huang L, Dong L, Tong P (2020) Drivable area detection and vehicle localization based on multi-sensor information (No. 2020–01–1027). SAE Technical Paper

  22. Wang Z, Wu Y, Niu Q (2019) Multi-sensor fusion in automated driving: a survey. IEEE Access 8:2847–2868

    Google Scholar 

  23. Qi J, Yang P, Newcombe L, Peng X, Yang Y, Zhao Z (2020) An overview of data fusion techniques for Internet of Things enabled physical activity recognition and measure. Inform Fusion 55:269–280

    Google Scholar 

  24. Diez-Olivan A, Del Ser J, Galar D, Sierra B (2019) Data fusion and machine learning for industrial prognosis: trends and perspectives towards Industry 4.0. Inform Fusion 50:92–111

    Google Scholar 

  25. Fayyad J, Jaradat MA, Gruyer D, Najjaran H (2020) Deep learning sensor fusion for autonomous vehicle perception and localization: a review. Sensors 20(15):4220

    Google Scholar 

  26. Mohanty SN, Lydia EL, Elhoseny M, Al Otaibi MMG, Shankar K (2020) Deep learning with LSTM based distributed data mining model for energy efficient wireless sensor networks. Phys Commun 40:101097

    Google Scholar 

  27. Wei P, Wang B (2020) Multi-sensor detection and control network technology based on parallel computing model in robot target detection and recognition. Comput Commun 159:215–221

    Google Scholar 

  28. Nguyen TT, Perschewski JO, Engel F, Kruesemann J, Sitzmann J, Spehr J, Kruse R (2019) Reliability-aware and robust multi-sensor fusion toward ego-lane estimation using artificial neural networks. In Information quality in Information Fusion and decision making (pp 423–454). Springer, Cham

  29. Kartsch VJ, Benatti S, Schiavone PD, Rossi D, Benini L (2018) A sensor fusion approach for drowsiness detection in wearable ultra-low-power systems. Inform Fusion 43:66–76

    Google Scholar 

  30. Xiong H, Mai Z, Tang J, He F (2019) Robust GPS/INS/DVL navigation and positioning method using adaptive federated strong tracking filter based on weighted least square principle. IEEE Access 7:26168–26178

    Google Scholar 

  31. Kong L, Peng X, Chen Y, Wang P, Xu M (2020) Multi-sensor measurement and data fusion technology for manufacturing process monitoring: a literature review. Int J Extreme Manuf 2(2):022001

    Google Scholar 

  32. Kumar AD, Karthika R, Soman KP (2020) Stereo camera and LIDAR sensor fusion-based collision warning system for autonomous vehicles. In Advances in Computational Intelligence Techniques (pp 239–252). Springer, Singapore

  33. Wu Y, Ta X, Xiao R, Wei Y, An D, Li D (2019) Survey of underwater robot positioning navigation. Appl Ocean Res 90:101845

    Google Scholar 

  34. Feng M, Chen Y, Zheng T, Cen M, Xiao H (2019) Research on information fusion method of millimetre wave radar and monocular camera for intelligent vehicle. J Phys: Conf Ser 1314:012059

    Google Scholar 

  35. Blasch E, Grewe LL, Waltz EL, Bendich P, Pavlovic V, Kadar I, Chong CY (2020) Machine learning in/with information fusion for infrastructure understanding, panel summary. Signal Process, Sens/Inform Fusion, Target Recognit XXIX Int Soc Opt Photon 11423:1142300

    Google Scholar 

  36. Feng X, Zhang J, Chen J, Wang G, Zhang L, Li R (2018) Design of intelligent bus positioning based on Internet of Things for smart campus. IEEE Access 6:60005–60015

    Google Scholar 

  37. Guan L, Cong X, Zhang Q, Liu F, Gao Y, An W, Noureldin A (2020) A comprehensive review of micro-inertial measurement unit based intelligent PIG multi-sensor fusion technologies for small-diameter pipeline surveying. Micromachines 11(9):840

    Google Scholar 

  38. Shen J, Won JY, Chen Z, Chen QA (2020) Drift with devil: security of multi-sensor fusion based localization in high-level autonomous driving under {GPS} Spoofing. In 29th {USENIX} Security Symposium ({USENIX} Security 20) (pp. 931–948)

  39. Chakraborty P, Adu-Gyamfi YO, Poddar S, Ahsani V, Sharma A, Sarkar S (2018) Traffic congestion detection from camera images using deep convolution neural networks. Transp Res Rec 2672(45):222–231

    Google Scholar 

  40. Wright RG (2019) Intelligent autonomous ship navigation using multi-sensor modalities. TransNav: International Journal on Marine Navigation and Safety of Sea Transportation, 13(3).

  41. Xia Y, Jian X, Yan B, Su D (2019) Infrastructure safety oriented traffic load monitoring using multi-sensor and single camera for short and medium span bridges. Rem Sens 11(22):2651

    Google Scholar 

  42. Li HX, Guo XF (2020) Research on Multi-Source Information Fusion Technology. In International Academic Conference on Frontiers in Social Sciences and Management Innovation (IAFSM 2019) (pp. 24–28). Atlantis Press

  43. Hu JW, Zheng BY, Wang C, Zhao CH, Hou XL, Pan Q, Xu Z (2020) A survey on multi-sensor fusion based obstacle detection for intelligent ground vehicles in off-road environments. Front Inform Technol Electron Eng 21:675–692

    Google Scholar 

  44. Shepelev V, Aliukov S, Nikolskaya K, Das A, Slobodin I (2020) The use of multi-sensor video surveillance system to assess the capacity of the road network. Transp Telecommun J 21(1):15–31

    Google Scholar 

  45. Bouain M, Ali KM, Berdjag D, Fakhfakh N, Atitallah RB (2018) An embedded multi-sensor data fusion design for vehicle perception tasks. J Commun 13(1):8–14

    Google Scholar 

  46. Gabela J, Kealy A, Li S, Hedley M, Moran W, Ni W, Williams S (2019) The effect of linear approximation and Gaussian noise assumption in multi-sensor positioning through experimental evaluation. IEEE Sens J 19(22):10719–10727

    Google Scholar 

  47. Liu Z, Suo C, Liu Y, Shen Y, Qiao Z, Wei H, Liu YH (2020) Deep learning-based localization and perception systems: approaches for autonomous cargo transportation vehicles in large-scale, semiclosed environments. IEEE Robot Autom Mag 27(2):139–150

    Google Scholar 

  48. Aydin I, Celebi SB, Barmada S, Tucci M (2018) Fuzzy integral-based multi-sensor fusion for arc detection in the pantograph-catenary system. Proc Inst Mech Eng, Part F: J Rail Rapid Transit 232(1):159–170

    Google Scholar 

  49. Ciaparrone G, Sánchez FL, Tabik S, Troiano L, Tagliaferri R, Herrera F (2020) Deep learning in video multi-object tracking: a survey. Neurocomputing 381:61–88

    Google Scholar 

  50. Li Y, Zhang W, Ji X, Ren C, Wu J (2019) Research on lane a compensation method based on multi-sensor fusion. Sensors 19(7):1584

    Google Scholar 

  51. Aguileta AA, Brena RF, Mayora O, Molino-Minero-Re E, Trejo LA (2019) Multi-sensor fusion for activity recognition—a survey. Sensors 19(17):3808

    Google Scholar 

  52. Chen X, Ji J, Wang Y (2020) Robust cooperative multi-vehicle tracking with inaccurate self-localization based on onboard sensors and inter-vehicle communication. Sensors 20(11):3212

    Google Scholar 

  53. Raz AK, Sabatini R (2021) Information Fusion as an Autonomy enabler for UAV Traffic Management. In AIAA Scitech 2021 Forum (p. 0658)

  54. Broughton G, Majer F, Rouček T, Ruichek Y, Yan Z, Krajník T (2021) Learning to see through the haze: multi-sensor learning-fusion system for vulnerable traffic participant detection in fog. Robot Auton Syst 136:103687

    Google Scholar 

  55. Shen J, Won JY, Chen Z, Chen QA (2020) Drift with devil: security of multi-sensor fusion based localization in high-level autonomous driving under GPS spoofing (Extended Version). arXiv preprint arXiv: 2006.10318

  56. Brambilla M, Nicoli M, Soatti G, Deflorio F (2020) Augmenting vehicle localization by cooperative sensing of the driving environment: insight on data association in urban traffic scenarios. IEEE Trans Intell Transp Syst 21(4):1646–1663

    Google Scholar 

  57. Colombaroni C, Fusco G, Isaenko N (2020) Coherence analysis of road safe speed and driving behaviour from floating car data. IET Intel Transp Syst 14(9):985–992

    Google Scholar 

  58. Yu Y, Tang X, Wu J, Kim B, Song T, Han Z (2019) Multi-leader–follower game for mec-assisted fusion-based vehicle on-road analysis. IEEE Trans Veh Technol 68(11):11200–11212

    Google Scholar 

  59. Cai P, Wang S, Sun Y, Liu M (2020) Probabilistic end-to-end vehicle navigation in complex dynamic environments with multimodal sensor fusion. IEEE Robotics and Automation Letters, pp. 1–1, 2020

  60. Belhajem I, Maissa YB, Tamtaoui A (2018) Improving low cost sensor based vehicle positioning with machine learning. Control Eng Pract 74:168–176

    Google Scholar 

  61. Balbin PPF, Barker JC, Leung CK, Tran M, Wall RP, Cuzzocrea A (2020) Predictive analytics on open big data for supporting smart transportation services. Procedia Comput Sci 176:3009–3018

    Google Scholar 

  62. Zhang Y, Martens K, Long Y (2018) Revealing group travel behavior patterns with public transit smart card data. Travel Behav Soc 10:42–52

    Google Scholar 

  63. Gohar M, Muzammal M, Rahman AU (2018) SMART TSS: defining transportation system behavior using big data analytics in smart cities. Sustain Cities Soc 41:114–119

    Google Scholar 

  64. Belhajem I, Maissa YB, Tamtaoui A (2017) Improving vehicle localization in a smart city with low cost sensor networks and support vector machines. Mobile Netw Appl 23(4):854–863

    Google Scholar 

  65. Babar M, Arif F (2018) Real-time data processing scheme using big data analytics in internet of things based smart transportation environment. J Ambient Intell Humaniz Comput 10(10):4167–4177

    Google Scholar 

  66. Belhagen I, Maissa YB, Tamtaoui A (2018) Improving vehicle localization in a smart city with low cost sensor networks and support vector machines. Mobile Net Appl 23(4):854–863

    Google Scholar 

  67. Xin C, Jingmei Z, Xiangmo Z, Hongfei W, Hui C (2019) Vehicle ego-localization based on the fusion of optical flow and feature points matching. IEEE Access 7:167310–167319

    Google Scholar 

  68. Alam F, Mehmood R, Katib I, Altowaijri SM, Albeshri A (2019) TAAWUN: a decision fusion and feature specific road detection approach for connected autonomous vehicles. Mobile Netw Appl. https://doi.org/10.1007/s11036-019-01319-2

    Article  Google Scholar 

  69. Zheng J-C, Wang Y, Lin C-C, Zhang X-L, Liu J, Ji L-W (2017) A fusion algorithm of target dynamic information for asynchronous multi-sensors. Microsyst Technol 24(10):3995–4005

    Google Scholar 

  70. Plangi S, Hadachi A, Lind A, Bensrhair A (2018) Real-time vehicles tracking based on mobile multi-sensor fusion. IEEE Sens J 18(24):10077–10084

    Google Scholar 

  71. Leung CK, Braun P, Cuzzocrea A (2019) AI-based sensor information fusion for supporting deep supervised learning. Sensors 19(6):1345

    Google Scholar 

  72. Wang H, Li S, Song L, Cui L, Wang P (2019) An enhanced intelligent diagnosis method based on multi-sensor image fusion via improved deep learning network. IEEE Trans Instrum Meas 69(6):2648–2657

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Information Technology, Vels Institute of Science, Technology and Advanced Studies, Chennai, India

    G. Suseendran & D. Akila

  2. Anna Adarsh College for Women, Chennai, India

    Hannah Vijaykumar & T. Nusrat Jabeen

  3. Shri Krishnaswamy College for Women, Chennai, India

    R. Nirmala

  4. Graduate School, Duy Tan University, Da Nang, 550000, Viet Nam

    Anand Nayyar

  5. Faculty of Information Technology, Duy Tan University, Da Nang, 550000, Viet Nam

    Anand Nayyar

Authors
  1. G. Suseendran
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. D. Akila
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hannah Vijaykumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. T. Nusrat Jabeen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. R. Nirmala
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Anand Nayyar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anand Nayyar.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Suseendran, G., Akila, D., Vijaykumar, H. et al. Multi-sensor information fusion for efficient smart transport vehicle tracking and positioning based on deep learning technique. J Supercomput 78, 6121–6146 (2022). https://doi.org/10.1007/s11227-021-04115-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-021-04115-6

Keywords

Search

Navigation