Skip to main content
Springer Nature Link
Log in

Human-Vehicle Cooperative Visual Perception for Autonomous Driving Under Complex Traffic Environments

  • Conference paper
  • First Online:
Computer Vision – ECCV 2022 Workshops (ECCV 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13801))

Included in the following conference series:

  • 2310 Accesses

Abstract

Human-vehicle cooperative driving has become one of the critical stages to achieve a higher level of driving automation. For an autonomous driving system, the complex traffic environments bring great challenges to its visual perception tasks. Based on the gaze points of human drivers and the images detected from a semi-automated vehicle, this work proposes a framework to fuse their visual characteristics based on the Laplacian Pyramid algorithm. By adopting Extended Kalman Filter, we improve the detection accuracy of objects with interaction risk. This work also reveals that the cooperative visual perception framework can predict the trajectory of objects with interaction risk better than simple object detection algorithms. The findings can be applied in improving visual perception ability and making proper decisions and control for autonomous vehicles.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Ballard, D.H.: Generalizing the hough transform to detect arbitrary shapes. Pattern Recogn. 13(2), 111–122 (1981)

    Article  MATH  Google Scholar 

  2. Bennett, R., Vijaygopal, R., Kottasz, R.: Attitudes towards autonomous vehicles among people with physical disabilities. Transport. Res. Part A: Policy Pract. 127, 1–17 (2019)

    Google Scholar 

  3. Bochkovskiy, A., Wang, C., Liao, H.M.: Yolov4: optimal speed and accuracy of object detection. CoRR abs/2004.10934 (2020). https://arxiv.org/abs/2004.10934

  4. Burt, P., Adelson, E.: The Laplacian pyramid as a compact image code. IEEE Trans. Commun. 31(4), 532–540 (1983)

    Article  Google Scholar 

  5. Correia, G., Arem, B.V.: Solving the user optimum privately owned automated vehicles assignment problem (uo-poavap): a model to explore the impacts of self-driving vehicles on urban mobility. Transport. Res. Part B: Methodol. 87, 64–88 (2016)

    Article  Google Scholar 

  6. Dabbour, E., Easa, S.: Proposed collision warning system for right-turning vehicles at two-way stop-controlled rural intersections. Transport. Res. Part C: Emerg. Technol. 42(42), 121–131 (2014)

    Article  Google Scholar 

  7. Endsley, M.R.: Toward a theory of situation awareness in dynamic systems. Human Fact. J. Human Factors Ergon. Soc. 37(1), 32–64 (1995)

    Google Scholar 

  8. Gai, W., Liu, Y., Zhang, J., Jing, G.: An improved tiny yolov3 for real-time object detection. Syst. Sci. Control Eng. Open Access J. 9(1), 314–321 (2021)

    Article  Google Scholar 

  9. Gazit, R.Y.: Steering wheels for vehicle control in manual and autonomous driving (2015)

    Google Scholar 

  10. Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 580–587 (2014)

    Google Scholar 

  11. Gold, C., Bengler, K.: Taking over control from highly automated vehicles. In: Human Factors and Ergonomics Society Annual Meeting Proceedings, vol. 8, no. 64 (2014)

    Google Scholar 

  12. Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)

    Google Scholar 

  13. Huang, T., Fu, R., Chen, Y.: Deep driver behavior detection model based on human brain consolidated learning for shared autonomy systems. Measurement 179, 109463 (2021)

    Article  Google Scholar 

  14. Kalra, N., Paddock, S.M.: Driving to safety: how many miles of driving would it take to demonstrate autonomous vehicle reliability? Transport. Res. Part A: Policy Pract. 94, 182–193 (2016)

    Google Scholar 

  15. LeCun, Y., Bengio, Y.: Deep learning. Nature 521(7553), 436–444 (2015)

    Article  Google Scholar 

  16. Li, J., Yao, L., Xu, X., Cheng, B., Ren, J.: Deep reinforcement learning for pedestrian collision avoidance and human-machine cooperative driving. Inf. Sci. 532, 110–124 (2020)

    Article  Google Scholar 

  17. Li, Y., Sun, D., Zhao, M., Dong, C., Cheng, S., Xie, F.: Switched cooperative driving model towards human vehicle copiloting situation: A cyberphysical perspective. J. Adv. Transp. 2018, 1–11 (2018)

    Google Scholar 

  18. Lin, T.Y., Dollar, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)

    Google Scholar 

  19. Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48

    Chapter  Google Scholar 

  20. Loshchilov, I., Hutter, F.: SGDR: stochastic gradient descent with warm restarts. In: International Conference on Learning Representations (2017)

    Google Scholar 

  21. Ma, R., Kaber, D.B.: Situation awareness and workload in driving while using adaptive cruise control and a cell phone. Int. J. Ind. Ergon. 35(10), 939–953 (2005)

    Article  Google Scholar 

  22. Murthy, C., Hashmi, M.F., Bokde, N., Geem, Z.W.: Investigations of object detection in images/videos using various deep learning techniques and embedded platforms-a comprehensive review. Appl. Sci. 10(9), 3280 (2020)

    Google Scholar 

  23. NgoGia, T., Li, Y., Jin, D., Guo, J., Li, J., Tang, Q.: Real-time sea cucumber detection based on YOLOv4-tiny and transfer learning using data augmentation, pp. 119–128 (2021). https://doi.org/10.1007/978-3-030-78811-7-12

  24. Pan, S.J., Qiang, Y.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2010)

    Article  Google Scholar 

  25. Politis, I., Brewster, S.: Language-based multimodal displays for the handover of control in autonomous cars. In: Proceedings of the 7th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (2015)

    Google Scholar 

  26. Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 779–788 (2016). https://doi.org/10.1109/CVPR.2016.91

  27. Son, T., Mita, S., Takeuchi, A.: Road detection using segmentation by weighted aggregation based on visual information and a posteriori probability of road regions. In: 2008 IEEE International Conference on Systems, Man and Cybernetics, pp. 3018–3025 (2008). https://doi.org/10.1109/ICSMC.2008.4811758

  28. Spence, C., Ho, C.: Tactile and multisensory spatial warning signals for drivers. IEEE Trans. Haptics 1(2), 121–129 (2008)

    Article  Google Scholar 

  29. Stender-Vorwachs, J., Steege, H.: Legal aspects of autonomous driving. Internationales verkehrswesen 70(MAYSPEC.), 18–20 (2018)

    Google Scholar 

  30. Walt, S., Schnberger, J.L., Nunez-Iglesias, J., Boulogne, F., Yu, T.: Scikit-image: image processing in python. arXiv e-prints (2014)

    Google Scholar 

  31. Wang, J., Huang, H., Zhi, P., Shen, Z., Zhou, Q.: Review of development and key technologies in automatic driving. Application of Electronic Technique (2019)

    Google Scholar 

  32. Wang, Y., Papageorgiou, M.: Real-time freeway traffic state estimation based on extended kalman filter: a general approach. Transport. Res. Part B: Methodol. 39(2), 141–167 (2007)

    Google Scholar 

  33. Liu, W., et al.: SSD: single shot multibox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_2

    Chapter  Google Scholar 

  34. Xu, J., Wang, H., Huang, H.: Research of adaptive threshold edge detection algorithm based on statistics canny operator. In: MIPPR 2015: Automatic Target Recognition and Navigation, vol. 9812, p. 98121D (2015)

    Google Scholar 

  35. Yang, W., Zhang, X., Lei, Q., Cheng, X.: Research on longitudinal active collision avoidance of autonomous emergency braking pedestrian system (AEB-P). Sensors 19(21), 4671 (2019)

    Google Scholar 

  36. Yue, K., Victorino, A.C.: Human-vehicle cooperative driving using image-based dynamic window approach: System design and simulation. In: 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC) (2016)

    Google Scholar 

  37. Zhang, J., Xiao, W., Coifman, B., Mills, J.P.: Vehicle tracking and speed estimation from roadside lidar. IEEE J. Selected Top. Appl. Earth Observ. Remote Sens. 13, 5597–5608 (2020)

    Article  Google Scholar 

  38. Zhang, T., Mu, D., Ren, S.: Information hiding algorithm based on gaussian pyramid and geronimo hardin massopust multi-wavelet transformation. Int. J. Digit. Content Technol. Appl. 5(3), 210–218 (2011)

    Google Scholar 

  39. Ziemke, T., Schaefer, K.E., Endsley, M.: Situation awareness in human-machine interactive systems. Cogn. Syst. Res. 46, 1–2 (2017)

    Article  Google Scholar 

Download references

Acknowledgments

This work is supported by the National Key Research and Development Program of China under Grant No. 2020AAA0108101.

Author information

Authors and Affiliations

  1. Tongji University, Shanghai, 201804, China

    Yiyue Zhao, Cailin Lei, Yu Shen, Yuchuan Du & Qijun Chen

Authors
  1. Yiyue Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Cailin Lei
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yu Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yuchuan Du
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Qijun Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yu Shen .

Editor information

Editors and Affiliations

  1. IBM Research AI and MIT-IBM Watson AI Lab, Haifa, Israel

    Leonid Karlinsky

  2. Technion – Israel Institute of Technology, Haifa, Israel

    Tomer Michaeli

  3. Kyoto University, Kyoto, Japan

    Ko Nishino

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhao, Y., Lei, C., Shen, Y., Du, Y., Chen, Q. (2023). Human-Vehicle Cooperative Visual Perception for Autonomous Driving Under Complex Traffic Environments. In: Karlinsky, L., Michaeli, T., Nishino, K. (eds) Computer Vision – ECCV 2022 Workshops. ECCV 2022. Lecture Notes in Computer Science, vol 13801. Springer, Cham. https://doi.org/10.1007/978-3-031-25056-9_41

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-25056-9_41

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-25055-2

  • Online ISBN: 978-3-031-25056-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics