Hao Yan
ABSTRACT
One of the most serious factors compromising driving safety is when people in drivers' non-line-of-sight areas rush out suddenly. Existing studies on non-line-of-sight imaging rely on expensive equipment or are limited to severe laboratory conditions (e.g., massive planar reflectors and controlled illumination), rendering these technologies inapplicable in complex driving scenarios. In this paper, we propose a non-line-of-sight moving obstacle detection system Ghost-Probe, which can provide an advanced driver assistance system (ADAS) with sufficient time to respond and stop safely. We design a shadow signal discriminator to assess the weak shadows created by a moving obstacle, such as pedestrians in the blind area, while simultaneously filtering out the impacts of other complicated illumination. Note that we merely use commercial monocular cameras and our system is robust to a wide range of lighting scenarios and planar reflectors. We evaluate the generalizability of our approach using the datasets collected in real-world driving scenarios with a variety of road surface and lighting circumstances. The results indicate that our system can detect the moving pedestrian in the non-line-of-sight area at a distance of 20 meters and offer the ADAS system advance warning to keep a safe distance.
- National Highway Traffic Safety Administration(NHTSA). 2021. NHTSA promotes safe behaviors on our nation's roads. (2021). https://www.nhtsa.gov/road-safetyGoogle Scholar
- automotive World. 2021. V2X is close. Here's what still needs to happen. (2021). https://www.automotiveworld.com/articles/v2x-is-close-heres-what-still-needs-to-happen/Google Scholar
- Alexey Bochkovskiy, Chien-Yao Wang, and Hong-Yuan Mark Liao. 2020. Yolov4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934 (2020).Google Scholar
- Gary Bradski and Adrian Kaehler. 2008. Learning OpenCV: Computer vision with the OpenCV library. " O'Reilly Media, Inc.".Google Scholar
- Yanpeng Cao, Rui Liang, Jiangxin Yang, Yanlong Cao, Zewei He, Jian Chen, and Xin Li. 2022. Computational framework for steady-state NLOS localization under changing ambient illumination conditions. Optics Express 30, 2 (2022), 2438--2452.Google ScholarCross Ref
- Tim Charlet. 2015. Headlight Use Laws for All 50 States. (2015). https://www.yourmechanic.com/article/headlight-use-laws-for-all-50-statesGoogle Scholar
- Wenzheng Chen, Fangyin Wei, Kiriakos N Kutulakos, Szymon Rusinkiewicz, and Felix Heide. 2020. Learned feature embeddings for non-line-of-sight imaging and recognition. ACM Transactions on Graphics (ToG) 39, 6 (2020), 1--18.Google ScholarDigital Library
- Zhihao Chen, Liang Wan, Lei Zhu, Jia Shen, Huazhu Fu, Wennan Liu, and Jing Qin. 2021. Triple-cooperative video shadow detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2715--2724.Google ScholarCross Ref
- CSGNetwork. 2023. Brake Distance. (2023). http://www.csgnetwork.com/stopdistcalc.htmlGoogle Scholar
- Graham D Finlayson, Mark S Drew, and Cheng Lu. 2009. Entropy minimization for shadow removal. International Journal of Computer Vision 85, 1 (2009), 35--57.Google ScholarDigital Library
- Graham D Finlayson, Steven D Hordley, Cheng Lu, and Mark S Drew. 2005. On the removal of shadows from images. IEEE transactions on pattern analysis and machine intelligence 28, 1 (2005), 59--68.Google Scholar
- Martin A. Fischler and Robert C. Bolles. 1981. Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography. Commun. ACM 24, 6 (jun 1981), 381--395. Google ScholarDigital Library
- Maciej Gryka, Michael Terry, and Gabriel J Brostow. 2015. Learning to remove soft shadows. ACM Transactions on Graphics (TOG) 34, 5 (2015), 1--15.Google ScholarDigital Library
- Xiaowei Hu, Lei Zhu, Chi-Wing Fu, Jing Qin, and Pheng-Ann Heng. 2018. Direction-aware spatial context features for shadow detection. In Proceedings of the IEEE conference on computer vision and pattern recognition. 7454--7462.Google ScholarCross Ref
- Eddy Ilg, Nikolaus Mayer, Tonmoy Saikia, Margret Keuper, Alexey Dosovitskiy, and Thomas Brox. 2017. Flownet 2.0: Evolution of optical flow estimation with deep networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2462--2470.Google ScholarCross Ref
- Julian Iseringhausen and Matthias B Hullin. 2020. Non-line-of-sight reconstruction using efficient transient rendering. ACM Transactions on Graphics (ToG) 39, 1 (2020), 1--14.Google ScholarDigital Library
- Achuta Kadambi, Hang Zhao, Boxin Shi, and Ramesh Raskar. 2016. Occluded imaging with time-of-flight sensors. ACM Transactions on Graphics (ToG) 35, 2 (2016), 1--12.Google ScholarDigital Library
- Salman H Khan, Mohammed Bennamoun, Ferdous Sohel, and Roberto Togneri. 2015. Automatic shadow detection and removal from a single image. IEEE transactions on pattern analysis and machine intelligence 38, 3 (2015), 431--446.Google Scholar
- Martin Laurenzis, Andreas Velten, and Jonathan Klein. 2016. Dual-mode optical sensing: three-dimensional imaging and seeing around a corner. Optical Engineering 56, 3 (2016), 031202.Google Scholar
- Xin Lei, Liangyu He, Yixuan Tan, Ken Xingze Wang, Xinggang Wang, Yihan Du, Shanhui Fan, and Zongfu Yu. 2019. Direct object recognition without line-of-sight using optical coherence. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 11737--11746.Google Scholar
- Guan-Ting Lin, Vinay Malligere Shivanna, and Jiun-In Guo. 2020. A Deep-learning model with task-specific bounding box regressors and conditional back-propagation for moving object detection in ADAS applications. Sensors 20, 18 (2020), 5269.Google Scholar
- Zhenguang Liu, Shuang Wu, Shuyuan Jin, Qi Liu, Shijian Lu, Roger Zimmermann, and Li Cheng. 2019. Towards Natural and Accurate Future Motion Prediction of Humans and Animals. In CVPR. 10004--10012. Google ScholarCross Ref
- Taichi Nakashima and Yoshito Yabuta. 2018. Object Detection by using Interframe Difference Algorithm. In 2018 12th France-Japan and 10th Europe-Asia Congress on Mechatronics. 98--102. Google ScholarCross Ref
- Felix Naser, Igor Gilitschenski, Guy Rosman, Alexander Amini, Fredo Durand, Antonio Torralba, Gregory W Wornell, William T Freeman, Sertac Karaman, and Daniela Rus. 2018. Shadowcam: Real-time detection of moving obstacles behind a corner for autonomous vehicles. In 2018 21st International Conference on Intelligent Transportation Systems (ITSC). IEEE, 560--567.Google ScholarDigital Library
- The National Safety Council of America. 2018. The Most Dangerous Time to Drive. (2018). https://www.nsc.org/road/safety-topics/driving-at-night?Google Scholar
- Rohit Pandharkar, Andreas Velten, Andrew Bardagjy, Everett Lawson, Moungi Bawendi, and Ramesh Raskar. 2011. Estimating motion and size of moving non-line-of-sight objects in cluttered environments. In CVPR 2011. IEEE, 265--272.Google Scholar
- Prashant W Patil and Subrahmanyam Murala. 2018. MSFgNet: A novel compact end-to-end deep network for moving object detection. IEEE Transactions on Intelligent Transportation Systems 20, 11 (2018), 4066--4077.Google ScholarCross Ref
- Han Qiu, Yuchen Ma, Zeming Li, Songtao Liu, and Jian Sun. 2020. Borderdet: Border feature for dense object detection. In European Conference on Computer Vision. Springer, 549--564.Google ScholarDigital Library
- Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2015. Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in neural information processing systems 28 (2015).Google Scholar
- Chang-Gyun Roh, Jisoo Kim, and I-Jeong Im. 2020. Analysis of impact of rain conditions on ADAS. Sensors 20, 23 (2020), 6720.Google ScholarCross Ref
- Ethan Rublee, Vincent Rabaud, Kurt Konolige, and Gary Bradski. 2011. ORB: An efficient alternative to SIFT or SURF. In 2011 International Conference on Computer Vision. 2564--2571. Google ScholarDigital Library
- Sheila W Seidel, John Murray-Bruce, Yanting Ma, Christopher Yu, William T Freeman, and Vivek K Goyal. 2020. Two-dimensional non-line-of-sight scene estimation from a single edge occluder. IEEE Transactions on Computational Imaging 7 (2020), 58--72.Google Scholar
- Dongeek Shin, Ahmed Kirmani, Vivek K Goyal, and Jeffrey H Shapiro. 2015. Photon-efficient computational 3-D and reflectivity imaging with single-photon detectors. IEEE Transactions on Computational Imaging 1, 2 (2015), 112--125.Google ScholarCross Ref
- Le-Anh Tran, Truong-Dong Do, Dong-Chul Park, and My-Ha Le. 2021. Enhancement of Robustness in Object Detection Module for Advanced Driver Assistance Systems. In 2021 International Conference on System Science and Engineering (ICSSE). IEEE, 158--163.Google Scholar
- Arizona State University. 2023. Pedestrian Injuries and Fatalities. (2023). https://popcenter.asu.edu/content/pedestrian-injuries-fatalities-0Google Scholar
- Cadillac vehicles. 2016. super-cruise: the first hands free driver-assistance technology for compatible roads. (2016). https://www.gmc.com/connectivity-technology/super-cruiseGoogle Scholar
- Tianyu Wang, Xiaowei Hu, Chi-Wing Fu, and Pheng-Ann Heng. 2021. Singlestage instance shadow detection with bidirectional relation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1--11.Google ScholarCross Ref
- Tianyu Wang, Xiaowei Hu, Qiong Wang, Pheng-Ann Heng, and Chi-Wing Fu. 2020. Instance shadow detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1880--1889.Google ScholarCross Ref
- Tai Wang, Xinge Zhu, Jiangmiao Pang, and Dahua Lin. 2021. FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops. 913--922.Google Scholar
- World Health Organization (WHO). 2016. Road safety. (2016). https://www.who.int/data/gho/data/themes/road-safetyGoogle Scholar
- A. Woo, P. Poulin, and A. Fournier. 1990. A survey of shadow algorithms. IEEE Computer Graphics and Applications 10, 6 (1990), 13--32. Google ScholarDigital Library
- Timothy Woodford, Xinyu Zhang, Eugene Chai, and Karthikeyan Sundaresan. 2022. Mosaic: leveraging diverse reflector geometries for omnidirectional aroundcorner automotive radar. In Proceedings of the 20th Annual International Conference on Mobile Systems, Applications and Services. 155--167.Google Scholar
- Jing Wu, Xin Du, Yun-fang Zhu, and Wei-kang Gu. 2008. Adaptive fuzzy filter algorithm for real-time video denoising. In 2008 9th International Conference on Signal Processing. 1287--1291. Google ScholarCross Ref
- Xian-Tao Wu, Wen Wu, Lin-Lin Zhang, and Yi Wan. 2022. Don't worry about noisy labels in soft shadow detection. The Visual Computer (2022), 1--12.Google Scholar
- Marie Yahiaoui, Hazem Rashed, Letizia Mariotti, Ganesh Sistu, Ian Clancy, Lucie Yahiaoui, Varun Ravi Kumar, and Senthil Yogamani. 2019. Fisheyemodnet: Moving object detection on surround-view cameras for autonomous driving. arXiv preprint arXiv:1908.11789 (2019).Google Scholar
- Michael Ying Yang, Wentong Liao, Xinbo Li, and Bodo Rosenhahn. 2018. Deep learning for vehicle detection in aerial images. In 2018 25th IEEE International Conference on Image Processing (ICIP). IEEE, 3079--3083.Google Scholar
- Rongjie Yu, Yin Zheng, and Xiaobo Qu. 2021. Dynamic driving environment complexity quantification method and its verification. Transportation Research Part C: Emerging Technologies 127 (2021), 103051.Google ScholarCross Ref
- Quanlong Zheng, Xiaotian Qiao, Ying Cao, and Rynson WH Lau. 2019. Distraction-aware shadow detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5167--5176.Google ScholarCross Ref
- Haibo Zhou, Wenchao Xu, Jiacheng Chen, and Wei Wang. 2020. Evolutionary V2X Technologies Toward the Internet of Vehicles: Challenges and Opportunities. Proc. IEEE 108, 2 (2020), 308--323. Google ScholarCross Ref
- Barbara Zitova and Jan Flusser. 2003. Image registration methods: a survey. Image and vision computing 21, 11 (2003), 977--1000.Google Scholar
Index Terms
- Ghost-Probe: NLOS Pedestrian Rushing Detection with Monocular Camera for Automated Driving
Recommendations
A Monitoring System to Understand Postal Motorcyclist's Driving Behavior
ITSC '15: Proceedings of the 2015 IEEE 18th International Conference on Intelligent Transportation SystemsIn Korea, motorcycle accidents constitute about 5% of total traffic accidents, however, the fatality rate approximates 12%, which is disproportionately higher per mile driven compared to other vehicular fatalities. Most of these fatalities are related ...
Transition to Automated: The Interaction of Activating the In-vehicle Automated Driving System
Design, User Experience, and Usability. Application DomainsAbstractAutomated driving system (ADS) becomes more popularized in recent years, and the level 2 and level 3 ADS have already been equipped on several production car models in the market. The activating interaction of the ADS is very important since it is ...
Gaze guidance reduces the number of vehicle-pedestrian collisions in a driving simulator
APGV '11: Proceedings of the ACM SIGGRAPH Symposium on Applied Perception in Graphics and VisualizationDriving and visual perception are tightly linked. Every moment, a multitude of visual stimuli compete for the driver's limited attentional resources. Despite modern safety measures, traffic accidents still remain a major source of fatalities. A large ...
Comments