Abstract
The advanced driver assistance system (ADAS) has been actively researched to enable adaptive cruise control and collision avoidance, however, conventional ADAS is not capable of more advanced functions due to the absence of intelligent decision making algorithms such as behavior analysis. Moreover, most algorithms in automotive applications are accelerated by GPUs where its power consumption exceeds the power requirement for practical usage. In this paper, we present a deep risk prediction algorithm, which predicts risky objects prior to collision by behavior prediction. Also, a real-time embedded system with high energy efficiency is proposed to provide practical application of our algorithm to the intelligent ADAS, consuming only ~1 W in average. For validation, we build the risky urban scene stereo (RUSS) database including 50 stereo video sequences captured under various risky road situations. The system is tested with various databases including the RUSS, and it can maximally achieve 30 frames/s throughput with 720p stereo images with 98.1% of risk prediction accuracy.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Mao, X., et al.: Hierarchical CNN for traffic sign recognition. In: Proceedings of IEEE Intelligent Vehicles Symposium (IV), pp. 130–135, June 2016
Weber, M., Wolf, P., Zollner, J.M.: DeepTLR: a single deep convolutional network for detection and classification of traffic lights. In: Proceedings of IEEE Intelligent Vehicles Symposium (IV), pp. 342–348, June 2016
Yu, L., et al.: A monocular vision based pedestrian detection system for intelligent vehicles. In: Proceedings of IEEE Intelligent Vehicles Symposium (IV), pp. 524–259, June 2008
Fan, Q., Brown, L., Smith, J.: A closer look at faster R-CNN for vehicle detection. In: Proceedings of IEEE Intelligent Vehicles Symposium (IV), pp. 124–129, June 2016
Lange, S., et al.: Online vehicle detection using deep neural networks and lidar based preselected image patches. In: Proceedings of IEEE Intelligent Vehicles Symposium (IV), pp. 954–959, June 2016
Liebner, M., Klanner, F., Baumann, M., Ruhhammer, C., Stiller, C.: Velocity-based driver intent inference at urban intersections in the presence of preceding vehicles. IEEE Intell. Transp. Syst. Mag. 5(2), 10–21 (2013)
Kasper, D., et al.: Object-oriented Bayesian networks for detection of lane change maneuvers. IEEE Intell. Transp. Syst. Mag. 4(3), 19–31 (2012)
Barth, A., Franke, U.: Tracking oncoming and turning vehicles at intersections. In Proceedings of IEEE International Conference on Intelligent Transportation Systems, pp. 861–868, September 2010
Hermes, C., Wohler, C., Schenk, K., Kummert, F.: Long-term vehicle motion prediction. In: Proceedings of IEEE Intelligent Vehicles Symposium (IV), pp. 652–657, July 2009
Forster, F.: Heterogeneous processors for advanced driver assistance systems. Atz Elektronik Worldwide 9(1), 14–18 (2014)
Stein, F.: The challenge of putting vision algorithms into a car. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 89–94, June 2012
Gehrig, S.K., Eberli, F., Meyer, T.: A Real-Time Low-Power Stereo Vision Engine Using Semi-Global Matching. In: Fritz, M., Schiele, B., Piater, J.H. (eds.) ICVS 2009. LNCS, vol. 5815, pp. 134–143. Springer, Heidelberg (2009). doi:10.1007/978-3-642-04667-4_14
Ernst, I., Hirschmüller, H.: Mutual Information Based Semi-Global Stereo Matching on the GPU. In: Bebis, G., Boyle, R., Parvin, B., Koracin, D., Remagnino, P., Porikli, F., Peters, J., Klosowski, J., Arns, L., Chun, Y.K., Rhyne, T.-M., Monroe, L. (eds.) ISVC 2008. LNCS, vol. 5358, pp. 228–239. Springer, Heidelberg (2008). doi:10.1007/978-3-540-89639-5_22
Gehrig, S.K., Rabe, C.: Real-time semiglobal matching on the CPU. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 85–92, June 2010
Michael, M., et al.: Real-time stereo vision: optimizing semiglobal matching. In: Proceedings of IEEE Intelligent Vehicles Symposium (IV), pp. 1197–1202, June 2013
Tanabe, J., et al.: A 1.9TOPS and 564GOPS/W heterogeneous multi-core SoC with color-based object classification accelerator for image-recognition applications. In: IEEE International Solid-State Circuits Conference (ISSCC) Dig. Tech. Papers, pp. 328–329, February 2015
Tanabe, Y., et al.: A 464GOPS 620GOPS/W heterogeneous multicore SoC for image-recognition applications. In: IEEE ISSCC Digest, pp. 222–223, February 2012
Park, J., et al.: A 646GOPS/W multi-classifier many-core processor with cortex-like architecture for super-resolution recognition. In: IEEE ISSCC Digest, pp. 168–169, February 2013
Juang, C.-F., Lin, C.-T.: An online self-constructing neural fuzzy inference network and its applications. IEEE Trans. Fuzzy Syst. 6(1), 12–32 (1998)
Giles, C., et al.: Noisy time series prediction using recurrent neural network and grammatical inference. J. Mach. Learn. 44(1), 161–183 (2001)
Williams, R.J., Peng, J.: An efficient gradient-based algorithm for on line training of recurrent network trajectories. Neural Comput. 2, 490–501 (1990)
Lee, K.J., et al.: A 502GOPS and 0.984mW dual-mode ADAS SoC with RNN-FIS engine for intention prediction in automotive black-box system. In: IEEE ISSCC Digest, pp. 256–257, February 2016
Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? The KITTI vision benchmark suit. In: Proceedings of IEEE CVPR, pp. 3354–3361, June 2012
Keller, C., Enzweiler, M., Gavrila, D.M.: A new benchmark for stereo-based pedestrian detection. In: Proceedings of IEEE Intelligent Vehicles Symposium (IV), pp. 691–696, June 2011
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Lee, K., Choe, G., Bong, K., Kim, C., Kweon, I.S., Yoo, HJ. (2017). A Real-Time and Energy-Efficient Embedded System for Intelligent ADAS with RNN-Based Deep Risk Prediction using Stereo Camera. In: Liu, M., Chen, H., Vincze, M. (eds) Computer Vision Systems. ICVS 2017. Lecture Notes in Computer Science(), vol 10528. Springer, Cham. https://doi.org/10.1007/978-3-319-68345-4_31
Download citation
DOI: https://doi.org/10.1007/978-3-319-68345-4_31
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-68344-7
Online ISBN: 978-3-319-68345-4
eBook Packages: Computer ScienceComputer Science (R0)