Abstract
Besides the use of information transmission, vehicular communications also perform an essential role in intelligent transportation systems (ITS) for exchanging critical driving information among end users, vehicles, and infrastructures. Moreover, to enhance the understanding of the local environment, increasingly more data are collected by sensors, inducing an extensive use of deep learning (DL)-based algorithms in ITS. To further promote the development of DL-based algorithms in ITS, in this paper, we present a concise introduction of DL technologies. Then, we conduct an in-depth investigation on two popular DL-based applications used in ITS, traffic flow forecasting and trajectory prediction, focusing on when and how the authors employ different DL models and training schemes in these tasks. Finally, we raise two existing problems while employing DL-based algorithms in practical ITS and further discuss certain recent advances in DL-based research to tackle these challenges. To encourage more researchers to focus on the development of DL-based algorithms in ITS for a better world, we hope this paper can be treated as an informational material for prospective researchers, which contains the essential background knowledge of DL-based ITS applications; we also hope this paper will encourage experienced researchers to counter the open challenges and achieve a technical breakthrough to ITS.
Similar content being viewed by others
References
Choi, J., Va, V., Gonzalez-Prelcic, N., Daniels, R., Bhat, C. R., & Heath, R. W. (Dec. 2016). Millimeter-wave vehicular communication to support massive automotive sensing. IEEE Communications Magazine, 54(12), 160–167.
Lu, Z., Qu, G., & Liu, Z. (Feb. 2019). A survey on recent advances in vehicular network security, trust, and privacy. IEEE Trans. on Intell. Transp. Syst., 20(2), 760–776.
G. Karagiannis et al., (2011) “+” IEEE Commun. Surveys Tut., vol. 13, no. 4, pp. 584–616, 4th Quart.
Alam, M. et al., (2016). Introduction to intelligent transportation systems, Intell. Transp. Syst., Springer, pp. 1–17
Zhou, Z. H., Chawla, N. V., Jin, Y., & Williams, G. J. (Nov. 2014). Big data opportunities and challenges: Discussions from data analytics perspectives. IEEE Computational Intelligence Magazine, 9(4), 62–74.
Simeone, O. (Dec. 2018). A very brief introduction to machine learning with applications to communication systems. IEEE Trans. Cogn. Commun. Netw., 4(4), 648–664.
Ye, H., Liang, L., Ye Li, G., Kim, J. B., Lu, L., & Wu, M. (Jun. 2018). Machine learning for vehicular networks: Recent advances and application examples. IEEE Vehicular Technology Magazine, 13(2), 94–101.
Chen, Y. C., Huang, S. F., Lee, H. Y., Wang, Y. H., & Shen, C. H. (Sep. 2019). Audio word2vec: Sequence-to-sequence autoencoding for unsupervised learning of audio segmentation and sepresentation. IEEE/ACM Trans. Audio, Speech, Language Process., 27(9), 1481–1493.
Sun, K. et al., (2019) Deep high-resolution representation learning for human pose estimation, in IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), pp. 5693–5703.
Tu, T. et al., (2019) End-to-end text-to-speech for low-resource languages by cross-lingual transfer learning. [Online]. Available: https://arxiv.org/abs/1904.06508
Lee, H., et al. (Mar. 2017). Personalizing recurrent-neural-network-based language model by social network. IEEE/ACM Trans. Audio, Speech, Language Process., 25(3), 519–530.
Veres, M. et al., (2019) Deep learning for intelligent transportation systems: A survey of emerging trends, IEEE Trans. on Intell. Transp. Syst., Early Access.
McCulloch, W. S., & Pitts, W. (1943). A logical calculus of ideas immanent in nervous activity. The Bulletin of Mathematical Biophysics, 5, 115–133.
David, D. E., et al. (1986). Learning representations by back-propagating errors. Nature, 323, 533–536.
Hinton, G. E., Osindero, S., & Teh, Y. W. (2006). A fast learning algorithm for deep belief nets neural computation. Neural Computation, 18(8), 1527–1554.
Mikolov, T. et al., (2013) Distributed representations of words and phrases and their compositionality, in Conf. on Neural Inf. Process. Syst. (NIPS), Lake Tahoe, NV, pp. 3111–3119.
Krizhevsky, A. et al., (2012) ImageNet classification with deep convolutional neural networks, in Conf. on Neural Inf. Process. Syst. (NIPS), Lake Tahoe, NV, pp. 1097–1105.
Lin, C. H. et al., (2019) BsNet: A deep learning-based beam selection method for mmWave communications, in IEEE 90th Veh. Technol. Conf.
Lin, C. H. et al., (2019) DL-CFAR: A novel CFAR target detection method based on deep learning, in IEEE 90th Veh. Technol. Conf.
Farsad, N. and Goldsmith, A. (2018) Neural network detection of data sequences in communication systems, [Online]. Available: https://arxiv.org/abs/1802.02046
Liu, Z., Zhang, L., & Ding, Z. (Jun. 2019). Exploiting bi-directional channel reciprocity in deep learning for low rate massive MIMO CSI feedback. IEEE Wireless Communications Letters, 8(3), 889–892.
Csáji, B. C. (2001) Approximation with artificial neural networks, M.S. thesis, Faculty Sci., Eötvös Loránd Univ., Budapest, Hungary.
Cao C., Li D. and Fair I. (2018) Deep Learning-Based Decoding for Constrained Sequence Codes, in IEEE Globecom Workshops, Abu Dhabi, pp. 1–7.
Y. Lecun, L. Bottou, Y. Bengio, P. Haffner, “Gradient-based learning applied to document recognition,” in Proc. IEEE, Nov. 1998, vol. 86, no. 11, pp. 2278–2324.
Dai, X. et al., (2017) Deeptrend: A deep hierarchical neural network for traffic flow prediction, [Online]. Available: https://arxiv.org/abs/1707.03213v1
Wu, Z. et al., (2019) A comprehensive survey on graph neural networks, [Online]. Available: https://arxiv.org/abs/1901.00596
Huang, H., Xia, W., Xiong, J., Yang, J., Zheng, G., & Zhu, X. (2019). Unsupervised learning-based fast beamforming design for downlink MIMO. IEEE Access, 7, 7599–7605.
Goodfellow, I. et al., (2014) Generative adversarial nets, in Conf. on Neural Inf. Process. Syst. (NIPS), pp. 2672–2680.
Karras, T. et al., (2019) A style-based generator architecture for generative adversarial networks, in Conf. On computer vision and pattern recognition.
Mirza, M. and Osindero, S., (2014) Conditional generative adversarial nets, [Online]. Available: https://arxiv.org/abs/1411.1784.
Luong, N. C. et al., (2018) Applications of deep reinforcement learning in communications and networking: a survey, [Online]. Available: https://arxiv.org/abs/1810.07862
Li, Y. et al., (2017) Diffusion convolutional recurrent neural network: data-driven traffic forecasting, in Int. Learn. Representations (ICLR).
Lv, Y., et al. (Apr. 2015). Traffic flow prediction with big data: A deep learning approach. IEEE Transactions on Intelligent Transportation Systems, 16(2), 865–873.
Bengio, Y. et al., (2007) Greedy layer wise training of deep networks, in Conf. on Neural Inf. Process. Syst. (NIPS), pp. 153–160.
Ma, X., Dai, Z., He, Z., Ma, J., Wang, Y., & Wang, Y. (2017). Learning traffic as images: A deep convolutional neural network for large-scale transportation network speed prediction. Sensors J., 17(4), 818–833.
Di, Z., et al. (Apr. 2017). Traffic parameters prediction using a three-channel convolutional neural network. IEEE Sensors Journal, 17(4), 363–371.
Fouladgar, M. et al., (2017) Scalable deep traffic flow neural networks for urban traffic congestion prediction, in IEEE Int. Joint Conf. on Neural Netw., pp. 2251–2258.
Guo, S., Lin, Y., Li, S., Chen, Z., & Wan, H. (Oct. 2019). Deep spatial–temporal 3D convolutional neural networks for traffic data forecasting. IEEE Transactions on Intelligent Transportation Systems, 20(10), 3913–3926.
Nair, V. and Hinton, G.E., (2010) Rectified linear units improve restricted Boltzmann machines, in Int. Conf. Machine Learn., pp. 807–814.
He K. et al., (2016) Deep residual learning for image recognition, in IEEE Conf. Comput. Vi. and Pattern Recognition, Las Vegas, NV, pp. 770–778.
Yasdi, R. (May 1999). Prediction of road traffic using a neural network approach. Neural Comput. Appl., 8(2), 135–142.
Ma, X., et al. (2015). Large-scale transportation network congestion evolution prediction using deep learning theory. PLoS One, 10(3), 1–17.
Ishak, S., Kotha, P., & Alecsandru, C. (2003). Optimization of dynamic neural network performance for short-term traffic prediction. Transp. Res. Rec., J. Transp. Res. Board, 1836(1), 45–56.
Jia, Y., Wu, J., Ben-Akiva, M., Seshadri, R., & du, Y. (Nov., 2017). Rainfall-integrated traffic speed prediction using deep learning method. IET Trans. Intell. Transp. Syst., 11(9), 531–536.
Zhang, K., Zuo, W., Chen, Y., Meng, D., & Zhang, L. (Jul. 2017). Beyond a Gaussian denoiser: Residual learning of deep CNN for image denoising. IEEE Trans. on Image Process., 26(7), 3142–3155.
Zhao, Z. and Zhang, Y., (2018) A traffic flow prediction approach: LSTM with detrending, in IEEE Int. Conf. on Progress in Informat. and Comput., Suzhou, pp. 101–105.
Zhao, Z., Chen, W., Wu, X., Chen, P. C. Y., & Liu, J. (2017). LSTM network: A deep learning approach for short-term traffic forecast. IET Intelligent Transport Systems, 11(2), 68–75.
Zhiyong, C. et al., (2018) Deep bidirectional and unidirectional LSTM recurrent neural network for network-wide traffic speed prediction, [Online]. Available: https://arxiv.org/abs/1801.02143
Wangyang, W. et al., (2019) An autoencoder and LSTM-based traffic flow prediction method, Sensors J., vol.19.
Tian, Y. and Pan, L., (2015) Predicting short-term traffic flow by long short-term memory recurrent neural network, in IEEE Int. Conf. on Smart City, Chengdu, pp. 153–158.
Du, S., Li, T., Gong, X., Yu, Z., and Horng, S.-J., (2018) A hybrid method for traffic flow forecasting using multimodal deep learning. [Online]. Available: https://arxiv.org/abs/1803.02099
Liang, Y., Cui, Z., Tian, Y., Chen, H., & Wang, Y. (2018). A deep generative adversarial architecture for network-wide spatial-temporal traffic-state estimation. Transp. Res. Rec., J. Transp. Res. Board, 2672(45), 87–105.
Lin, Y. et al., (2018) Pattern sensitive prediction of traffic flow based on generative adversarial framework, IEEE Trans. Intell. Transp. Syst., vol. PP, no. 99, pp. 1–6.
Saxena, D. and Jiannong, C., (2019) D-GAN : deep generative adversarial nets for spatio-temporal prediction. [Online]. Available: https://arxiv.org/abs/1907.08556
Zang, D., Fang, Y., Wei, Z., Tang, K., & Cheng, J. (2019). Traffic flow data prediction using residual deconvolution based deep generative network. IEEE Access, 7, 71311–71322.
Cui, Z. et al., (2018) Traffic graph convolutional recurrent neural network: a deep learning framework for network-scale traffic learning and forecasting. [Online]. Available: https://arxiv.org/abs/1802.07007
Wu, T. et al., (2018) Graph attention LSTM network: a new model for traffic flow forecasting, in 5th Int. Conf. on Inf. Sci. and Control Eng., Zhengzhou, pp. 241–245.
Lefèvre, S., Vasquez, D., & Laugier, C. (Jul. 2014). A survey on motion prediction and risk assessment for intelligent vehicles. Robomech J., 1(1), 1–14.
A. Ng and S. Russell, (2000) Algorithms for inverse reinforcement learning, in Int. Conf. Mach. Learn., pp. 663–670.
K. Kitani et al., (2012) Activity forecasting, in Eur. Conf. on Comput. Vision (ECCV), Florence, pp. 201–214.
Alahi, A. et al., (2016) Social LSTM: human trajectory prediction in crowded spaces, in IEEE Conf. on Comput. Vision and pattern recognition (CVPR), Las Vegas, NV, pp. 961–971.
Kim, B. et al., (2017) Probabilistic vehicle trajectory prediction over occupancy grid map via recurrent neural network, in IEEE 20th Int. Conf. on Intell. Transp. Syst. (ITSC), Yokohama, pp. 399–404.
Khosroshahi, A. et al., (2016) Surround vehicles trajectory analysis with recurrent neural networks, in IEEE 19th Int. Conf. on Intell. Transp. Syst. (ITSC), Rio de Janeiro, pp. 2267–2272.
Ondruska, P. and Posner, I., (2016) Deep tracking: seeing beyond seeing using recurrent neural network, in Thirtieth AAAI Conf. on Artificial Intell. (AAAI), Phoenix, vol. 12, pp. 3361–3367.
Park, H. et al., (2018) Sequence-to-sequence prediction of vehicle trajectory via LSTM encoder-decoder architecture, in IEEE Intell. Veh. Symp. (IV).
Deo, N. and Trivedi, M., (2018) Multi-modal trajectory prediction of surrounding vehicles with maneuver based lstms, [Online]. Available: https://arxiv.org/abs/1805.05499
Ma, Y. et al., (2019) Trafficpredict: trajectory prediction for heterogeneous traffic-agents, in Thirtieth AAAI Conf. on Artificial Intell. (AAAI), Honolulu.
Djuric, N. et al., (2018) Short-term motion prediction of traffic actors for autonomous driving using deep convolutional networks, [Online]. Available: https://arxiv.org/abs/1808.05819
Fragkiadaki, K. et al., (2016) Learning visual predictive models of physics for playing billiards, in Int. Conf. on Learn. Representations (ICLR), San Juan, pp. 1–12.
Watters, N. et al., (2017) Visual interaction networks: Learning a physics simulator from video, in Conf. on Neural Inf. Process. Syst. (NIPS), Long beach, CA, pp. 4539–4547.
Battaglia, P. W. et al., (2016) Interaction networks for learning about objects relations and physics, in Conf. on Neural Inf. Process. Syst. (NIPS), Barcelona.
Kuefler, A. et al., (2017) Imitating driver behavior with generative adversarial networks, in IEEE Intell. Veh. Symp. (IV), Redondo, CA.
Gupta, A. et al., (2018) Social gan: socially acceptable trajectories with generative adversarial networks, in IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Salt lake city, UT.
Cheng, Y. et al., (2017) A survey of model compression and acceleration for deep neural networks, [Online]. Available: https://arxiv.org/abs/1710.09282v8
Long, X. et al., (2019) A survey of related research on compression and acceleration of deep neural networks, J. Physics, vol. 1213.
Li, H. et al., (2016) Pruning filters for efficient convnets, [Online]. Available: https://arxiv.org/abs/1608.08710
Luo, J. and Wu, J., (2017) An entropy-based pruning method for CNN compression. [Online]. Available: https://arxiv.org/abs/1706.05791
Huang, H., Xia, W., Xiong, J., Yang, J., Zheng, G., & Zhu, X. (Jan. 2019). Unsupervised learning based beamforming design for downlink MIMO. IEEE Access, 7, 7599–7605.
Pan, S., & Yang, Q. (Oct. 2010). A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering, 22(10), 1345–1359.
Yosinski, J. et al., (2014) How transferable are features in deep neural networks, in Conf. on Neural Inf. Process. Syst. (NIPS), Montreal, pp 3320–3328.
Huang, J. et al., (2013) Cross-language knowledge transfer using multilingual deep neural network with shared hidden layers, in IEEE Int. Conf. on Acoustics, Speech and Sig. Process. (ICASSP), Vancouver, BC, pp. 7304–7308.
Acknowledgements
This work was partially supported by the “Center for mmWave Smart Radar Systems and Technologies” and the “Center for Open Intelligent Connectivity” under the Featured Areas Research Center Program within the framework of the Higher Education Sprout Project by the Ministry of Education (MOE) of Taiwan, and partially supported by the Ministry of Science and Technology (MOST) of Taiwan under grant MOST 109-2634-F-009-030, MOST 109-2218-E-009-002.
Author information
Authors and Affiliations
Corresponding authors
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Lin, CH., Lin, YC., Wu, YJ. et al. A Survey on Deep Learning-Based Vehicular Communication Applications. J Sign Process Syst 93, 369–388 (2021). https://doi.org/10.1007/s11265-020-01587-2
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11265-020-01587-2