skip to main content
10.1145/3615984.3616504acmconferencesArticle/Chapter ViewAbstractPublication PagesmobicomConference Proceedingsconference-collections
research-article

Sensing-assisted Communication Beamforming Based on Multi-Modal Feature Extraction for High-Reliable IoV

Published: 02 October 2023 Publication History

Abstract

This paper introduces a sensing-assisted communication method, which relies on the extraction of multi-modal features. Multi-modal data, e.g. vision, radar, lidar, and position are employed as the input data of the proposed beamforming method. The recognition and beamforming accuracy are therefore improved. Initially, the 3D-Conv model is utilized to extract features from the encoded multimodal data. Subsequently, the generative pre-trained transformer (GPT) is employed to grasp correlations across diverse models and fuse their latent features. These fusion features are used to facilitate beam prediction, thereby approximating the optimal beam index for real-world data. Experimental results based on real-world data validate the effectiveness of our approach, achieving an accuracy of 85%, surpassing traditional single-modal schemes by over 25%.

References

[1]
X. Li, Q. Li, F. Meng, Z. Xu, Z. Xu and Y. Gong, "Deep Learning-Based Channel Estimation for HPO-MIMO Systems in IoV Scenario," 2022 15th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI), Beijing, China, 2022, pp. 1--5.
[2]
Z. Du, F. Liu, Y. Li, W. Yuan, Y. Cui, Z. Zhang, C. Masouros, B. Ai, "Towards ISAC-Empowered Vehicular Networks: Framework, Advances, and Opportunities," (2023). https://doi.org/10.48550/arXiv.2305.00681.
[3]
Y. Sun and C. Qi, "Analog Beamforming and Combining Based on Codebook in Millimeter Wave Massive MIMO Communications," GLOBECOM 2017 - 2017 IEEE Global Communications Conference, Singapore, 2017, pp. 1--6.
[4]
Z. Ying, H. Yang, J. Gao and K. Zheng, "A New Vision-Aided Beam Prediction Scheme for mmWave Wireless Communications," 2020 IEEE 6th International Conference on Computer and Communications (ICCC), Chengdu, China, 2020, pp. 232--237.
[5]
M. Alrabeiah, A. Hredzak and A. Alkhateeb, "Millimeter Wave Base Stations with Cameras: Vision-Aided Beam and Blockage Prediction," 2020 IEEE 91st Vehicular Technology Conference (VTC2020-Spring), Antwerp, Belgium, 2020, pp. 1--5.
[6]
F. Liu, W. Yuan, C. Masouros and J. Yuan, "Radar-Assisted Predictive Beamforming for Vehicular Links: Communication Served by Sensing," IEEE Transactions on Wireless Communications (TWC), vol. 19, no. 11, pp. 7704--7719, Nov. 2020.
[7]
G. Charan, A. Hredzak, C. Stoddard, B. Berrey, M. Seth, H. Nunez, and A. Alkhateeb, "Towards real-world 6G drone communication: Position and camera aided beam prediction," GLOBECOM 2022. [Online].
[8]
G. Charan, U. Demirhan, J. Morais, A. Behboodi, H. Pezeshki, and A. Alkhateeb, "Multi-modal beam prediction challenge 2022: Towards generalization," arXiv preprint arXiv:2209.07519, 2022.
[9]
A. Prakash, K. Chitta, and A. Geiger, "Multi-modal fusion transformer for end-to-end autonomous driving," in Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
[10]
A. Alkhateeb, G. Charan, T. Osman, A. Hredzak, and N. Srinivas, "DeepSense 6G: Large-scale real-world multi-modal sensing and communication datasets," to be available on arXiv, 2022. [Online]. Available: https://www.DeepSense6G.net.
[11]
M. Mercuri, I.R. Lorato, YH. Liu, et al "Vital-sign monitoring and spatial tracking of multiple people using a contactless radar-based sensor". Nat Electron 2, 252--262 (2019). https://doi.org/10.1038/s41928-019-0258-6.

Cited By

View all
  • (2023)Integrated Sensing and Communication via Orthogonal Time Frequency Space Signaling with Hybrid Message Passing Detection and Fractional Parameter EstimationSensors10.3390/s2324987423:24(9874)Online publication date: 16-Dec-2023

Index Terms

  1. Sensing-assisted Communication Beamforming Based on Multi-Modal Feature Extraction for High-Reliable IoV

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      ISACom '23: Proceedings of the 3rd ACM MobiCom Workshop on Integrated Sensing and Communications Systems
      October 2023
      46 pages
      ISBN:9798400703645
      DOI:10.1145/3615984
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 02 October 2023

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. Beamforming
      2. Deep Learning
      3. IOV
      4. Multi-Modal

      Qualifiers

      • Research-article
      • Research
      • Refereed limited

      Conference

      ACM MobiCom '23
      Sponsor:

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)168
      • Downloads (Last 6 weeks)16
      Reflects downloads up to 15 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2023)Integrated Sensing and Communication via Orthogonal Time Frequency Space Signaling with Hybrid Message Passing Detection and Fractional Parameter EstimationSensors10.3390/s2324987423:24(9874)Online publication date: 16-Dec-2023

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media