skip to main content
10.1145/3633637.3633724acmotherconferencesArticle/Chapter ViewAbstractPublication PagesiccprConference Proceedingsconference-collections
research-article

3D SSVEP Visual stimulus in Augmented Reality for Robotic Arm Grasping

Published: 28 February 2024 Publication History

Abstract

Brain-computer interfaces (BCI) and augmented reality (AR) are the most promising technologies in the metaverse. Inspired by traditional steady-state visual evoked potentials (SSVEP), technologies that combine SSVEP with AR are rapidly developing. In the existing studies on AR-SSVEP, the visual stimuli in AR are completely separated from the target object, while the SSVEP stimuli are always 2D graphics, which greatly limits the practical applications in real dynamic scenes. In this paper, we propose a novel AR-based visual stimulus scheme that uses a depth camera to obtain the position information of the target object in the displayed scene, and then completely overlaps the visual stimulus of SSVEP with the target object. The subject wearing the AR headset can move his head or walk in the AR scene, while the 3D position of the stimulus always overlaps the target object to avoid the effect of head shaking. Moreover, subjects can observe the 3D stimulus from different angles. We designed experiments comparing 2D stimuli with 3D stimuli from different angles in this paper to demonstrate the superiority of 3D stimuli in an AR environment. In addition to this, we used a user study to evaluate subjects’ experience with the AR-SSVEP stimulus paradigm in this paper and confirmed that the AR-SSVEP system with 3D stimulus overlapping with the target object has a better interaction experience.

References

[1]
P. Arpaia, E. De Benedetto, L. De Paolis, G. D'Errico, N. Donato, and L. Duraccio. Performance enhancement of wearable instrumentation for AR-based SSVEP BCI. Measurement, 196:111188, 2022.
[2]
L. Chen, P. Chen, S. Zhao, Z. Luo, W. Chen, Y. Pei, H. Zhao, J. Jiang, M. Xu, Y. Yan, Adaptive asynchronous control system of robotic arm based on augmented reality-assisted brain–computer interface. Journal of Neural Engineering, 18(6):066005, 2021.
[3]
B. Fang, W. Ding, F. Sun, J. Shan, X. Wang, C. Wang, and X. Zhang. Brain-computer interface integrated with augmented reality for humanrobot interaction. IEEE Transactions on Cognitive and Developmental Systems, 2022.
[4]
S. Ge, Y. Jiang, M. Zhang, R. Wang, K. Iramina, P. Lin, Y. Leng, H. Wang, and W. Zheng. SSVEP-based brain-computer interface with a limited number of frequencies based on dual-frequency biased coding. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 29:760–769, 2021.
[5]
Y. Ke, P. Liu, X. An, X. Song, and D. Ming. An online SSVEP-BCI system in an optical see-through augmented reality environment. Journal of neural engineering, 17(1):016066, 2020.
[6]
Y. Wang, K. Li, X. Zhang, J. Wang, and R. Wei. Research on the application of augmented reality in SSVEP-BCI. In Proceedings of the 2020 6th International Conference on Computing and Artificial Intelligence, pp. 505–509, 2020.
[7]
J. R. Wolpaw, G. E. Loeb, B. Z. Allison, E. Donchin, O. F. do Nascimento, W. J. Heetderks, F. Nijboer, W. G. Shain, and J. N. Turner. Bci meeting 2005-workshop on signals and recording methods. IEEE Transactions on neural systems and rehabilitation engineering, 14(2):138– 141, 2006.
[8]
S. Zhang, X. Gao, and X. Chen. Humanoid robot walking in maze controlled by SSVEP-BCI based on augmented reality stimulus. Frontiers in Human Neuroscience, 16, 2022.
[9]
X. Zhao, Z. Wang, M. Zhang, and H. Hu. A comfortable steady state visual evoked potential stimulation paradigm using peripheral vision. Journal of Neural Engineering, 18(5):056021, 2021.
[10]
X. Chen, Y. Wang, S. Gao, T.-P. Jung, and X. Gao. Filter bank canonical correlation analysis for implementing a high-speed SSVEP-based brain– computer interface. Journal of neural engineering, 12(4):046008, 2015. 2.
[11]
Y. Du and X. Zhao. Visual stimulus color effect on SSVEP-BCI in augmented reality. Biomedical Signal Processing and Control, 78:103906, 2022. 2
[12]
P. Liu, Y. Ke, J. Du, W. Liu, L. Kong, N. Wang, X. An, and D. Ming. An SSVEP-BCI in augmented reality. In 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 5548–5551. IEEE, 2019. 2.
[13]
Y. Wang, K. Li, X. Zhang, J. Wang, and R. Wei. Research on the application of augmented reality in SSVEP-BCI. In Proceedings of the 2020 6th International Conference on Computing and Artificial Intelligence, pp. 505–509, 2020. 2 .
[14]
X. Zhao, C. Liu, Z. Xu, L. Zhang, and R. Zhang. SSVEP stimulus layout effect on accuracy of brain-computer interfaces in augmented reality glasses. IEEE Access, 8:5990–5998, 2020. 2.
[15]
R. Zhang, Z. Xu, L. Zhang, L. Cao, Y. Hu, B. Lu, L. Shi, D. Yao, and X. Zhao. The effect of stimulus number on the recognition accuracy and information transfer rate of SSVEP-BCI in augmented reality. Journal of Neural Engineering, 19(3):036010, 2022. 2.

Index Terms

  1. 3D SSVEP Visual stimulus in Augmented Reality for Robotic Arm Grasping

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Other conferences
      ICCPR '23: Proceedings of the 2023 12th International Conference on Computing and Pattern Recognition
      October 2023
      589 pages
      ISBN:9798400707988
      DOI:10.1145/3633637
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 28 February 2024

      Permissions

      Request permissions for this article.

      Check for updates

      Qualifiers

      • Research-article
      • Research
      • Refereed limited

      Conference

      ICCPR 2023

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • 0
        Total Citations
      • 40
        Total Downloads
      • Downloads (Last 12 months)40
      • Downloads (Last 6 weeks)9
      Reflects downloads up to 20 Feb 2025

      Other Metrics

      Citations

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      HTML Format

      View this article in HTML Format.

      HTML Format

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media