Skip to main content

Advertisement

Springer Nature Link
Log in

Indoor space target searching based on EEG and EOG for UAV

  • Methodologies and Application
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

This paper puts forward a noninvasive electrooculography (EOG) and electroencephalogram (EEG)-based hybrid computer interface (HCI) system to implement the indoor target searching in three-dimensional (3D) space for a low-speed multi-rotor aircraft. The HCI system mainly consists of three subsystems, including the interface switching, decision and semi-autonomous navigation. The interface switching subsystem is accomplished by detecting the eyeblink EOG. The continuous wavelet transform is employed to indentify eyeblink features which are used to switch interfaces between horizontal and vertical motor imagery (MI) tasks. The average accuracy of the eyeblink feature detection reaches to 97.95%. The decision subsystem employs the joint regression (JR) model and spectral powers methods to extract the time and frequency domain features of MI tasks by analyzing the left- and right-hand MI EEG. Simultaneously, the support vector machine is applied to accomplish the MI tasks classification and final decision. The average classification accuracy of the HCI system reaches to 93.99%. The semi-autonomous navigation subsystem extracts the environmental features to avoid obstacles semi-automatically in 3D space and provide feasible directions for the decision subsystem. The actual indoor 3D space target searching experiments are put forward to verify the feasibility and adaptation performances of this proposed HCI system.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+
from $39.99 /Month
  • Starting from 10 chapters or articles per month
  • Access and download chapters and articles from more than 300k books and 2,500 journals
  • Cancel anytime
View plans

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Explore related subjects

Discover the latest articles and news from researchers in related subjects, suggested using machine learning.

References

  • Ahn JW, Ku Y, Kim DY, Sohn J, Kim JH, Kim HC (2018) Wearable in-the-ear eeg system for ssvep-based brain–computer interface. Electron Lett 54(7):413–414

    Article  Google Scholar 

  • Beniczky S, Conradsen I, Henning O, Fabricius M, Wolf P (2018) Automated real-time detection of tonic-clonic seizures using a wearable emg device. Neurology 90(5):428–434

    Article  Google Scholar 

  • Blankertz B, Mller KR, Curio G, Vaughan TM, Schalk G, Wolpaw JR, Schloegl A, Neuper C, Pfurtscheller G, Hinterberger T, Schroeder M, Birbaumer N (2004) The BCI competition 2003: progress and perspectives in detection and discrimination of EEG single trials. IEEE Trans Biomed Eng 51(6):1044–1051

    Article  Google Scholar 

  • Bulling A, Ward JA, Gellersen H, Troster G (2011) Eye movement analysis for activity recognition using electrooculography. IEEE Trans Pattern Anal Mach Intell 33(4):741–753

    Article  Google Scholar 

  • Chau AL, Li X, Yu W (2014) Support vector machine classification for large datasets using decision tree and Fisher linear discriminant. Future Gener Comput Syst 36:57–65

    Article  Google Scholar 

  • Choi H, Kim Y (2014) Uav guidance using a monocular-vision sensor for aerial target tracking. Control Eng Pract 22(1):10–19

    Article  Google Scholar 

  • Cristianini N, Shawe-Taylor J (2000) An introduction to support vector machines and other kernel-based learning methods. Cambridge University Press, Cambridge

    Book  Google Scholar 

  • Fattahi D, Nasihatkon B, Boostani R (2013) A general framework to estimate spatial and spatio-spectral filters for EEG signal classification. Neurocomputing 119:165–174

    Article  Google Scholar 

  • Fu K, Qu JF, Chai Y, Dong Y (2014) Classification of seizure based on the time-frequency image of EEG signals using HHT and SVM. Biomed Signal Process Control 13:15–22

    Article  Google Scholar 

  • Gandolfi M, Formaggio E, Geroin C, Storti SF, Galazzo IB, Bortolami M, Saltuari L, Picelli A, Waldner A, Manganotti P, Smania N (2018) Quantification of upper limb motor recovery and eeg power changes after robot-assisted bilateral arm training in chronic stroke patients: a prospective pilot study. Neural Plast 1:1–15

    Article  Google Scholar 

  • Ghasemi JB, Heidari Z, Jabbari A (2013) Toward a continuous wavelet transform-based search method for feature selection for classification of spectroscopic data. Chemometr Intell Lab Syst 127:185–194

    Article  Google Scholar 

  • Ghayab HA, Li Y, Siuly S, Abdulla S (2018) Epileptic EEG signal classification using optimum allocation based power spectral density estimation. IET Signal Proc 12(6):1–13

  • Heo J, Yoon H, Park KS (2017) A novel wearable forehead EOG measurement system for human computer interfaces. Sensors 17(7):1485

    Article  Google Scholar 

  • Hu S, Tian Q, Cao Y (2013) Motor imagery classification based on joint regression model and spectral power. Neural Comput Appl 23:1931–1936

    Article  Google Scholar 

  • Huang Q, He S, Wang Q, Gu Z, Peng N, Kai L, Zhang Y, Shao M, Li Y (2017) An eog-based human–machine interface for wheelchair control. IEEE Trans Biomed Eng 99:1

    Google Scholar 

  • Jung TP, Humphries C, Lee TW, Makeig S, McKeown MJ, Iragui V, Sejnowski TJ (1998) Extended ICA removes artifacts from electroencephalographic recordings. Adv Neural Inf Process Syst 10:894–900

    Google Scholar 

  • Kar S, Bhagat M, Routray A (2010) EEG signal analysis for the assessment and quantification of driver’s fatigue. Transp Res Part F Traffic Psychol Behav 13(5):297–306

    Article  Google Scholar 

  • Karson CN, Berman KF, Donnelly EF, Mendelson WB, Kleinman JE, Wyatt RJ (1981) Speaking, thinking, and blinking. Psychiatry Res 5(3):243–246

    Article  Google Scholar 

  • Kim BH, Kim M, Jo S (2014) Quadcopter flight control using a low-cost hybrid interface with eeg-based classification and eye tracking. Comput Biol Med 51(15):82–92

    Article  Google Scholar 

  • Lanillos P, Gan SK, Besada-Portas E, Pajares G, Sukkarieh S (2014) Multi-uav target search using decentralized gradient-based negotiation with expected observation. Inf Sci 282:92–110

    Article  MathSciNet  Google Scholar 

  • Li J, Yan J, Liu X, Ouyang G (2014) Using permutation entropy to measure the changes in EEG signals during absence seizures. Entropy 16(6):3049–3061

    Article  Google Scholar 

  • López A, Ferrero F, Yangüela D, Álvarez C, Postolache O (2017) Development of a computer writing system based on EOG. Sensors 17(7):1505

    Article  Google Scholar 

  • Lotte F, Guan C (2011) Regularizing common spatial patterns to improve BCI designs: unified theory and new algorithms. IEEE Trans Biomed Eng 58(2):355–362

    Article  Google Scholar 

  • Lotte F, Congedo M, Lecuyer A, Lamarche F, Arnaldi B (2007) A review of classification algorithms for EEG-based brain–computer interfaces. J Neural Eng 4(2):1–13

    Article  Google Scholar 

  • Lührs M, Goebel R (2017) Turbo-satori: a neurofeedback and brain–computer interface toolbox for real-time functional near-infrared spectroscopy. Neurophotonics 4(4):041504

    Article  Google Scholar 

  • Ma J, Zhang Y, Cichocki A, Matsuno F (2015) A novel EOG/EEG hybrid human–machine interface adopting eye movements and ERPS: application to robot control. IEEE Trans Bio Med Eng 62(3):876–889

    Article  Google Scholar 

  • Minati L, Yoshimura N, Koike Y (2016) Hybrid control of a vision-guided robot arm by EOG, EMG, EEG biosignals and head movement acquired via a consumer-grade wearable device. IEEE Access 4:9528–9541

    Article  Google Scholar 

  • Moghadamfalahi M, Akcakaya M, Nezamfar H, Sourati J, Erdogmus D (2017) An active RBSE framework to generate optimal stimulus sequences in a BCI for spelling. IEEE Trans Signal Process 65(20):5381–5392

    Article  MathSciNet  Google Scholar 

  • Nguyen QX, Jo S (2012) Electric wheelchair control using head pose free eyegaze tracker. Electron Lett 48:750–752

    Article  Google Scholar 

  • Orfanus D, Freitas ED, Eliassen F (2016) Self-organization as a supporting paradigm for military UAV relay networks. IEEE Commun Lett 20(4):804–807

    Article  Google Scholar 

  • Ramli R, Arof H, Ibrahim F, Mokhtar N, Idris MYI (2015) Using finite state machine and a hybrid of EEG signal and EOG artifacts for an asynchronous wheelchair navigation. Expert Syst Appl 42(5):2451–2463

    Article  Google Scholar 

  • Riccio A, Leotta F, Bianchi L, Aloise F, Zickler C, Hoogerwerf EJ, Kübler A, Mattia D, Cincotti F (2011) Workload measurement in a communication application operated through a P300-based brain–computer interface. J Neural Eng 8(2):025–028

    Article  Google Scholar 

  • Salvo G, Caruso L, Scordo A (2014) Urban traffic analysis through an UAV. Procedia—Soc Behav Sci 111:1083–1091

    Article  Google Scholar 

  • Samek W, Vidaurre C, Müller KR, Kawanabe M (2012) Stationary common spatial patterns for brain–computer interfacing. J Neural Eng 9(2):026013

    Article  Google Scholar 

  • Shi T, Wang H, Zhang C (2015) Brain computer interface system based on indoor semi-autonomous navigation and motor imagery for unmanned aerial vehicle control. Expert Syst Appl 42(9):4196–4206

    Article  Google Scholar 

  • Siebert S, Teizer J (2014) Mobile 3D mapping for surveying earthwork projects using an unmanned aerial vehicle (UAV) system. Autom Constr 41:1–14

    Article  Google Scholar 

  • Tuna G, Nefzi B, Conte G (2014) Unmanned aerial vehicle-aided communications system for disaster recovery. J Netw Comput Appl 41:27–36

    Article  Google Scholar 

  • Varela G, Caamaño P, Orjales F, Deibe Á, López-Peña F, Duro RJ (2014) Autonomous UAV based search operations using constrained sampling evolutionary algorithms. Neurocomputing 132:54–67

    Article  Google Scholar 

  • Vourvopoulos A, Badia SBI, Liarokapis F (2017) EEG correlates of video game experience and user profile in motor-imagery-based brain–computer interaction. Vis Comput 33(4):533–546

    Article  Google Scholar 

  • Wu W, Gao X, Hong B, Gao S (2008) Classifying single-trial EEG during motor imagery by iterative spatio-spectral patterns learning (ISSPL). IEEE Trans Biomed Eng 55(6):1733–1743

  • Xu B, Song A, Wu J (2007) Algorithm of imagined left-right hand movement classification based on wavelet transform and AR parameter model. In: 1st international conference on bioinformatics and biomedical engineering, pp 539–542

  • Yahyanejad S, Rinner B (2015) A fast and mobile system for registration of low-altitude visual and thermal aerial images using multiple small-scale UAVs. ISPRS J Photogramm Remote Sens 104:189–202

    Article  Google Scholar 

  • Youn E, Koenig L, Jeong MK, Baek SH (2010) Support vector-based feature selection using Fisher’s linear discriminant and support vector machine. Expert Syst Appl 37(9):6148–6156

    Article  Google Scholar 

  • Zhang C, Wang H, Fu R (2014) Automated detection of driver fatigue based on entropy and complexity measures. IEEE Trans Intell Transp Syst 15(1):168–177

    Article  Google Scholar 

  • Zhao Z, Luo H, Song GH, Chen Z, Lu ZM, Wu X (2018) Web-based interactive drone control using hand gesture. Rev Sci Instrum 89(1):014707

    Article  Google Scholar 

Download references

Acknowledgements

Tianwei Shi has been supported by the Department of Education of Liaoning Province (2017FWDF03), Natural Science Foundation of Liaoning Province of China (20180550567) and University of Science and Technology Liaoning Youth Fund (2017QN05). Wenhua Cui has been supported by the Department of Education of Liaoning Province (2016HZZD05).

Author information

Authors and Affiliations

  1. School of International Finance and Banking, University of Science and Technology Liaoning, Anshan, 114051, Liaoning, China

    Tianwei Shi & Wenhua Cui

  2. Department of Mechanical Engineering and Automation, Northeastern University, Shenyang, 110004, Liaoning, China

    Hong Wang

  3. Research and Development Center, Liaoning Systemteq Information Technology Co., Ltd., Anshan, 114051, China

    Ling Ren

Authors
  1. Tianwei Shi
    View author publications

    Search author on:PubMed Google Scholar

  2. Hong Wang
    View author publications

    Search author on:PubMed Google Scholar

  3. Wenhua Cui
    View author publications

    Search author on:PubMed Google Scholar

  4. Ling Ren
    View author publications

    Search author on:PubMed Google Scholar

Corresponding author

Correspondence to Tianwei Shi.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Human and animal rights

This article contains the calibration experiments and actual indoor 3D space target searching experiments with human participants performed by the authors. Five males and five females participated in the experiments. This study was approved by the Human Research Protections Program of University of Science and Technology Liaoning and Northeastern University. Simultaneously, it was performed in accordance with the Declaration of Helsinki. All participants were asked to read and sign an informed consent form before participating in the study.

Additional information

Communicated by V. Loia.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shi, T., Wang, H., Cui, W. et al. Indoor space target searching based on EEG and EOG for UAV. Soft Comput 23, 11199–11215 (2019). https://doi.org/10.1007/s00500-018-3670-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-018-3670-3

Keywords