Electroencephalogram (EEG) based brain-computer interfaces (BCIs) are becoming popular in clinical diagnosis applications. However, this raises a new issue on the robustness of deep neural networks-based BCIs against environmental noise and adversarial attacks. Unfortunately, there is no adversarial defense approach tailored for EEG adversarial robustness so far. In this work, we systematically evaluate the performance of 5 popular adversarial training (AT)-based defense approaches on three large-scale and real-world EEG datasets with 3 popular EEG classification models, under 3 different white-box attacks. Through extensive experiments, we demonstrate that the naïve AT is a promising adversarial defense approach in EEG-based BCIs. However, existing regularization terms originated from vision tasks do not generalize well to EEG signals. Our results shed light on the future development of the EEG adversarial defense approach.