Although advances in deep learning technologies have greatly facilitated the brain intention decoding from electroencephalogram (EEG) in motor imagery brain–computer interfaces (MI-BCIs), significant individual differences hinder the practical cross-subject MI-BCI applications. Unlike other existing domain adversarial transfer networks that focus on designing different discriminators to reduce individual differences, inspired by the motor lateralization phenomenon, we innovatively utilize transformer and the spatiotemporal pattern differences of EEG as prior knowledge to enhance the feature discriminability in our brain decoding adversarial network. In addition, to address adversarial network decision boundaries bias toward the source domain, we propose a data augmentation method, EEGMix to rapidly mix and enrich the target domain data. With an adaptive adversarial factor, our decoding model reduces the differences in marginal and conditional distribution simultaneously. Three public MI datasets, 2a, 2b, and OpenBMI verified our model's effectiveness. The accuracy achieved 77.49%, 85.19%, and 79.37%, superior to other state-of-the-art algorithms.