EEG serves as an essential tool for brain source localization due to its high temporal resolution. However, the inference of brain activities from the EEG data is, in general, a challenging ill-posed inverse problem. To better retrieve task related discriminative source patches from strong spontaneous background signals, we propose a novel EEG source imaging model based on spatial and temporal graph structures. In particular, graph fractional-order total variation (gFOTV) is used to enhance spatial smoothness, and the label information of brain state is enclosed in a temporal graph regularization term to guarantee intra-class consistency of estimated sources. The proposed model is efficiently solved by the alternating direction method of multipliers (ADMM). A two-stage algorithm is proposed as well to further improve the result. Numerical experiments have shown that our method localizes source extents more effectively than the benchmark methods.