Generalized transformer in fault diagnosis of Tennessee Eastman process

Abstract

Fault diagnosis is an important yet challenging task. Because of the powerful feature representation capabilities of deep model, intelligent fault diagnosis on deep learning becomes a research hotspot in the field. Although many deep models as sparse autoencoder, deep belief network is developed for fault diagnosis with encouraging performance, integrating the merits of deep learning into fault diagnosis still has a long way to go. In this paper, we propose a novel method, namely generalized transformer. Compared to previous deep models, generalized transformer excavates relations among inputs and nonlinearity between inputs and outputs by attention mechanism. To deal with structured data, generalized transformer further borrows the idea from graph attention network. By replacing dot product between query and key information in transformer, we introduce a forward network with learned weight vector to compute the similarity. Through limiting the similarity calculations in a neighbor region, prior knowledge can be injected into generalized transformer. On Tennessee Eastman process dataset, our new model can produce high performance, which is better or competitive to state-of-the-art models. Extensive ablation studies validate the effectiveness of the proposed model.