Elsevier

Neurocomputing

Volume 394, 21 June 2020, Pages 61-69
Neurocomputing

Driving amount based stochastic configuration network for industrial process modeling

https://doi.org/10.1016/j.neucom.2020.02.029Get rights and content

Abstract

Stochastic configuration network (SCN) that randomly assigns the weights connecting the input layer and the hidden layer with an inequality constraint can achieve a fast learning speed for dealing with regression tasks. In this paper, a driving amount based SCN (DASCN) is proposed to improve the performance in terms of generalization and structure compactness, which have gained considerable attention in the industrial process. In the proposed DASCN, the driving amount incorporated into SCN is used to further improve the structural parameters, especially the output weights, of SCN. The performance of DASCN is evaluated by function approximation, four benchmark datasets and practical application in the industrial process. The simulation results indicate that the DASCN has better generalization capability and a more compact network structure compared to other methods.

Introduction

The stable and efficient operation of industrial processes is realized on the basis of the corresponding optimization and control algorithms [1], [2], [3]. In fact, both optimization and control of industrial processes are difficult to achieve. The reason lies in the fact that optimal operation of an actual industrial system relies greatly on good measurements of the operation indexes, which are difficult to be real-time measured using the existing conventional technologies [4]. Fortunately, the state of the operation indexes can be estimated using an appropriate model. The modeling approaches can be mainly divided into two types. The first type is first-principles method that requires the priori process-related knowledge [5], [6], [7]. The main bottleneck is that due to the high-complexity of the industry process, deriving the prior mechanical information from the process becomes very difficult. Moreover, it is generally necessary to make some theoretical assumptions in first-principles modeling, which may lead to a modeling error.

The second type, which uses the process data to build the input-output models of industrial systems, is known as data-driven modeling method [8], [9], [10], [11]. The data-driven modeling method has gained increasing attention in the industrial field, since it does not require the prior knowledge of the process. Hence, the data-driven modeling method is a good alternative to solve the problem of operation index measurement in the industry process [12], [13].

Neural networks have been widely employed in the modeling of industrial processes [14], [15]. The neural network is trained using the slow gradient-based algorithm, which makes the learning speed far slower than required. A random vector functional link network (RVFLN) is proposed [16], [17]. RVFLN is established by the two-step approach: first randomly generate the hidden-layer parameters, then evaluate the output weights by minimizing a line equation. As a result, the RVFLN has superior performance in terms of training speed and simplicity of computation.

The network structure plays a key role in the performance of RVFLN. Recently, numerous algorithms have been developed to automatically select the suitable network structure. Generally speaking, these methods can fall into two main types, constructive algorithms [18], [19], [20] and pruning algorithms [21], [22], [23]. For constructive algorithms, the hidden units are added to the network one by one (or group by group) until a satisfactory solution is found. Contrary to constructive algorithms, the pruning algorithms remove the hidden units one by one (or group by group) until an acceptable tolerance is satisfied. Compared with the pruning algorithms, the constructive algorithms perform better in terms of settings of network size and computational efficiency [24].

Recently, a class of incremental RVFLN (IRVFLN) is fully discussed in [25]. It has been pointed out that the universal approximation capacity of IRVFLN cannot be ensured as the hidden parameters are randomly chosen from a fixed scope. This implies that some additional conditions required are constructed to ensure the universal approximation capacity of these RVFLN-based models. A new learner model namely stochastic configuration network (SCN) is proposed to solve the constructing problem of RVFLN [26]. An inequality constraint is given to achieve the universal approximation capacity. The inequality constraint is used to randomly assign the hidden parameters and adaptively select their scopes, which is helpful for successfully building a universal approximator.

Inspired by the existing IRVFLN optimization algorithms mentioned above, a new driving amount based SCN called DASCN is proposed in this paper. The DASCN algorithm consists of two versions: DASCN-I and DASCN-II. Both DASCN-I and DASCN-II employ the same inequality constraint for random assignment of the hidden parameters, and evaluate the output weights using different approaches. The former calculates the output weights by minimizing the local network residual error, and the output weights of DASCN-II are evaluated by solving a global least squares problem. The simulation results with comparisons show that the proposed algorithm DASCN-II is superior to normal IRVFLN, DASCN-I and SCN in terms of generalization and structure compactness. The contribution of this paper can be summarized as follows:

  • (1)

    The driving amount, which adds the error-feedback term to the output layer, can dynamically adjust the training parameters of network and enhance generalization.

  • (2)

    Under the framework of SCN, it is proved in theory that DASCN with high performance in structure compactness and generalization possesses the property of universal approximation.

The rest of this paper is organized as follows. In Section 2, two randomized learning algorithms (normal IRVFLN and SCN) are briefly introduced. In Section 3, we introduce the DASCN, which includes the theoretical analysis and algorithm description. In Section 4, we measure the performance of the DASCN by function approximation and four regression tasks. The data-driven modeling of a real world application is performed to verify the DASCN in contrast to other algorithms in Section 5. Concluding remarks are given in Section 6.

Section snippets

IRVFLN

Let Γ:={g1,g2,g3} be a set of real-valued functions. Denote by span(Γ) and L2(D) a function space spanned by Γ and the space of all Lebesgue measurable functions f=[f1,f2,,fm]:RdRm defined on DRd, respectively.

The output of a single layer feed-forward network (SLFN) with L1 hidden nodes can be expressed asfL1(x)=j=1L1βjgj(wjTx+bj)(L=1,2,,f0=0),where wj, bj and βj denote the parameters of the jth hidden node, and gj(wjTx+bj) is the output function of the jth hidden node.

The current

Stochastic configuration network based on driving amount

In this section, we first verify that the proposed DASCN has the universal approximation capacity, then give the implementation code of DASCN in Section 3.2.

Simulations and discussions

In this section, we investigate the generalization performance of the DASCN. Comparison of the IRVFLN, SCN, DASCN-I and DASCN-II is performed through function approximation and four benchmark datasets. All the experiments are carried out on Matlab 2016a, using a PC equipped with a core i5, 3.4 G Hz CPU and 8 GB RAM.

In this experiment, the root mean square error (RMSE) is used to evaluate the generalization ability of the IRVFLN, SCN, DASCN-I and DASCN-II, and the final results are obtained by

Application in industrial data modeling

The above section has verified the effectiveness of the proposed DASCN. In this section, the data-driven modeling of main fan switchover process (MFSP) is used to illustrate the DASCN. We first give a detailed description of MFSP. Finally, the experiment results are discussed in Section 5.2.

Conclusion

In this paper, two new DASCN based learning method, namely DASCN-I based method and DASCN-II based method, are developed. It has been shown by simulation results on function approximation and regression tasks from the KEEL dataset repository that the proposed DASCN-II can achieve better generalization performance with compact network structure than IRVFLN, DASCN-I and SCN. In addition, the DASCN-I and DASCN-II are used to estimate the state of the UAQ index in the MFSP. The DASCN-II can achieve

CRediT authorship contribution statement

Qianjin Wang: Writing - original draft, Writing - review & editing. Wei Dai: Visualization, Writing - original draft, Writing - review & editing. Xiaoping Ma: Writing - review & editing. Zhigen Shang: Writing - review & editing.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

This work was supported by the National Natural Science Foundation of China (61973306, 61873272, 61603392, 61603393, 61503384), Natural Science Foundation of Jiangsu Province (BK20191043), Jiangsu Dual Creative Teams Programme Project (2017) and Funding for School-Level Research Projects of Yancheng Institute of Technology (xjr2019017, xjr2019018).

Qianjin Wang received the M.S. and Ph.D. degrees from the School of Information and Control Engineering, China University of Mining and Technology, Xuzhou, in 2011 and 2018, respectively. He is currently a Lecturer in the Department of Automation at Yancheng Institute of Technology. His research interests cover data-driven modeling and control, and machine learning algorithm.

References (33)

  • D. Wang et al.

    Stochastic configuration networks: Fundamentals and algorithms

    IEEE Trans. Cybern.

    (2017)
  • J. Alcalá-Fdez et al.

    Keel data-mining software tool: data set repository, integration of algorithms and experimental analysis framework

    J. Mult. Valued Logic Soft Comput.,

    (2011)
  • Q.J. Wang et al.

    Multiple models and neural networks based adaptive PID decoupling control of mine main fan switchover system

    IET Control Theory Appl.

    (2018)
  • J. Barreiro-Gomez et al.

    Distributed population dynamics: Optimization and control applications

    IEEE Trans. Syst. Man Cybern. Syst.

    (2016)
  • T.C. Zeng et al.

    Joint communication and control for wireless autonomous vehicular platoon systems

    (2018)
  • P. Zhou et al.

    Data-driven robust RVFLNs modeling of a blast furnace iron-making process using cauchy distribution weighted m-estimation

    IEEE Trans. Ind. Electron.

    (2017)
  • Cited by (0)

    Qianjin Wang received the M.S. and Ph.D. degrees from the School of Information and Control Engineering, China University of Mining and Technology, Xuzhou, in 2011 and 2018, respectively. He is currently a Lecturer in the Department of Automation at Yancheng Institute of Technology. His research interests cover data-driven modeling and control, and machine learning algorithm.

    Wei Dai received the M.S. and Ph.D. degrees in control theory and control engineering from Northeastern University, Shenyang, China, in 2009 and 2015, respectively. He is currently with the China University of Mining and Technology, Xuzhou, China. His current research interests include modeling, optimization and control of complex system, data mining, and machine learning.

    Xiaoping Ma received the B.S., M.S., and Ph.D. degrees from the School of Information and Electrical Engineering, China University of Mining and Technology, Xuzhou, in 1982, 1989, and 2001, respectively. He is currently a Professor with the China University of Mining and Technology. His research interests include process control, networked control system, and fault detection.

    Zhigen Shang was born in 1979. He received the M.S. degree and Ph.D. degree in control theory and engineering from Shanghai University in 2006, and from Southeast University in 2013, respectively. Currently, he is an associate professor in the Department of Automation at Yancheng Institute of Technology. His research interests include intelligent algorithm, and applications of forecasting technology.

    View full text