Abstract
Broad learning system (BLS) is viewed as a class of neural networks with a broad structure, which exhibits an efficient training process through incremental learning. An incremental Bayesian framework broad learning system is proposed in this study, where the posterior mean and covariance over the output weights are both derived and updated in an incremental manner for the increment of feature nodes, enhancement nodes, and input data, respectively, and the hyper-parameters are simultaneously updated by maximizing the evidence function. In such a way, the scale of matrix operations is capable of being effectively reduced. To verify the performance of this proposed approach, a number of experiments by using four benchmark datasets and an industrial case are carried out. The experimental results demonstrate that the proposed method can not only achieve a better outcome compared to the classical BLS and other comparative algorithms but also incrementally remodel the system.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Chen CLP (2019) Broad learning system. https://broadlearning.ai
Chen T, Guestrin C (2016) Xgboost: a scalable tree boosting system. In: Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, pp. 785–794
Chen CLP, Liu Z (2017) Broad learning system: an effective and efficient incremental learning system without the need for deep architecture. IEEE Trans Neural Netw Learn Syst 29(1):10–24
Chen L, Liu Y, Zhao J et al (2016) Prediction intervals for industrial data with incomplete input using kernel-based dynamic Bayesian networks. Artif Intell Rev 46(3):307–326
Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297
Center for Machine Learning and Intelligent Systems, UCI machine learning repository: Datasets (2017). https://archive.ics.uci.edu/ml/datasets.html
Documentation for GPML matlab code version 4.2. (2018) www.GaussianProcesses.org/gpml
Feng S, Chen CLP (2018) Fuzzy broad learning system: a novel neuro-fuzzy model for regression and classification. IEEE Trans Cybern. https://doi.org/10.1109/TCYB.2018.2857815
Fish-kong, LSTM-regression (2019). https://github.com/fish-kong/LSTM-regression
Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780
Hoerl AE, Kennard RW (1970) Ridge regression: Biased estimation for nonorthogonal problems. Technometrics 12(1):55–67
Home Toolbox Book People Publications Faq Links (2018) https://www.esat.kuleuven.be/sista/lssvmlab
Jin J, Chen CLP (2018) Regularized robust broad learning system for uncertain data modeling. Neurocomputing 322:58–69
Jin F, Zhao J, Han Z, Wang W (2018) A joint scheduling method for multiple byproduct gases in steel industry. Control Eng Pract 80:174–184
Jin J, Liu Z, Chen CLP (2018) Discriminative graph regularized broad learning system for image recognition. Sci China Inf Sci 61(11):179–192
Kong Y, Wang X, Cheng Y, Chen CLP (2018) Hyperspectral imagery classification based on semi-supervised broad learning system. Remote Sens 10:685
Kong Y, Cheng Y, Chen CLP, Wang X (2019) Hyperspectral image clustering based on unsupervised broad learning. IEEE Geosci Remote Sens Lett 11(16):1741–1745
Li D, Ji S, Zhang C (2018) Improved broad learning system: partial weights modification based on BP algorithm. IOP Conf Ser Mater Sci Eng 439(3):032083
Nguyen G, Dlugolinsky S, Bobák M et al (2019) Machine learning and deep learning frameworks and libraries for large-scale data mining: a survey. Artif Intell Rev 52:77–124
Petersen KB, Pedersen MS (2008) The Matrix Cookbook. Tech Univ Den 7(15):510
Rasmussen CE, Williams C (2006) Gaussian processes for machine learning. MIT Press, Cambridge
Suykens JAK, Vandewalle J (1999) Least squares support vector machine classifiers. Neural Process Lett 9(3):293–300
ThunderSVM: A fast SVM library on GPUs and CPUs (2020). https://thundersvm.readthedocs.io/en/latest/index.html
Tipping ME (2001) Sparse bayesian learning and the relevance vector machine. J Mach Learn Res 1:211–244
Tipping ME (2003) Bayesian inference: an introduction to principles and practice in machine learning. Summer School on Machine Learning, Berlin, pp 41–62
Tipping ME (2016) Sparse Bayesian models (and the RVM). https://www.relevancevector.com
Weimer D, Scholz-Reiter B, Shpitalni M (2016) Design of deep convolutional neural network architectures for automated feature extraction in industrial inspection. CIRP Ann 65(1):417–420
XGBoost Documentation (2020) https://xgboost.readthedocs.io/en/latest/
Xu M, Han M, Chen CLP, Qiu T (2018) Recurrent broad learning systems for time series prediction. IEEE Trans Cybern. https://doi.org/10.1109/TCYB.2018.2863020
Zhang Q, Yang L, Chen Z, Li P, Bu F (2018) An adaptive dropout deep computation model for industrial IoT big data learning with crowdsourcing to cloud computing. IEEE Trans Industr Inf 15(4):2330–2337
Zhao J, Chen L, Pedrycz W, Wang W (2018) Variational inference based automatic relevance determination kernel for embedded feature selection of noisy industrial data. IEEE Trans Industr Electron 66(1):416–428
Acknowledgements
This work was supported by the National Natural Sciences Foundation of China under Grant 61873048, Grant 61833003, Grant 61533005, Grant U1908218, Grant 61773085, the Fundamental Research Funds for the Central Universities under Grant DUT19JC40, Grant DUT18TD07, Grant DUT20RC(3)013, the National Key R&D Program of China under Grant 2017YFA0700300, and the Outstanding Youth Sci-Tech Talent Program of Dalian under Grant 2018RJ01.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Liu, Y., Wang, Y., Chen, L. et al. Incremental Bayesian broad learning system and its industrial application. Artif Intell Rev 54, 3517–3537 (2021). https://doi.org/10.1007/s10462-020-09929-z
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10462-020-09929-z