Abstract
Considered in this paper is the design of output layer nodes of extreme learning machine (ELM) for solving multi-class classification problems with r (\(r\ge 3\)) classes of samples. The common and conventional setting of output layer, called “one-to-one approach” in this paper, is as follows: The output layer contains r output nodes corresponding to the r classes. And for an input sample of the ith class (\(1\le i\le r\)), the ideal output is 1 for the ith output node, and 0 for all the other output nodes. We propose in this paper a new “binary approach”: Suppose \(2^{q-1}< r\le 2^q\) with \(q\ge 2\), then we let the output layer contain q output nodes, and let the ideal outputs for the r classes be designed in a binary manner. Numerical experiments carried out in this paper show that our binary approach does equally good job as, but uses less output nodes and hidden-output weights than, the traditional one-to-one approach.
Similar content being viewed by others
References
Haykin Simon (1994) Neural networks: a comprehensive foundation. Prentice Hall PTR, Upper Saddle River
Zurada Jacek M (1992) Introduction to artificial neural systems, vol 8. West publishing company, St. Paul
Rumelhart David E, Hinton Geoffrey E, Williams Ronald J (1986) Learning representations by back-propagating errors. Nature 323(6088):553–536
LeCun Yann, Boser Bernhard, Denker John S, Henderson Donnie, Howard Richard E, Hubbard Wayne, Jackel Lawrence D (1989) Backpropagation applied to handwritten zip code recognition. Neural Comput 1(4):541–551
Huang Guang-Bin, Zhu Qin-Yu, Siew Chee-Kheong et al (2004) Extreme learning machine: a new learning scheme of feedforward neural networks. Neural Netw 2:985–990
Huang Guang-Bin, Zhu Qin-Yu, Siew Chee-Kheong (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1–3):489–501
Huang Guang-Bin, Wang Dian Hui, Lan Yuan (2011) Extreme learning machines: a survey. Int J Mach Learn Cybern 2(2):107–122
Huang Guang-Bin, Chen Lei (2008) Enhanced random search based incremental extreme learning machine. Neurocomputing 71(16–18):3460–3468
Huang Gao, Song Shiji, Gupta Jatinder ND, Wu Cheng (2014) Semi-supervised and unsupervised extreme learning machines. IEEE Trans Cybern 44(12):2405–2417
Luo Jiahua, Vong Chi-Man, Wong Pak-Kin (2013) Sparse bayesian extreme learning machine for multi-classification. IEEE Trans Neural Netw Learn Syst 25(4):836–843
Tang Jiexiong, Deng Chenwei, Huang Guang-Bin (2015) Extreme learning machine for multilayer perceptron. IEEE Trans Neural Netw Learn Syst 27(4):809–821
Law Rob (2000) Back-propagation learning in improving the accuracy of neural network-based tourism demand forecasting. Tourism Manag 21(4):331–340
O’Hara Toby, Bull Larry (2005) Building anticipations in an accuracy-based learning classifier system by use of an artificial neural network. In: IEEE congress on evolutionary computation, vol 3. IEEE, pp 2046–2052
Specht Donald F, Shapiro Philip D (1991) Generalization accuracy of probabilistic neural networks compared with backpropagation networks. In: IJCNN-91-Seattle international joint conference on neural networks, vol 1. IEEE, pp 887–892
Wu Wei, Fan Qinwei, Zurada Jacek M, Wang Jian, Yang Dakun, Liu Yan (2014) Batch gradient method with smoothing l1/2 regularization for training of feedforward neural networks. Neural Netw 50:72–78
Fan Qinwei, Zurada Jacek M, Wei Wu (2014) Convergence of online gradient method for feedforward neural networks with smoothing l1/2 regularization penalty. Neurocomput 131:208–216
Han S, Mao H, Dally WJ (2015) Deep compression: compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv preprint arXiv:1510.00149
Nie Qingfeng, Jin Lizuo, Fei Shumin, Ma Junyong (2015) Neural network for multi-class classification by boosting composite stumps. Neurocomputing 149:949–956
Sateesh Babu G, Suresh Sundaram (2012) Meta-cognitive neural network for classification problems in a sequential learning framework. Neurocomputing 81:86–96
Quteishat Anas, Lim Chee Peng, Tweedale Jeffrey, Jain Lakhmi C (2009) A neural network-based multi-agent classifier system. Neurocomputing 72(7–9):1639–1647
Guobin O, Yi L (2007) Multi-class pattern classification using neural networks. Pattern Recognit 40(1):4–18
Banerjee KS (1973) Generalized inverse of matrices and its applications
Serre D (2002) Matrices: theory and applications. Springer, New York
Yang Sibo, Zhang Chao, Wei Wu (2018) Binary output layer of feedforward neural networks for solving multi-class classification problems. IEEE Access 7:5085–5094
Yao M, Sun Q, Yang S, Wang J, Wu W (2014) Binary output of multiple linear perceptrons with three hidden nodes for classification problems. In: Proceedings on the international conference on artificial intelligence (ICAI), p 1. The Steering Committee of The World Congress in Computer Science, Computer
Sun Q, Liu Y, Li Z, Yang S, Wu W, Jin J (2013) The binary output units of neural network. In: Guo C, Hou Z-G, Zeng Z (eds) Advances in neural networks. Springer, Berlin, pp 250–257
Chen Xue Zhen, Zhu Yong Li, Pei Fei (2014) Prediction research of transformer fault based on regular extreme learning machine. In Advanced Materials Research, vol 1049. Trans Tech Publ, Stafa-Zurich, pp 1205–1209
Rohra JG, Perumal B, Narayanan SJ, Thakur P, Bhatt RB (2017) User localization in an indoor environment using fuzzy hybrid of particle swarm optimization & gravitational search algorithm with neural networks. In: Proceedings of 6th international conference on soft computing for problem solving. Springer, pp 286–295
Acknowledgements
This work is supported by the National Natural Science Foundation of China (No. 11401076, 61473328, 11201051 and 61473059), Shanghai Municipal Science and Technology Major Project (No. 2018SHZDZX01) and ZJLab.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Yang, S., Zhang, C., Bao, Y. et al. Binary Output Layer of Extreme Learning Machine for Solving Multi-class Classification Problems. Neural Process Lett 52, 153–167 (2020). https://doi.org/10.1007/s11063-020-10236-5
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11063-020-10236-5