Abstract
In this study, we introduce an online ensemble method based on convolutional neural networks (CNNs) for streaming data. Recent work has shown that a convolution operation has been an effective way to extract features. In particular, we proposed a CNN working in an online manner as a base classifier. Then, an ensemble approach is devised to boost the performance of all base classifiers. We also propose two loss terms which can adapt to the imbalanced data stream as well as handling the forgetting issue of deep networks. The experiments conducted on a number of datasets chosen from different sources demonstrate that the proposed ensemble approach performs significantly better than a single network and some well-known online learning algorithms including additive models and Online Bagging.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Libsvm. https://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/
Moa dataset. http://moa.cms.waikato.ac.nz/datasets/
The UCI dataset website. http://archive.ics.uci.edu/ml/datasets.html
Bifet, A., et al.: MOA: massive online analysis, a framework for stream classification and clustering. In: Proceedings of the First Workshop on Applications of Pattern Analysis. Proceedings of Machine Learning Research, vol. 11, pp. 44–50 (2010)
Crammer, K., Dekel, O., Keshet, J., Shalev-Shwartz, S., Singer, Y.: Online passive-aggressive algorithms. J. Mach. Learn. Res. 7, 551–585 (2006)
Crammer, K., Kulesza, A., Dredze, M.: Adaptive regularization of weight vectors. In: Proceedings of the 22nd International Conference on Neural Information Processing Systems, pp. 414–422 (2009)
Demsar, J.: Statistical comparisons of classifiers over multiple datasets. J. Mach. Learn. Res. 7, 1–30 (2006)
Domingos, P., Hulten, G.: Mining high-speed data streams. In: Proceedings of the KDD Conference, pp. 71–80 (2000)
Hoi, S.C.H., Wang, J., Zhao, P.: LIBOL: a library for online learning algorithms. J. Mach. Learn. Res. 15, 495–499 (2014)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: International Conference on Learning Representations (2014)
Krawczyk, B., Minku, L.L., Gama, J., Stefanowski, S.J., Woźniak, M.: Ensemble learning for data stream analysis: a survey. Inf. Fusion 37, 132–156 (2017)
Kubat, M., Matwin, S.: Addressing the curse of imbalanced training sets: one-sided selection. In: Proceedings of the Fourteenth International Conference on Machine Learning, pp. 179–186 (1997)
Lin, T., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2999–3007 (2017)
Lopez-Paz, D., Ranzato, M.A.: Gradient episodic memory for continual learning. In: Advances in Neural Information Processing Systems 30, pp. 6467–6476 (2017)
McCloskey, M., Cohen, N.J.: Catastrophic interference in connectionist networks: the sequential learning problem. Psychol. Learn. Motiv. 24, 109–165 (1989)
Nguyen, T.T., Liew, A.W.C., Tran, M.T., Pham, X.C., Nguyen, M.P.: A novel genetic algorithm approach for simultaneous feature and classifier selection in multi classifier system. In: IEEE Congress on Evolutionary Computation (CEC), pp. 1698–1705 (2014)
Nguyen, T.T., Nguyen, T.T.T., Pham, X.C., Liew, A.W.C.: A novel combining classifier method based on variational inference. Pattern Recogn. 49, 198–212 (2016)
Nguyen, T.T., Pham, X.C., Liew, A.W.C., Pedrycz, W.: Aggregation of classifiers: a justifiable information granularity approach. IEEE Trans. Cybern. 49(6), 2168–2177 (2019)
Nguyen, T.T.T., et al.: A novel online bayes classifier. In: 2016 International Conference on Digital Image Computing: Techniques and Applications (DICTA), pp. 1–6 (2016)
Oza, N., Russell, S.: Online bagging and boosting. In: Proceedings of the International Conference on Systems, Man and Cybernetics, pp. 2340–2345 (2005)
Pham, X.C., Dang, M.T., Dinh, V.S., Hoang, S., Nguyen, T.T., Liew, A.W.C.: Learning from data stream based on random projection and hoeffding tree classifier. In: 2017 International Conference on Digital Image Computing: Techniques and Applications, DICTA 2017, Sydney, Australia, pp. 1–8 (2017)
Riemer, M., et al.: Learning to learn without forgetting by maximizing transfer and minimizing interference. In: International Conference on Learning Representations (ICLR) (2019)
Riemer, M., Franceschini, M., Klinger, T.: Generation and consolidation of recollections for efficient deep lifelong learning (2017). http://arxiv.org/abs/1711.06761
Rosenblatt, F.: The perceptron: a probabilistic model for information storage and organization in the brain. Psychol. Rev. 65(6), 386–408 (1958)
Sokolova, M., Lapalme, G.: A systematic analysis of performance measures for classification tasks. Inf. Process. Manag. 45, 427–437 (2009)
Wang, J., Zhao, P., Hoi, S.C.H.: Exact soft confidence-weighted learning. In: Proceedings of ICML (2012)
Zinkevich, M.: Online convex programming and generalized infinitesimal gradient ascent. In: Proceedings of the ICML, pp. 928–936 (2003)
Acknowledgment
This research was supported by the Griffith University International Postgraduate Research Scholarship (GUIPRS).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Pham, X.C., Nguyen, T.T.T., Liew, A.WC. (2019). A Novel Online Ensemble Convolutional Neural Networks for Streaming Data. In: Gedeon, T., Wong, K., Lee, M. (eds) Neural Information Processing. ICONIP 2019. Lecture Notes in Computer Science(), vol 11953. Springer, Cham. https://doi.org/10.1007/978-3-030-36708-4_17
Download citation
DOI: https://doi.org/10.1007/978-3-030-36708-4_17
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-36707-7
Online ISBN: 978-3-030-36708-4
eBook Packages: Computer ScienceComputer Science (R0)