Abstract
In streaming data applications, the underlying concept often changes with time which necessitates the update of employed classifiers. Most approaches in the literature utilize the arriving labeled data to continually update the classifier system. However, it is often difficult/expensive to continuously receive the labels for the arriving data. Moreover, in domains such as embedded sensing, resource-aware classifiers that do not update frequently are needed. To tackle these issues, recent works have proposed to predict classifiers at a future time instance by additionally learning the dynamics of changing classifier weights during the initial training phase. This strategy bypasses the need to retrain/relearn the classifiers, and thus the additional labeled data is no longer required. However, the current progress is limited to the prediction of linear classifiers. As a step forward, in this work, we propose a probabilistic model for predicting future non-linear classifiers given time-stamped labeled data. We develop a variational inference based learning algorithm and demonstrate the effectiveness of our approach through experiments using synthetic and real-world datasets.
D.Dhaka—Contributed while working at NEC Corporation.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abdallah, Z.S., Gaber, M.M., Srinivasan, B., Krishnaswamy, S.: Adaptive mobile activity recognition system with evolving data streams. Neurocomputing 150, 304–317 (2015)
Alippi, C., Liu, D., Zhao, D., Bu, L.: Detecting and reacting to changes in sensing units: the active classifier case. IEEE Trans. Syst. Man Cybern. Syst. 44(3), 353–362 (2013)
Angelov, P., Zhou, X.: Evolving fuzzy systems from data streams in real-time. In: 2006 International Symposium on Evolving Fuzzy Systems, pp. 29–35. IEEE (2006)
Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, Boston (2006). https://doi.org/10.1007/978-1-4615-7566-5
Brzezinski, D., Stefanowski, J.: Reacting to different types of concept drift: the accuracy updated ensemble algorithm. IEEE Trans. Neural Netw. Learn. Syst. 25(1), 81–94 (2014)
Crammer, K., Even-Dar, E., Mansour, Y., Vaughan, J.W.: Regret minimization with concept drift. In: Proceedings of the 23rd Annual Conference on Learning Theory (COLT) (2010)
Ditzler, G.: Incremental learning of concept drift from imbalanced data (2011)
Ferguson, T.S.: A bayesian analysis of some nonparametric problems. The annals of statistics pp. 209–230 (1973)
Fernandes, K., Vinagre, P., Cortez, P.: A proactive intelligent decision support system for predicting the popularity of online news. In: Pereira, F., Machado, P., Costa, E., Cardoso, A. (eds.) EPIA 2015. LNCS (LNAI), vol. 9273, pp. 535–546. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-23485-4_53
Gama, J., Žliobaitė, I., Bifet, A., Pechenizkiy, M., Bouchachia, A.: A survey on concept drift adaptation. ACM Comput. Surv. (CSUR) 46(4), 44 (2014)
Griffiths, T.L., Jordan, M.I., Tenenbaum, J.B., Blei, D.M.: Hierarchical topic models and the nested Chinese restaurant process. In: Advances in Neural Information Processing Systems, pp. 17–24 (2004)
Hannah, L.A., Blei, D.M., Powell, W.B.: Dirichlet process mixtures of generalized linear models. J. Mach. Learn. Res. 12, 1923–1953 (2011)
Haque, A., Khan, L., Baron, M.: Sand: semi-supervised adaptive novel class detection and classification over data stream. In: THIRTIETH AAAI Conference on Artificial Intelligence (2016)
Haque, A., Khan, L., Baron, M., Thuraisingham, B., Aggarwal, C.: Efficient handling of concept drift and concept evolution over stream data. In: 2016 IEEE 32nd International Conference on Data Engineering (ICDE), pp. 481–492. IEEE (2016)
Hofer, V.: Adapting a classification rule to local and global shift when only unlabelled data are available. Eur. J. Oper. Res. 243(1), 177–189 (2015)
Jaakkola, T.S., Jordan, M.I.: Bayesian parameter estimation via variational methods. Stat. Comput. 10(1), 25–37 (2000)
Klinkenberg, R.: Learning drifting concepts: example selection vs. example weighting. Intell. Data Anal. 8(3), 281–300 (2004)
Kolter, J.Z., Maloof, M.A.: Using additive expert ensembles to cope with concept drift. In: Proceedings of the 22nd International Conference on Machine Learning, pp. 449–456. ACM (2005)
Koychev, I.: Gradual forgetting for adaptation to concept drift. In: Proceedings of ECAI 2000 Workshop on Current Issues in Spatio-Temporal ... (2000)
Kumagai, A., Iwata, T.: Learning future classifiers without additional data. In: Thirtieth AAAI Conference on Artificial Intelligence (2016)
Kumagai, A., Iwata, T.: Learning non-linear dynamics of decision boundaries for maintaining classification performance. In: Thirty-First AAAI Conference on Artificial Intelligence (2017)
Kumagai, A., Iwata, T.: Learning dynamics of decision boundaries without additional labeled data. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1627–1636. ACM (2018)
Lee, K.H., Verma, N.: A low-power processor with configurable embedded machine-learning accelerators for high-order and adaptive analysis of medical-sensor signals. IEEE J. Solid-State Circuits 48(7), 1625–1637 (2013)
Lindstrom, P., Delany, S.J., Mac Namee, B.: Handling concept drift in a text data stream constrained by high labelling cost. In: Twenty-Third International FLAIRS Conference (2010)
Lorena, A.C., Garcia, L.P., Lehmann, J., Souto, M.C., Ho, T.K.: How complex is your classification problem? A survey on measuring classification complexity. ACM Comput. Surv. (CSUR) 52(5), 1–34 (2019)
Lütkepohl, H.: Vector Autoregressive Models. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-04898-2_609
MartÃnez-Rego, D., Pérez-Sánchez, B., Fontenla-Romero, O., Alonso-Betanzos, A.: A robust incremental learning method for non-stationary environments. Neurocomputing 74(11), 1800–1808 (2011)
Sethuraman, J.: A constructive definition of Dirichlet priors. Statistica Sinica, 639–650 (1994)
Shahbaba, B., Neal, R.: Nonlinear models using Dirichlet process mixtures. J. Mach. Learn. Res. 10, 1829–1850 (2009)
Škrjanc, I., Iglesias, J.A., Sanchis, A., Leite, D., Lughofer, E., Gomide, F.: Evolving fuzzy and neuro-fuzzy approaches in clustering, regression, identification, and classification: a survey. Inf. Sci. 490, 344–368 (2019)
Teh, Y.W., Jordan, M.I., Beal, M.J., Blei, D.M.: Sharing clusters among related groups: Hierarchical Dirichlet processes. In: Advances in Neural Information Processing Systems, pp. 1385–1392 (2005)
Wang, C., Paisley, J., Blei, D.: Online variational inference for the hierarchical Dirichlet process. In: Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, pp. 752–760 (2011)
Wang, J., Zhao, P., Hoi, S.C.H.: Exact soft confidence-weighted learning. In: Proceedings of the 29th International Conference on Machine Learning, pp. 121–128 (2012)
Yang, X., Dinh, A., Chen, L.: Implementation of a wearerable real-time system for physical activity recognition based on Naive Bayes classifier. In: 2010 International Conference on Bioinformatics and Biomedical Technology, pp. 101–105. IEEE (2010)
Yuan, C., Neubauer, C.: Variational mixture of gaussian process experts. In: Advances in Neural Information Processing Systems, pp. 1897–1904 (2009)
Zhu, X., Zhang, P., Lin, X., Shi, Y.: Active learning from stream data using optimal weight classifier ensemble. IEEE Trans. Syst. Man Cybern. Part B (Cybern.) 40(6), 1607–1621 (2010)
Žliobaitė, I., Bifet, A., Pfahringer, B., Holmes, G.: Active learning with drifting streaming data. IEEE Trans. Neural Netw. Learn. Syst. 25(1), 27–39 (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Khandelwal, K., Dhaka, D., Barsopia, V. (2021). Predicting Future Classifiers for Evolving Non-linear Decision Boundaries. In: Hutter, F., Kersting, K., Lijffijt, J., Valera, I. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2020. Lecture Notes in Computer Science(), vol 12457. Springer, Cham. https://doi.org/10.1007/978-3-030-67658-2_36
Download citation
DOI: https://doi.org/10.1007/978-3-030-67658-2_36
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-67657-5
Online ISBN: 978-3-030-67658-2
eBook Packages: Computer ScienceComputer Science (R0)