Abstract
Most conventional literature on online learning implicitly assumes a static feature space. However, in real-world applications, the feature space may vary over time due to the emergence of new features and the vanishing of outdated features. This phenomenon is referred to as online learning with Varying Feature Space (VFS). Recently, there has been increasing attention towards exploring this online learning paradigm. However, none of the existing approaches have taken into account the potentially informative information conveyed by the presence or absence (i.e., variation in this paper) of each feature. This indicates that the existence of certain features in the VFS can be correlated with the class labels. If properly utilized for the learning process, such information can potentially enhance predictive performance. To this end, we formally define and present a learning framework to address this specific learning scenario, which we refer to as Online learning in Varying Feature space with Informative Variation (abbreviated as OVFIV). The framework aims to answer two key questions: how to learn a model that captures the association between the existence of features and the class labels, and how to incorporate this information into the prediction process to improve performance. The validity of our proposed method is verified through theoretical analyses and empirical studies conducted on 17 datasets from diverse fields.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Alagurajah, J., Yuan, X., Wu, X.: Scale invariant learning from trapezoidal data streams. In: Proceedings of the 35th Annual ACM Symposium on Applied Computing, pp. 505–508 (2020)
Asuncion, A., Newman, D.: UCI machine learning repository (2007)
Beyazit, E., Alagurajah, J., Wu, X.: Online learning from data streams with varying feature spaces. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 3232–3239 (2019)
Cesa-Bianchi, N., Lugosi, G.: Prediction, Learning, and Games. Cambridge University Press, Cambridge (2006)
Demšar, J.: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7, 1–30 (2006)
Gu, S., Qian, Y., Hou, C.: Incremental feature spaces learning with label scarcity. ACM Trans. Knowl. Discovery Data (TKDD) 16(6), 1–26 (2022)
Gu, S., Qian, Y., Hou, C.: Learning with incremental instances and features. IEEE Trans. Neural Networks Learn. Syst. (2023)
He, Y., Dong, J., Hou, B.J., Wang, Y., Wang, F.: Online learning in variable feature spaces with mixed data. In: 2021 IEEE International Conference on Data Mining (ICDM), pp. 181–190. IEEE (2021)
He, Y., Wu, B., Wu, D., Beyazit, E., Chen, S., Wu, X.: Toward mining capricious data streams: a generative approach. IEEE Trans. Neural Networks Learn. Syst. 32(3), 1228–1240 (2020)
He, Y., Yuan, X., Chen, S., Wu, X.: Online learning in variable feature spaces under incomplete supervision. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 4106–4114 (2021)
Hou, B.J., Zhang, L., Zhou, Z.H.: Learning with feature evolvable streams. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Hou, B.J., Zhang, L., Zhou, Z.H.: Prediction with unpredictable feature evolution. IEEE Trans. Neural Networks Learn. Syst. 33(10), 5706–5715 (2021)
Huynh, N.A., Ng, W.K., Ariyapala, K.: Learning under concept drift with follow the regularized leader and adaptive decaying proximal. Expert Syst. Appl. 96, 49–63 (2018)
Lachin, J.M.: Worst-rank score analysis with informatively missing observations in clinical trials. Control. Clin. Trials 20(5), 408–422 (1999)
Lian, H., Atwood, J.S., Hou, B.J., Wu, J., He, Y.: Online deep learning from doubly-streaming data. In: Proceedings of the 30th ACM International Conference on Multimedia, pp. 3185–3194 (2022)
Liu, N., Beerman, I., Lifton, R., Zhao, H.: Haplotype analysis in the presence of informatively missing genotype data. Genet. Epidemiol.: The Official Publication of the International Genetic Epidemiology Society 30(4), 290–300 (2006)
Liu, Y., Fan, X., Li, W., Gao, Y.: Online passive-aggressive active learning for trapezoidal data streams. IEEE Trans. Neural Networks Learn. Syst. 34, 6725–6739 (2022)
McMahan, H.B., et al.: Ad click prediction: a view from the trenches. In: Proceedings of the 19th ACM SIGKDD international conference on Knowledge Discovery and Data Mining, pp. 1222–1230 (2013)
Murphy, K.P.: Machine Learning: A Probabilistic Perspective. MIT Press, Cambridge (2012)
Nguyen, T.T., Chang, K., Hui, S.C.: Two-view online learning. In: Tan, P.-N., Chawla, S., Ho, C.K., Bailey, J. (eds.) PAKDD 2012, Part I. LNCS (LNAI), vol. 7301, pp. 74–85. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-30217-6_7
Oza, N.C., Russell, S.J.: Online bagging and boosting. In: International Workshop on Artificial Intelligence and Statistics, pp. 229–236. PMLR (2001)
Schreckenberger, C., Glockner, T., Stuckenschmidt, H., Bartelt, C.: Restructuring of hoeffding trees for trapezoidal data streams. In: 2020 International Conference on Data Mining Workshops (ICDMW), pp. 416–423. IEEE (2020)
Schreckenberger, C., He, Y., Lüdtke, S., Bartelt, C., Stuckenschmidt, H.: Online random feature forests for learning in varying feature spaces. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, pp. 4587–4595 (2023)
Shih, W.J.: Problems in dealing with missing data and informative censoring in clinical trials. Curr. Control. Trials Cardiovasc. Med. 3(1), 1–7 (2002)
Singer, Y., Duchi, J.C.: Efficient learning using forward-backward splitting. In: Advances in Neural Information Processing Systems, vol. 22 (2009)
Tian, Y., Zhang, Y.: A comprehensive survey on regularization strategies in machine learning. Inf. Fusion 80, 146–166 (2022)
Xiao, L.: Dual averaging method for regularized stochastic learning and online optimization. In: Advances in Neural Information Processing Systems, vol. 22 (2009)
Yao, Z.J., Bi, J., Chen, Y.X.: Applying deep learning to individual and community health monitoring data: a survey. Int. J. Autom. Comput. 15, 643–655 (2018)
You, D., et al.: Online learning from incomplete and imbalanced data streams. IEEE Trans. Knowl. Data Eng. 35, 10650–10665 (2023)
Zhang, Q., Zhang, P., Long, G., Ding, W., Zhang, C., Wu, X.: Online learning from trapezoidal data streams. IEEE Trans. Knowl. Data Eng. 28(10), 2709–2723 (2016)
Acknowledgments
This work was supported by National Natural Science Foundation of China (NSFC) under Grant Nos. 62002148 and 62250710682, Guangdong Provincial Key Laboratory under Grant No. 2020B121201001, and Research Institute of Trustworthy Autonomous Systems (RITAS).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 IFIP International Federation for Information Processing
About this paper
Cite this paper
Qin, P., Song, L. (2024). Online Learning in Varying Feature Spaces with Informative Variation. In: Shi, Z., Torresen, J., Yang, S. (eds) Intelligent Information Processing XII. IIP 2024. IFIP Advances in Information and Communication Technology, vol 703. Springer, Cham. https://doi.org/10.1007/978-3-031-57808-3_2
Download citation
DOI: https://doi.org/10.1007/978-3-031-57808-3_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-57807-6
Online ISBN: 978-3-031-57808-3
eBook Packages: Computer ScienceComputer Science (R0)
