Skip to main content
SpringerLink
Log in

Online Learning in Varying Feature Spaces with Informative Variation

  • Conference paper
  • First Online:
Intelligent Information Processing XII (IIP 2024)

Part of the book series: IFIP Advances in Information and Communication Technology ((IFIPAICT,volume 703))

Included in the following conference series:

  • 40 Accesses

Abstract

Most conventional literature on online learning implicitly assumes a static feature space. However, in real-world applications, the feature space may vary over time due to the emergence of new features and the vanishing of outdated features. This phenomenon is referred to as online learning with Varying Feature Space (VFS). Recently, there has been increasing attention towards exploring this online learning paradigm. However, none of the existing approaches have taken into account the potentially informative information conveyed by the presence or absence (i.e., variation in this paper) of each feature. This indicates that the existence of certain features in the VFS can be correlated with the class labels. If properly utilized for the learning process, such information can potentially enhance predictive performance. To this end, we formally define and present a learning framework to address this specific learning scenario, which we refer to as Online learning in Varying Feature space with Informative Variation (abbreviated as OVFIV). The framework aims to answer two key questions: how to learn a model that captures the association between the existence of features and the class labels, and how to incorporate this information into the prediction process to improve performance. The validity of our proposed method is verified through theoretical analyses and empirical studies conducted on 17 datasets from diverse fields.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

References

  1. Alagurajah, J., Yuan, X., Wu, X.: Scale invariant learning from trapezoidal data streams. In: Proceedings of the 35th Annual ACM Symposium on Applied Computing, pp. 505–508 (2020)

    Google Scholar 

  2. Asuncion, A., Newman, D.: UCI machine learning repository (2007)

    Google Scholar 

  3. Beyazit, E., Alagurajah, J., Wu, X.: Online learning from data streams with varying feature spaces. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 3232–3239 (2019)

    Google Scholar 

  4. Cesa-Bianchi, N., Lugosi, G.: Prediction, Learning, and Games. Cambridge University Press, Cambridge (2006)

    Book  Google Scholar 

  5. Demšar, J.: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7, 1–30 (2006)

    MathSciNet  Google Scholar 

  6. Gu, S., Qian, Y., Hou, C.: Incremental feature spaces learning with label scarcity. ACM Trans. Knowl. Discovery Data (TKDD) 16(6), 1–26 (2022)

    Article  Google Scholar 

  7. Gu, S., Qian, Y., Hou, C.: Learning with incremental instances and features. IEEE Trans. Neural Networks Learn. Syst. (2023)

    Google Scholar 

  8. He, Y., Dong, J., Hou, B.J., Wang, Y., Wang, F.: Online learning in variable feature spaces with mixed data. In: 2021 IEEE International Conference on Data Mining (ICDM), pp. 181–190. IEEE (2021)

    Google Scholar 

  9. He, Y., Wu, B., Wu, D., Beyazit, E., Chen, S., Wu, X.: Toward mining capricious data streams: a generative approach. IEEE Trans. Neural Networks Learn. Syst. 32(3), 1228–1240 (2020)

    Article  MathSciNet  Google Scholar 

  10. He, Y., Yuan, X., Chen, S., Wu, X.: Online learning in variable feature spaces under incomplete supervision. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 4106–4114 (2021)

    Google Scholar 

  11. Hou, B.J., Zhang, L., Zhou, Z.H.: Learning with feature evolvable streams. In: Advances in Neural Information Processing Systems, vol. 30 (2017)

    Google Scholar 

  12. Hou, B.J., Zhang, L., Zhou, Z.H.: Prediction with unpredictable feature evolution. IEEE Trans. Neural Networks Learn. Syst. 33(10), 5706–5715 (2021)

    Article  MathSciNet  Google Scholar 

  13. Huynh, N.A., Ng, W.K., Ariyapala, K.: Learning under concept drift with follow the regularized leader and adaptive decaying proximal. Expert Syst. Appl. 96, 49–63 (2018)

    Article  Google Scholar 

  14. Lachin, J.M.: Worst-rank score analysis with informatively missing observations in clinical trials. Control. Clin. Trials 20(5), 408–422 (1999)

    Article  Google Scholar 

  15. Lian, H., Atwood, J.S., Hou, B.J., Wu, J., He, Y.: Online deep learning from doubly-streaming data. In: Proceedings of the 30th ACM International Conference on Multimedia, pp. 3185–3194 (2022)

    Google Scholar 

  16. Liu, N., Beerman, I., Lifton, R., Zhao, H.: Haplotype analysis in the presence of informatively missing genotype data. Genet. Epidemiol.: The Official Publication of the International Genetic Epidemiology Society 30(4), 290–300 (2006)

    Article  Google Scholar 

  17. Liu, Y., Fan, X., Li, W., Gao, Y.: Online passive-aggressive active learning for trapezoidal data streams. IEEE Trans. Neural Networks Learn. Syst. 34, 6725–6739 (2022)

    Article  MathSciNet  Google Scholar 

  18. McMahan, H.B., et al.: Ad click prediction: a view from the trenches. In: Proceedings of the 19th ACM SIGKDD international conference on Knowledge Discovery and Data Mining, pp. 1222–1230 (2013)

    Google Scholar 

  19. Murphy, K.P.: Machine Learning: A Probabilistic Perspective. MIT Press, Cambridge (2012)

    Google Scholar 

  20. Nguyen, T.T., Chang, K., Hui, S.C.: Two-view online learning. In: Tan, P.-N., Chawla, S., Ho, C.K., Bailey, J. (eds.) PAKDD 2012, Part I. LNCS (LNAI), vol. 7301, pp. 74–85. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-30217-6_7

    Chapter  Google Scholar 

  21. Oza, N.C., Russell, S.J.: Online bagging and boosting. In: International Workshop on Artificial Intelligence and Statistics, pp. 229–236. PMLR (2001)

    Google Scholar 

  22. Schreckenberger, C., Glockner, T., Stuckenschmidt, H., Bartelt, C.: Restructuring of hoeffding trees for trapezoidal data streams. In: 2020 International Conference on Data Mining Workshops (ICDMW), pp. 416–423. IEEE (2020)

    Google Scholar 

  23. Schreckenberger, C., He, Y., Lüdtke, S., Bartelt, C., Stuckenschmidt, H.: Online random feature forests for learning in varying feature spaces. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, pp. 4587–4595 (2023)

    Google Scholar 

  24. Shih, W.J.: Problems in dealing with missing data and informative censoring in clinical trials. Curr. Control. Trials Cardiovasc. Med. 3(1), 1–7 (2002)

    Article  Google Scholar 

  25. Singer, Y., Duchi, J.C.: Efficient learning using forward-backward splitting. In: Advances in Neural Information Processing Systems, vol. 22 (2009)

    Google Scholar 

  26. Tian, Y., Zhang, Y.: A comprehensive survey on regularization strategies in machine learning. Inf. Fusion 80, 146–166 (2022)

    Article  Google Scholar 

  27. Xiao, L.: Dual averaging method for regularized stochastic learning and online optimization. In: Advances in Neural Information Processing Systems, vol. 22 (2009)

    Google Scholar 

  28. Yao, Z.J., Bi, J., Chen, Y.X.: Applying deep learning to individual and community health monitoring data: a survey. Int. J. Autom. Comput. 15, 643–655 (2018)

    Article  Google Scholar 

  29. You, D., et al.: Online learning from incomplete and imbalanced data streams. IEEE Trans. Knowl. Data Eng. 35, 10650–10665 (2023)

    Article  Google Scholar 

  30. Zhang, Q., Zhang, P., Long, G., Ding, W., Zhang, C., Wu, X.: Online learning from trapezoidal data streams. IEEE Trans. Knowl. Data Eng. 28(10), 2709–2723 (2016)

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported by National Natural Science Foundation of China (NSFC) under Grant Nos. 62002148 and 62250710682, Guangdong Provincial Key Laboratory under Grant No. 2020B121201001, and Research Institute of Trustworthy Autonomous Systems (RITAS).

Author information

Authors and Affiliations

  1. Southern University of Science and Technology, Shenzhen, China

    Peijia Qin & Liyan Song

  2. Faculty of Computing, Harbin Institute of Technology, Harbin, China

    Liyan Song

Authors
  1. Peijia Qin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Liyan Song
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Liyan Song .

Editor information

Editors and Affiliations

  1. Chinese Academy of Sciences, Beijing, China

    Zhongzhi Shi

  2. University of Oslo, Oslo, Norway

    Jim Torresen

  3. De Montfort University, Leicester, UK

    Shengxiang Yang

Rights and permissions

Reprints and permissions

Copyright information

© 2024 IFIP International Federation for Information Processing

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Qin, P., Song, L. (2024). Online Learning in Varying Feature Spaces with Informative Variation. In: Shi, Z., Torresen, J., Yang, S. (eds) Intelligent Information Processing XII. IIP 2024. IFIP Advances in Information and Communication Technology, vol 703. Springer, Cham. https://doi.org/10.1007/978-3-031-57808-3_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-57808-3_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-57807-6

  • Online ISBN: 978-3-031-57808-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation