Skip to main content
Springer Nature Link
Log in

Unsupervised Virtual Drift Detection Method in Streaming Environment

  • Conference paper
  • First Online:
Computer Vision and Machine Intelligence

Part of the book series: Lecture Notes in Networks and Systems ((LNNS,volume 586))

  • 820 Accesses

Abstract

Real-time applications generate an enormous amount of data that can potentially change data distribution. The underline change in data distribution concerning time causes concept drift. The learning model of the data stream encounters concept drift problems while predicting the patterns. It leads to deterioration in the learning model’s performance. Additional challenges of high-dimensional data create memory and time requirements. The proposed work develops an unsupervised concept drift detection method to detect virtual drift in non-stationary data. The K-means clustering algorithm is applied to the relevant features to find the stream’s virtual drift. The proposed work reduces the complexity by detecting the drifts using the k highest score features suitable with high-dimensional data. Here, we analyze the data stream’s virtual drift by considering the changes in data distribution of recent and current window data instances.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Agrahari, S., Singh, A.K.: Concept drift detection in data stream mining: A literature review. J. King Saud Univer. Comput. Inf. Sci. (2021). ISSN 1319-1578. https://doi.org/10.1016/j.jksuci.2021.11.006. URL https://www.sciencedirect.com/science/article/pii/S1319157821003062

  2. Agrahari, Supriya, Singh, Anil Kumar: Disposition-based concept drift detection and adaptation in data stream. Arab. J. Sci. Eng. 47(8), 10605–10621 (2022). https://doi.org/10.1007/s13369-022-06653-4

    Article  Google Scholar 

  3. Souza, V., Parmezan, A.R.S., Chowdhury, F.A., Mueen, A.: Efficient unsupervised drift detector for fast and high-dimensional data streams. Knowl. Inf. Syst. 63(6), 1497–1527 (2021)

    Google Scholar 

  4. Xuan, Junyu, Jie, Lu., Zhang, Guangquan: Bayesian nonparametric unsupervised concept drift detection for data stream mining. ACM Trans. Intell. Syst. Technol. (TIST) 12(1), 1–22 (2020)

    Google Scholar 

  5. Huang, H., Yoo, S., Kasiviswanathan, S.P.: Unsupervised feature selection on data streams. In: Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, pp. 1031–1040 (2015)

    Google Scholar 

  6. de Mello, R.F., Vaz, Y., Grossi, C.H., Bifet, A.: On learning guarantees to unsupervised concept drift detection on data streams. Expert Syst. Appl. 117, 90–102 (2019)

    Google Scholar 

  7. Wiwatcharakoses, C., Berrar, D.: Soinn+, a self-organizing incremental neural network for unsupervised learning from noisy data streams. Expert Syst. Appl. 143, 113069 (2020)

    Google Scholar 

  8. Gözüaçık, Ömer., Can, Fazli: Concept learning using one-class classifiers for implicit drift detection in evolving data streams. Artif. Intell. Rev. 54(5), 3725–3747 (2021)

    Article  Google Scholar 

  9. Pinto, F., Sampaio, M.O.P., Bizarro, P.: Automatic model monitoring for data streams. arXiv preprint arXiv:1908.04240 (2019)

    Google Scholar 

  10. Gama, J., Medas, P., Castillo, G., Rodrigues, P.: Learning with drift detection. In: Brazilian Symposium on Artificial Intelligence, pp. 286–295. Springer, Berlin (2004)

    Google Scholar 

  11. Bifet, Albert: Adaptive learning and mining for data streams and frequent patterns. ACM SIGKDD Explor. Newsl. 11(1), 55–56 (2009)

    Article  Google Scholar 

  12. Ross, G.J., Adams, N.M., Tasoulis, D.K., Hand, D.J.: Exponentially weighted moving average charts for detecting concept drift. Pattern Recogn. Lett. 33(2), 191–198 (2012)

    Google Scholar 

  13. Huang, D.T.J., Koh, Y.S., Dobbie, G., Pears, R.: Detecting volatility shift in data streams. In: 2014 IEEE International Conference on Data Mining, pp. 863–868 (2014). https://doi.org/10.1109/ICDM.2014.50

  14. Pears, R., Sakthithasan, S., Koh, Y.S.: Detecting concept change in dynamic data streams. Mach. Learn. 97(3), 259–293 (2014)

    Google Scholar 

  15. Nishida, K., Yamauchi, K.: Detecting concept drift using statistical testing. In: Corruble, V., Takeda, M., Suzuki, E. (eds.) Discovery Science, pp. 264–269, Springer, Berlin (2007). ISBN 978-3-540-75488-6

    Google Scholar 

  16. de Barros, R.S.M., Hidalgo, J.I.G., de Lima Cabral, D.R.: Wilcoxon rank sum test drift detector. Neurocomputing 275, 1954–1963 (2018)

    Google Scholar 

  17. Franke, T.M., Ho, T., Christie, C.A.: The chi-square test: Often used and more often misinterpreted. Am. J. Eval. 33(3), 448–458 (2012)

    Google Scholar 

  18. Gama, João., Žliobaitė, Indrė, Bifet, Albert, Pechenizkiy, Mykola, Bouchachia, Abdelhamid: A survey on concept drift adaptation. ACM Comput. Surv. (CSUR) 46(4), 1–37 (2014)

    Article  MATH  Google Scholar 

  19. Demšar, Janez: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7, 1–30 (2006)

    MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Motilal Nehru National Institute of Technology Allahabad, Prayagraj, India

    Supriya Agrahari & Anil Kumar Singh

Authors
  1. Supriya Agrahari
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anil Kumar Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Supriya Agrahari .

Editor information

Editors and Affiliations

  1. Computer Vision Laboratory, University of Sassari, Alghero, Sassari, Italy

    Massimo Tistarelli

  2. Computer Vision and Biometrics Lab, Department of Information Technology, Indian Institute of Information Technology Allahabad, Prayagraj, India

    Shiv Ram Dubey

  3. Computer Vision and Biometrics Lab, Department of Information Technology, Indian Institute of Information Technology, Allahabad, India

    Satish Kumar Singh

  4. University of Münster, Münster, Germany

    Xiaoyi Jiang

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Agrahari, S., Singh, A.K. (2023). Unsupervised Virtual Drift Detection Method in Streaming Environment. In: Tistarelli, M., Dubey, S.R., Singh, S.K., Jiang, X. (eds) Computer Vision and Machine Intelligence. Lecture Notes in Networks and Systems, vol 586. Springer, Singapore. https://doi.org/10.1007/978-981-19-7867-8_25

Download citation

Publish with us

Policies and ethics