Abstract
Correctly detecting the position where a concept begins to drift is important in mining data streams. In this paper, we propose a new method for detecting concept drift. The proposed method, which can detect different types of drift, is based on processing data chunk by chunk and measuring differences between two consecutive batches, as drift indicator. In order to evaluate the proposed method we measure its performance on a set of artificial datasets with different levels of severity and speed of drift. The experimental results show that the proposed method is capable to detect drifts and can approximately find concept drift locations.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Markou, M., Singh, S.: Novelty detection: A review, Part I: Statistical approaches. Signal Processing 83(12), 2481–2521 (2003)
Kuncheva, L.I.: Classifier ensembles for detecting concept change in streaming data: Overview and perspectives. In: Proc. of the Second Workshop SUEMA, Patras, Greece, pp. 5–9 (2008)
Gama, J., Medas, P., Castillo, G., Rodrigues, P.: Learning with drift detection. In: Bazzan, A.L.C., Labidi, S. (eds.) SBIA 2004. LNCS (LNAI), vol. 3171, pp. 286–295. Springer, Heidelberg (2004)
Baena-Garcia, M., Del Campo-Avila, J., Fidalgo, R., Bifet, A.: Early drift detection method. In: Proc. of the 4th ECML PKDD International Workshop on Knowledge Discovery from Data Streams, Berlin, Germany, pp. 77–86 (2006)
Wilson, D.R., Martinez, T.R.: Improved heterogeneous distance functions. Journal of Artificial Intelligence Research 6(1), 1–34 (1997)
Nishida, K., Yamauchi, K.: Detecting concept drift using statistical testing. In: Corruble, V., Takeda, M., Suzuki, E. (eds.) DS 2007. LNCS (LNAI), vol. 4755, pp. 264–269. Springer, Heidelberg (2007)
Minku, F.L., White, A., Yao, X.: The impact of diversity on on-line ensemble learning in the presence of concept drift. In: IEEE TKDE, vol. 22, pp. 730–742 (2010)
Salganicoff, M.: Density-adaptive learning and forgetting. In: Proc. of the 10th Int. Conf. on Machine Learning, pp. 276–283 (1993)
Klinkenberg, R., Joachims, T.: Detecting concept drift with support vector machines. In: Proc. of the 17th Int. Conf. on Machine Learning, pp. 487–494 (2000)
Dries, A., Ruckert, U.: Adaptive concept drift detection. In: SDM, pp. 233–244. SIAM, Philadelphia (2009)
Gama, J.: Knowledge discovery from data streams. Data Mining and Knowledge Discovery Series. USA (2010)
Mitchell, T.M.: Machine Learning. McGraw-Hill, New York (1997)
Chu, F., Zaniolo, C.: Fast and light boosting for adaptive mining of data streams. In: Dai, H., Srikant, R., Zhang, C. (eds.) PAKDD 2004. LNCS (LNAI), vol. 3056, pp. 282–292. Springer, Heidelberg (2004)
Friedman, J.H., Rafsky, L.C.: Multivariate generalizations of the Wald-Wolfowitz and Smirnov two-sample tests. Annals of Statistics 7(4), 697–717 (1979)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Sobhani, P., Beigy, H. (2011). New Drift Detection Method for Data Streams. In: Bouchachia, A. (eds) Adaptive and Intelligent Systems. ICAIS 2011. Lecture Notes in Computer Science(), vol 6943. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-23857-4_12
Download citation
DOI: https://doi.org/10.1007/978-3-642-23857-4_12
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-23856-7
Online ISBN: 978-3-642-23857-4
eBook Packages: Computer ScienceComputer Science (R0)