Skip to main content
SpringerLink
Log in

Adaptive cascade of boosted ensembles for face detection in concept drift

  • ICONIP2010
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

We propose an adaptive learning algorithm for cascades of boosted ensembles that is designed to handle the problem of concept drift in nonstationary environments. The goal was to create a real-time adaptive algorithm for dynamic environments that exhibit varying degrees of drift in high-volume streaming data. This we achieved using a hybrid of detect-and-retrain and constant-update approaches. The uniqueness of our method is found in two aspects of our framework. The first is the manner in which individual weak classifiers within each cascade layer of an ensemble are clustered during training and assigned a competence value. Secondly, the idea of learning optimal cascade-layer thresholds during runtime, which enables rapid adaptation to dynamic environments. The proposed adaptive learning method was applied to a binary-class problem with rare-event detection characteristics. For this, we chose the domain of face detection and demonstrate experimentally the ability of our algorithm to achieve an effective trade-off between accuracy and speed of adaptations in dense data streams with unknown rates of change.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Algorithm 1
Fig. 3
Algorithm 2
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. http://www.frvt.org/FERET/default.htm

References

  1. Muhlbaier M, Polikar R (2007) Multiple classifiers based incremental learning algorithm for learning in nonstationary environments. In: 2007 international conference on machine learning and cybernetics, vol 6. pp 3618–3623

  2. Wang P, Wang H, Wu X, Wang W, Shi B (2007) A low-granularity classifier for data streams with concept drifts and biased class distribution. IEEE Trans Knowl Data Eng 19:1202–1213

    Google Scholar 

  3. May M, Berendt B, Cornuejols A, Gama J, Giannotti F, Hotho A, Malerba D, Menasalvas E, Morik K, Pedersen R et al (2008) Research challenges in ubiquitous knowledge discovery. Chapman & Hall/CRC Press, London

    Google Scholar 

  4. May M, Saitta L (2010) Introduction: the challenge of ubiquitous knowledge discovery. In: May M, Saitta L (eds) Ubiquitous knowledge discovery. Volume 6202 of lecture notes in computer science. Springer, Berlin, pp 3–18

    Chapter  Google Scholar 

  5. Widmer G, Kubat M (1996) Learning in the presence of concept drift and hidden contexts. Mach Learn 23:69–101

    Google Scholar 

  6. Schlimmer J, Granger R (1986) Incremental learning from noisy data. Mach Learn 1(3):317–354

    Google Scholar 

  7. Kuncheva LI (2004) Classifier ensembles for changing environments. In: Roli F, Kittler J, Windeatt T (eds) Multiple classifier systems. Volume 3077 of lecture notes in computer science. Springer, Berlin, pp 1–15

    Google Scholar 

  8. Tsymbal A (2004) The problem of concept drift: definitions and related work. TCD-CS-2004-15, vol 4. Department of Computer Science, Trinity College, Dublin

  9. Nishida K, Yamauchi K, Omori T (2005) Ace: adaptive classifiers-ensemble system for concept-drifting environments. Mult Classif Syst 3541:176–185

    Google Scholar 

  10. Japkowicz N (2000) The class imbalance problem: significance and strategies. In: Proceedings of the 2000 international conference on artificial intelligence (ICAI 2000), vol 1. pp 111–117

  11. Zhu X, Zhang P, Lin X, Shi Y (2010) Active learning from stream data using optimal weight classifier ensemble. IEEE Trans Syst Man Cybern B Cybern 40:1607–1621

    Google Scholar 

  12. Procopio M, Mulligan J, Grudic G (2009) Learning terrain segmentation with classifier ensembles for autonomous robot navigation in unstructured environments. J Field Robot 26(2):145–175

    Article  Google Scholar 

  13. Street W, Kim Y (2001) A streaming ensemble algorithm (sea) for large-scale classification. In: Proceedings of the seventh ACM SIGKDD international conference on knowledge discovery and data mining. KDD ’01. ACM, New York, pp 377–382

  14. Wang H, Fan W, Yu P, Han J (2003) Mining concept-drifting data streams using ensemble classifiers. In: Proceedings of the ninth ACM SIGKDD international conference on knowledge discovery and data mining. KDD ’03. ACM, New York, pp 226–235

  15. Scholz M, Klinkenberg R (2007) Boosting classifiers for drifting concepts. Intell Data Anal 11(1):3–28

    Google Scholar 

  16. Elwell R, Polikar R (2009) Incremental learning of variable rate concept drift. Mult Classif Syst 5519:142–151

    Google Scholar 

  17. Rodriguez J, Kuncheva L (2010) Combining online classification approaches for changing environments. Struct Syntactic Stat Pattern Recognit 5342:520–529

    Google Scholar 

  18. Karnick M, Muhlbaier M, Polikar R (2008) Incremental learning in non-stationary environments with concept drift using a multiple classifier based approach. In: 19th international conference on pattern recognition, 2008 (ICPR 2008). pp 1–4

  19. Kolter J, Maloof M (2003) Dynamic weighted majority: a new ensemble method for tracking concept drift. In: Third IEEE international conference on data mining, 2003 (ICDM 2003). pp 123–130

  20. Kolter J, Maloof M (2007) Dynamic weighted majority: an ensemble method for drifting concepts. J Mach Learn Res 8:2755–2790

    MATH  Google Scholar 

  21. Pocock A, Yiapanis P, Singer J, Luján M, Brown G (2010) Online non-stationary boosting. Mult Classif Syst 5997:205–214

    Google Scholar 

  22. Oza N (2001) Online ensemble learning. PhD thesis, University of California, Berkeley

  23. Huang C, Ai H, Yamashita T, Lao S, Kawade M (2007) Incremental learning of boosted face detector. In: IEEE 11th international conference on computer vision, 2007 (ICCV 2007). pp 1–8

  24. Pelossof R, Jones M, Vovsha I, Rudin C (2009) Online coordinate boosting. In: IEEE 12th international conference on computer vision workshops (ICCV workshops), 2009. pp 1354–1361

  25. Grabner H, Leistner C, Bischof H (2008) Semi-supervised on-line boosting for robust tracking. In: Forsyth D, Torr P, Zisserman A (eds) Computer vision—ECCV 2008. Volume 5302 of lecture notes in computer science. Springer, Berlin, pp 234–247

    Google Scholar 

  26. Bifet A, Holmes G, Pfahringer B, Kirkby R, Gavaldà R (2009) New ensemble methods for evolving data streams. In: Proceedings of the 15th ACM SIGKDD international conference on knowledge discovery and data mining. KDD ’09. ACM, New York, pp 139–148

  27. Barczak ALC, Johnson MJ, Messom CH (2008) Empirical evaluation of a new structure for adaboost. In: SAC ’08: proceedings of the 2008 ACM symposium on applied computing. ACM, Fortaleza, pp 1764–1765

  28. Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. In: CVPR01, Kauai, HI, IEEE (December), vol I. pp 511–518

  29. Susnjak T, Barczak A, Hawick K (2010) A modular approach to training cascades of boosted ensembles. In: Structural, syntactic, and statistical pattern recognition. Volume 6218 of lecture notes in computer science. Springer, Berlin, pp 640–649

  30. Freund Y, Schapire RE (1995) A decision-theoretic generalization of on-line learning and an application to boosting. In: EuroCOLT ’95: proceedings of the second European conference on computational learning theory. Springer, London, pp 23–37

  31. Georghiades A, Belhumeur P, Kriegman D (2001) From few to many: illumination cone models for face recognition under variable lighting and pose. IEEE Trans Pattern Anal Mach Intell 23(6):643–660

    Article  Google Scholar 

  32. Allwein EL, Schapire RE, Singer Y (2001) Reducing multiclass to binary: a unifying approach for margin classifiers. J Mach Learn Res 1:113–141

    MathSciNet  MATH  Google Scholar 

  33. Lorena AC, Carvalho AC, Gama JAM (2008) A review on the combination of binary classifiers in multiclass problems. Artif Intell Rev 30:19–37

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Massey University, Albany, New Zealand

    Teo Susnjak, Andre L. C. Barczak & Ken A. Hawick

Authors
  1. Teo Susnjak
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andre L. C. Barczak
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ken A. Hawick
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Teo Susnjak.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Susnjak, T., Barczak, A.L.C. & Hawick, K.A. Adaptive cascade of boosted ensembles for face detection in concept drift. Neural Comput & Applic 21, 671–682 (2012). https://doi.org/10.1007/s00521-011-0663-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-011-0663-x

Keywords

Search

Navigation