Skip to main content
SpringerLink
Log in

On improving performance of surface inspection systems by online active learning and flexible classifier updates

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Classification of detected events is a central component in state-of-the-art surface inspection systems that still relies on manual parametrization. While machine-learned classifiers promise supreme accuracy, their reliability depends on complete and correct annotation of an extensive training database, leaving the risk of unpredictable behavior in changing production environments. We propose an active learning-based training framework, which selectively presents questionable events for user annotation and is capable of online operation. Evaluation results on two data streams from microfluidic chips and elevator sheaves production show that annotation effort can be reduced by 90 % with negligible loss of accuracy. Simulation runs introducing new event classes show that the online active learning procedure is both efficient in terms of learning speed and robust in maintaining the accuracy levels of existing classes. The results underline the feasibility and potential of our approach that significantly reduces the required effort for inspection system setup and adapts to changes in the production process.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Angelov, P., Lughofer, E., Zhou, X.: Evolving fuzzy classifiers using different model architectures. Fuzzy Sets Syst. 159(23), 3160–3182 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  2. Belohlavek, R.: Fuzzy Relational Systems. Springer, Heidelberg, Berlin (2002)

    Book  MATH  Google Scholar 

  3. Bifet, A., Holmes, G., Kirkby, R., Pfahringer, B.: Data stream mining—a practical approach. Technical report, Department of Computer Sciences, University of Waikato, Japan (2011)

  4. Bifet, A., Kirkby, R.: Data stream mining—a practical approach. Technical report, Department of Computer Sciences, University of Waikato, Japan (2011)

  5. Bordes, A., Ertekin, S., Weston, J., Bottou, L.: Fast kernel classifiers with online and active learning. J. Mach. Learn. Res. 6, 1579–1619 (2005)

    MathSciNet  MATH  Google Scholar 

  6. Bouchachia, A.: Incremental induction of classification fuzzy rules. In: IEEE Workshop on Evolving and Self-Developing Intelligent Systems (ESDIS) 2009, Nashville, USA, pp. 32–39 (2009)

  7. Brazdil, P., Giraud-Carrier, C., Soares, C., Vilalta, R.: Metalearning. Springer, Berlin, Heidelberg (2009)

    MATH  Google Scholar 

  8. Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)

    Article  MATH  Google Scholar 

  9. Breiman, L., Friedman, J., Stone, C.J., Olshen, R.A.: Classification and Regression Trees. Chapman and Hall, Boca Raton (1993)

    Google Scholar 

  10. Brinkmann, R.: The Art and Science of Digital Compositing. Morgan Kaufmann, San Francisco (1999)

    Google Scholar 

  11. Caleb-Solly, P., Smith, J.E.: Adaptive surface inspection via interactive evolution. Image Vis. Comput. 25(7), 1058–1072 (2007)

    Article  Google Scholar 

  12. Chapelle, O., Schoelkopf, B., Zien, A.: Semi-Supervised Learning. MIT Press, Cambridge (2006)

    Book  Google Scholar 

  13. Cheng, J., Wang, K.: Active learning for image retrieval with co-svm. Pattern Recognit. 40, 330–334 (2006)

    Article  Google Scholar 

  14. Cheng, Y., Chen, Z., Liu, L., Wang, J., Agrawal, A., Choudhary, A.: Feedback-driven multiclass active learning for data streams. In: Proceedings of the 22nd ACM International Conference on Information and Knowledge management, CIKM ’13, pp. 1311–1320. ACM, New York (2013)

  15. Chu, W., Zinkevich, M., Li, L., Thomas, A., Zheng, B.: Unbiased online active learning in data streams. In: Proceedings of the KDD 2011, San Diego, California (2011)

  16. Dean, J.: Big Data, Data Mining, and Machine Learning: Value Creation for Business Leaders and Practitioners. Wiley, Hoboken (2012)

    Google Scholar 

  17. Demant, C., Streicher-Abel, B., Waszkewitz, P.: Industrial Image Processing: Visual Quality Control in Manufacturing. Springer, Berlin, Heidelberg (1999)

    Book  Google Scholar 

  18. Donmez, P., Carbonell, J.G.: From active to proactive learning methods. In: Kornoacki, J., Ras, Z.W., Wierzchon, S.T., Kacprzyk, J. (eds.) Advances in Machine Learning Part I. Studies in Computational Intelligence, vol. 262, pp. 97–120. Springer, Berlin, Heidelberg (2010)

    Google Scholar 

  19. Donmez, P., Carbonell, J.G., Bennett, P.N.: Dual strategy active learning. In: Kornoacki, J., Lopez de Mantaras, R., Matwin, S., Mladenic, D., Skowron, A. (eds.) Proceedings of the ECML 2007 Conference. LNCS (LNAI), vol. 4701, pp. 116–127. Springer, Heidelberg (2007)

  20. Efron, B., Tibshirani, R.: An Introduction to the Bootstrap. Chapman and Hall/CRC, Boca Raton (1993)

    Book  MATH  Google Scholar 

  21. Efron, B., Tibshirani, R.: Improvements on cross-validation: the 632 + bootstrap method. J. Am. Stat. Assoc. 92(438), 548–560 (1997)

    MathSciNet  MATH  Google Scholar 

  22. Eitzinger, C., Heidl, W., Lughofer, E., Raiser, S., Smith, J.E., Tahir, M.A., Sannen, D., Van Brussel, H.: Assessment of the influence of adaptive components in trainable surface inspection systems. Mach. Vis. Appl. 21(5), 613–626 (2009)

    Article  Google Scholar 

  23. Fürnkranz, J.: Round robin classification. J. Mach. Learn. Res. 2, 721–747 (2002)

    MathSciNet  MATH  Google Scholar 

  24. Guyon, I., Elisseeff, A.: An introduction to variable and feature selection. J. Mach. Learn. Res. 3, 1157–1182 (2003)

    MATH  Google Scholar 

  25. Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference and Prediction, 2nd edn. Springer, New York, Berlin, Heidelberg (2009)

    Book  Google Scholar 

  26. Heidl, W., Thumfart, S., Lughofer, E., Eitzinger, C., Klement, E.-P.: Machine learning based analysis of gender differences in visual inspection decision making. Inf. Sci. 224, 62–76 (2013)

    Article  MathSciNet  Google Scholar 

  27. Hu, W., Hu, W., Xi, N., Maybank, S.: Unsupervised active learning based on hierarchical graph-theoretic clustering. IEEE Trans. Syst. Man Cybern. Part B: Cybern. 39(5), 1147–1161 (2009)

    Article  Google Scholar 

  28. Hua, G., Long, C., Yang, M., Gao, Y.: Collaborative active learning of a kernel machine ensemble for recognition. In: 2013 IEEE International Conference on Computer Vision (ICCV), pp. 1209–1216, Dec (2013)

  29. Hühn, J., Hüllermeier, E.: FR3: a fuzzy rule learner for inducing reliable classifiers. IEEE Trans. Fuzzy Syst. 17(1), 138–149 (2009)

    Article  Google Scholar 

  30. Hüllermeier, E., Brinker, Klaus: Learning valued preference structures for solving classification problems. Fuzzy Sets Syst. 159(18), 2337–2352 (2008)

    Article  MATH  Google Scholar 

  31. Joshi, A.J., Porikli, F., Papanikolopoulos, N.: Multi-class active learning for image classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2372–2379. IEEE Press (2009)

  32. Kim, C.W., Koivo, A.J.: Hierarchical classification of surface defects on dusty wood boards. Pattern Recognit. Lett. 15(7), 713–721 (1994)

    Article  Google Scholar 

  33. Krishnamoorthy, K., Mathew, T.: Statistical Tolerance Regions: Theory, Applications, and Computation. Wiley, Hoboken (2009)

    Book  Google Scholar 

  34. Kuncheva, L.: Fuzzy Classifier Design. Physica-Verlag, Heidelberg (2000)

    Book  MATH  Google Scholar 

  35. Kuncheva, L.: Combining Pattern Classifiers: Methods and Algorithms. Wiley-Interscience (Wiley), Southern Gate, Chichester (2004)

  36. Lewis, D.D., Catlett, J.: Heterogeneous uncertainty sampling for supervised learning. In: Proceedings of the 11th International Conference on Machine Learning, New Brunswick, New Jersey, pp. 148–156 (1994)

  37. Li, X., Wang, L., Sung, E.: Multilabel SVM active learning for image classification. In: Proceedings of the International Conference on Image Processing (ICIP), Singapore, vol. 4, pp. 2207–2010 (2004)

  38. Long, C., Hua, G., Kapoor, A.: Active visual recognition with expertise estimation in crowdsourcing. In: 2013 IEEE International Conference on Computer Vision (ICCV), pp. 3000–3007, Dec (2013)

  39. Loy, C.C., Hospedales,T.M., Xiang, T., Gong, S.: Stream-based joint exploration-exploitation active learning. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1560–1567, June (2012)

  40. Lughofer, E.: Extensions of vector quantization for incremental clustering. Pattern Recognit. 41(3), 995–1011 (2008)

    Article  MATH  Google Scholar 

  41. Lughofer, E.: On-line evolving image classifiers and their application to surface inspection. Image Vis. Comput. 28(7), 1065–1079 (2010)

    Article  Google Scholar 

  42. Lughofer, E.: On-line assurance of interpretability criteria in evolving fuzzy systems—achievements, new concepts and open issues. Inf. Sci. 251, 22–46 (2013)

    Article  Google Scholar 

  43. Lughofer, E., Buchtala, O.: Reliable all-pairs evolving fuzzy classifiers. IEEE Trans. Fuzzy Syst. 21(4), 625–641 (2013)

    Article  Google Scholar 

  44. Lughofer, E., Eitzinger, C., Guardiola, C.: On-line quality control with flexible evolving fuzzy systems. In: Sayed-Mouchaweh, M., Lughofer, E. (eds.) Learning in Non-Stationary Environments: Methods and Applications, pp. 375–406. Springer, New York (2012)

    Chapter  Google Scholar 

  45. Lughofer, E., Smith, J.E., Tahir, M.A., Caleb-Solly, P., Eitzinger, C., Sannen, D., Nuttin, M.: Human–machine interaction issues in quality control based on on-line image classification. IEEE Trans. Syst. Man Cybern. Part A: Syst. Hum. 39(5), 960–971 (2009)

    Article  Google Scholar 

  46. Nauck, D., Kruse, R.: NEFCLASS-X—a soft computing tool to build readable fuzzy classifiers. BT Technol. J. 16(3), 180–190 (1998)

    Article  Google Scholar 

  47. Pratama, M., Anavatti, S.G., Angelov, P., Lughofer, E.: PANFIS: a novel incremental learning machine. IEEE Trans. Neural Netw. Learn. Syst. 25(1), 55–68 (2014)

    Article  Google Scholar 

  48. Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann Publishers, San Francisco (1993)

    Google Scholar 

  49. Raghavan, H., Madani, O., Jones, R.: Active learning with feedback on both features and instances. J. Mach. Learn. Res. 7, 1655–1686 (2006)

    MathSciNet  MATH  Google Scholar 

  50. Raiser, S., Lughofer, E., Eitzinger, C., Smith, J.E.: Impact of object extraction methods on classification performance in surface inspection systems. Mach. Vis. Appl. 21(5), 627–641 (2010)

    Article  Google Scholar 

  51. Ravikumara, S., Ramachandran, K.I., Sugumaran, V.: Machine learning approach for automated visual inspection of machine components. Expert Syst. Appl. 38(4), 3260–3266 (2011)

    Article  Google Scholar 

  52. Rodrigues, F., Pereira, F., Ribeiro, B.: Gaussian process classification and active learning with multiple annotators. In: Proceedings of the 31st International Conference on Machine Learning (ICML-14), pp. 433–441 (2014)

  53. Saffari, A., Leistner, C., Santner, J., Godec, M., Bischof, H.: On-line random forests. In: 2009 IEEE 12th International Conference on Computer Vision Workshops (ICCV Workshops), pp. 1393–1400 (2009)

  54. Sannen, D., van Brussel, H.: A multilevel information fusion approach for visual quality inspection. Inf. Fusion 13(1), 48–59 (2012)

    Article  Google Scholar 

  55. Schoonahd, J.W., Gould, J.D., Miller, L.A.: Studies of visual inspection. Ergonomics 16(4), 365–379 (1973)

    Article  Google Scholar 

  56. Sculley, D.: Online active learning methods for fast label efficient spam filtering. In: Proceedings of the Fourth Conference on Email and AntiSpam, Mountain View, California (2007)

  57. Settles, B.: Active Learning. Morgan & Claypool Publishers, San Rafael (2012)

    MATH  Google Scholar 

  58. Stone, M.: Cross-validatory choice and assessment of statistical predictions. J. R. Stat. Soc. 36(1), 111–147 (1974)

    MATH  Google Scholar 

  59. Tong, S., Koller, D.: Support vector machine active learning with application to text classification. J. Mach. Learn. Res. 2, 45–66 (2001)

    Google Scholar 

  60. Wu, X., Kumar, V.: The Top 10 Algorithms in Data Mining. Chapman & Hall, Boca Raton (2009)

    Book  Google Scholar 

  61. Yang, Z., Tang, J., Zhang, Y.: Active learning for streaming networked data. In: Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management, CIKM ’14, pp. 1129–1138. ACM, New York (2014)

Download references

Acknowledgments

This work was funded via the project ‘Improving the usability of machine learning in industrial inspection systems’ (useML) by the Austrian Research Promotion Agency (FFG) under the scope of the ‘Information and communication technology of the future’ program. This program is promoted by the Austrian Federal Ministry of Transport, Innovation and Technology (BMVIT) and the Federal Ministry of Science, Research and Economy (BMWFW).

Author information

Authors and Affiliations

  1. Machine Vision Group, Profactor GmbH, Steyr-Gleink, Austria

    Eva Weigl, Wolfgang Heidl & Christian Eitzinger

  2. Department of Knowledge-Based Mathematical Systems, Johannes Kepler University Linz, Linz, Austria

    Edwin Lughofer

  3. Sony DADC Austria AG, Anif, Austria

    Thomas Radauer

Authors
  1. Eva Weigl
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wolfgang Heidl
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Edwin Lughofer
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Thomas Radauer
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Christian Eitzinger
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Eva Weigl.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Weigl, E., Heidl, W., Lughofer, E. et al. On improving performance of surface inspection systems by online active learning and flexible classifier updates. Machine Vision and Applications 27, 103–127 (2016). https://doi.org/10.1007/s00138-015-0731-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-015-0731-9

Keywords

Search

Navigation