Skip to main content

Advertisement

SpringerLink
Log in

Learning probabilistic classifiers for human–computer interaction applications

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Human–computer interaction (HCI) lies at the crossroads of many scientific areas including artificial intelligence, computer vision, face recognition, motion tracking, etc. It is argued that to truly achieve effective human–computer intelligent interaction, the computer should be able to interact naturally with the user, similar to the way HCI takes place. In this paper, we discuss training probabilistic classifiers with labeled and unlabeled data for HCI applications. We provide an analysis that shows under what conditions unlabeled data can be used in learning to improve classification performance, and we investigate the implications of this analysis to a specific type of probabilistic classifiers, Bayesian networks. Finally, we show how the resulting algorithms are successfully employed in facial expression recognition, face detection, and skin detection.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Baluja, S.: Probabilistic modelling for face orientation discrimination: learning from labeled and unlabeled data. In: Neural Information and Processing Systems, pp. 854–860 (1998)

  2. Black, M.J., Yacoob, Y.: Tracking and recognizing rigid and non-rigid facial motions using local parametric models of image motion. In: Proceedings of the International Conference on Computer Vision, pp. 374–381 (1995)

  3. Blum, A., Mitchell, T.: Combining labeled and unlabeled data with co-training. In: Conference on Learning Theory, pp. 92–100 (1998)

  4. Bosson, A., Cawley, G., Chan, Y., Harvey, R.: Non-retrieval: blocking pornographic images. In: International Conference on Image and Video Retrieval, pp. 50–60 (2002)

  5. Brand, M.: An entropic estimator for structure discovery. In: Neural Information and Processing Systems, pp. 723–729 (1998)

  6. Bruce, R.: Semi-supervised learning using prior probabilities and EM In: International Joint Conference on Artificial Intelligence, Workshop on Text Learning: Beyond Supervision (2001)

  7. Caetano, T., Olabarriaga, S., Barone, D.: Do mixture models in chromaticity space improve skin detection? Pattern Recog. 36, 3019–3021 (2003)

    Article  Google Scholar 

  8. Castelli, V.: The relative value of labeled and unlabeled samples in pattern recognition. PhD Thesis, Stanford University, Stanford, CA (1994)

    Google Scholar 

  9. Chen, L.S.: Joint processing of audio-visual information for the recognition of emotional expressions in human–computer interaction. PhD Thesis, University of Illinois at Urbana-Champaign (2000)

  10. Cheng, J., Greiner, R., Kelly, J., Bell, D.A., Liu, W.: Learning Bayesian networks from data: an information-theory based approach. Artif. Intell. J. 137, 43–90 (2002)

    MathSciNet  Google Scholar 

  11. Chow, C.K., Liu, C.N.: Approximating discrete probability distribution with dependence trees. IEEE Trans. Inf. Theory 14, 462–467 (1968)

    Article  Google Scholar 

  12. Cohen, I.: Semi-supervised learning of classifiers with application to human computer interaction. PhD Thesis, University of Illinois at Urbana-Champaign (2003)

  13. Cohen, I., Sebe, N., Cozman, F., Cirelo, M., Huang, T.S.: Learning Bayesian network classifiers for facial expression recognition using both labeled and unlabeled data. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 595–601 (2003)

  14. Cohen, I., Sebe, N., Garg, A., Chen, L., Huang, T.S.: Facial expression recognition from video sequences: temporal and static modelling. Comput. Vis. Image Understand. 91(1–2), 160–187 (2003)

    Google Scholar 

  15. Cohen, I., Cozman, F., Sebe, N., Cirello, M., Huang, T.S.: Semi-supervised learning of classifiers: theory, algorithms, and their applications to human–computer interaction. IEEE Trans. Pattern Anal. Mach. Intell. 26(12), 1553–1567 (2004)

    Article  Google Scholar 

  16. Colmenarez, A.J., Huang, T.S.: Face detection with information based maximum discrimination. In: IEEE Conference on Computer Vision and Pattern Recogntion, pp. 782–787 (1997)

  17. Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. Ser. B 39(1), 1–38 (1977)

    MathSciNet  Google Scholar 

  18. Donato, G., Bartlett, M.S., Hager, J.C., Ekman, P., Sejnowski, T.J.: Classifying facial actions. IEEE Trans. Pattern Anal. Mach. Intell. 21(10), 974–989 (1999)

    Article  Google Scholar 

  19. Dougherty, J., Kohavi, R., Sahami, M.: Supervised and unsupervised discretization of continuous features. In: International Conference on Machine Learning, pp. 194–202 (1995)

  20. Ekman, P.: Strong evidence for universals in facial expressions: a reply to Russell's mistaken critique. Psychol. Bull. 115(2), 268–287 (1994)

    Article  Google Scholar 

  21. Ekman, P., Friesen, W.V.: Facial Action Coding System: Investigator's Guide. Consulting Psychologists Press, Palo Alto, CA (1978)

    Google Scholar 

  22. Essa, I.A., Pentland, A.P.: Coding, analysis, interpretation, and recognition of facial expressions. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 757–763 (1997)

    Article  Google Scholar 

  23. Fasel, B., Luettin, J.: Automatic facial expression analysis: a survey. Pattern Recog. 36, 259–275 (2003)

    Article  Google Scholar 

  24. Fleck, M., Forsyth, D., Bregler, C.: Finding naked people. In: European Conference on Computer Vision, pp. 593–602 (1996)

  25. Friedman, N.: The Bayesian structural EM algorithm. In: Proceedings of the Conference on Uncertainty in Artificial Intelligence, pp. 129–138 (1998)

  26. Friedman, N., Geiger, D., Goldszmidt, M.: Bayesian network classifiers. Mach. Learn. 29(2), 131–163 (1997)

    Article  Google Scholar 

  27. Ghani, R.: Combining labeled and unlabeled data for multiclass text categorization. In: International Conference on Machine Learning, pp. 187–194 (2002)

  28. Goleman, D.: Emotional Intelligence. Bantam Books, New York (1995)

    Google Scholar 

  29. Hajek, B.: Cooling schedules for optimal annealing. Math. Oper. Res. 13, 311–329 (1988)

    MATH  MathSciNet  Google Scholar 

  30. Heisele, B., Ho, P., Wu, J., Poggio, T.: Face recognition: component-based versus global approaches. Comput. Vis. Image Understand. 91(1–2), 6–21 (2003)

    Google Scholar 

  31. Hjelmas, E., Low, B.K.: Face detection: a survey. Comput. Vis. Image Understand. 83, 236–274 (2003)

    Google Scholar 

  32. Jedynak, B., Zheng, H., Daoudi, M.: Statistical models for skin detection. In: IEEE Conference on Computer Vision and Pattern Recognition, Workshop on Statistical Analysis in Computer Vision (2003)

  33. Jones, M.J., Rehg, J.M.: Statistical color models with application to skin detection. Int. J. Comput. Vis. 46(1), 81–96 (2002)

    Article  Google Scholar 

  34. Kanade, T., Cohn, J.F., Tian, Y.: Comprehesive database for facial expression analysis. In: International Conference on Automatic Face and Gesture Recognition, pp. 46–53 (2000)

  35. Kouzani, A.Z.: Locating human faces within images. Comput. Vis. Image Understand. 91(3), 247–279 (2003)

    Article  Google Scholar 

  36. Madigan, D., York, J.: Bayesian graphical models for discrete data. Int. Stat. Rev. 63, 215–232 (1995)

    Google Scholar 

  37. McCallum, A.K., Nigam, K.: Employing EM in pool-based active learning for text classification. In: International Conference on Machine Learning, pp. 350–358 (1998)

  38. McKenna, S.J., Gong, S., Raja, Y.: Modeling facial colour and identity with Gaussian mixtures. Pattern Recog. 31, 1883–1892 (1998)

    Article  Google Scholar 

  39. Metropolis, N., Rosenbluth, A.W., Rosenbluth, M.N., Teller, A.H., Teller, E.: Equation of state calculation by fast computing machines. J. Chem. Phys. 21, 1087–1092 (1953)

    Article  Google Scholar 

  40. Nefian, A., Hayes, M.: Face recognition using an embedded HMM. In: IEEE Conference on Audio and Video-Based Biometric Person Authentication, pp. 19–24 (1999)

  41. Nigam, K., McCallum, A., Thrun, S., Mitchell, T.: Text classification from labeled and unlabeled documents using EM Mach. Learn. 39, 103–134 (2000)

    Article  Google Scholar 

  42. Oliver, N., Pentland, A., Bérard, F.: LAFTER: a real-time face and lips tracker with facial expression recognition. Pattern Recog. 33, 1369–1382 (2000)

    Article  Google Scholar 

  43. O'Neill, T.J.: Normal discrimination with unclassified obseravations. J. Am. Stat. Assoc. 73(364), 821–826 (1978)

    MATH  MathSciNet  Google Scholar 

  44. Osuna, E., Freund, R., Girosi, F.: Training support vector machines: an application to face detection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 130–136 (1997)

  45. Pantic, M., Rothkrantz, L.J.M.: Automatic analysis of facial expressions: the state of the art. IEEE Trans. Pattern Anal. Mach. Intell. 22(12), 1424–1445 (2000)

    Google Scholar 

  46. Pantic, M., Rothkrantz, L.J.M.: Toward an affect-sensitive multimodal human–computer interaction. Proc. IEEE 91(9), 1370–1390 (2003)

    Article  Google Scholar 

  47. Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, San Mateo, CA (1988)

    Google Scholar 

  48. Pentland, A.: Looking at people. Commun. ACM 43(3), 35–44 (2000)

    Article  Google Scholar 

  49. Pentland, A., Choudhury, T.: Face recognition for smart environments. IEEE Comput. 33(2), 50–55 (2000)

    Google Scholar 

  50. Ratsaby, J., Venkatesh, S.S.: Learning from a mixture of labeled and unlabeled examples with parametric side information. In: Conference on Computational Learning Theory, pp. 412–417 (1995)

  51. Rosenblum, M., Yacoob, Y., Davis, L.S.: Human expression recognition from motion using a radial basis function network architecture. IEEE Trans. Neural Netw. 7(5), 1121–1138 (1996)

    Google Scholar 

  52. Rowley, H., Baluja, S., Kanade, T.: Neural network-based face detection. IEEE Trans. Pattern Anal. Mach. Intell. 20(1), 23–38 (1998)

    Article  Google Scholar 

  53. Saxe, D., Foulds, R.: Toward robust skin identification in video images. In: Automatic Face and Gesture Recognition, pp. 379–384 (1996)

  54. Schneiderman, H.: Learning a restricted Bayesian network for object detection. In: IEEE International Conference on Computer Vision and Pattern Recognition, pp. 639–646 (2004)

  55. Schwerdt, K., Crowley, J.L.: Robust face tracking using color. In: Automatic Face and Gesture Recognition, pp. 90–95 (2000)

  56. Sebe, N., Cohen, I., Garg, A., Lew, M.S., Huang, T.S.: Emotion recognition using a Cauchy naive Bayes classifier. In: International Conference on Pattern Recognition (2002)

  57. Seeger, M.: Learning with labeled and unlabeled data. Technical Report, Edinburgh University, Edinburgh, UK (2001)

    Google Scholar 

  58. Shahshahani, B., Landgrebe, D.: Effect of unlabeled samples in reducing the small sample size problem and mitigating the Hughes phenomenon. IEEE Trans. Geosci. Remote Sens. 32(5), 1087–1095 (1994)

    Google Scholar 

  59. Tao, H., Huang, T.S.: Connected vibrations: a modal analysis approach to non-rigid motion tracking. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 735–740 (1998)

  60. Terrillon, J.-C., Shirazi, M.N., Fukamachi, H., Akamatsu, S.: Comparative performance of different skin chrominance models and chrominance spaces for the automatic detection of human faces in color images. In: Automatic Face and Gesture Recognition, pp. 54–61 (2000)

  61. Viola, P., Jones, M.J.: Robust real-time object detection. Int. J. Comput. Vis. 57(2) (2004)

  62. Wang, R.R., Huang, T.S., Zhong, J.: Generative and discriminative face modeling for detection. In: Automatic Face and Gesture Recognition (2002)

  63. White, H.: Maximum likelihood estimation of misspecified models. Econometrica 50(1), 1–25 (1982)

    MATH  MathSciNet  Google Scholar 

  64. Yang, M.-H., Ahuja, N.: Detecting human faces in color images. In: International Conference on Image Processing, pp. 127–130 (1998)

  65. Yang, M.-H., Roth, D., Ahuja, N.: SNoW based face detector. In: Neural Information Processing Systems, pp. 855–861 (2000)

  66. Yang, M.-H., Kriegman, D., Ahuja, N.: Detecting faces in images: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 24(1), 34–58 (2002)

    Google Scholar 

  67. Zhang, T., Oles, F.: A probability analysis on the value of unlabeled data for classification problems. In: International Conference on Machine Learning (2000)

Download references

Author information

Authors and Affiliations

  1. Faculty of Science, University of Amsterdam, The Netherlands

    Nicu Sebe & Theo Gevers

  2. HP Labs, Palo Alto, CA, USA

    Ira Cohen

  3. Escola Politécnica, Universidade de São Paulo, São Paulo, Brazil

    Fabio G. Cozman

  4. Beckman Institute, University of Illinois, Urbana-Champaign, IL, USA

    Thomas S. Huang

Authors
  1. Nicu Sebe
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ira Cohen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fabio G. Cozman
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Theo Gevers
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Thomas S. Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nicu Sebe.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Sebe, N., Cohen, I., Cozman, F.G. et al. Learning probabilistic classifiers for human–computer interaction applications. Multimedia Systems 10, 484–498 (2005). https://doi.org/10.1007/s00530-005-0177-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-005-0177-4

Keywords

Search

Navigation