Skip to main content
SpringerLink
Log in

Charting the behavioural state of a person using a backpropagation neural network

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

This paper describes the application of a backpropagation artificial neural network (ANN) for charting the behavioural state of previously unseen persons. In a simulated theft scenario participants stole or did not steal some money and were interviewed about the location of the money. A video of each interview was presented to an automatic system, which collected vectors containing nonverbal behaviour data. Each vector represented a participant’s nonverbal behaviour related to “deception” or “truth” for a short period of time. These vectors were used for training and testing a backpropagation ANN which was subsequently used for charting the behavioural state of previously unseen participants. Although behaviour related to “deception” or “truth” is charted the same strategy can be used to chart different psychological states over time and can be tuned to particular situations, environments and applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Scherer KR, Ekman P (eds) (1982) Handbook of methods in nonverbal behavior research. Cambridge University Press, Cambridge

  2. Zuckerman M, Driver RE (1985) Telling lies: Verbal and nonverbal correlates of deception. In: Siegman AW, Feldstein S (eds) Multichannel integrations of nonverbal behavior. L Erlbaum Associates, Hillside

    Google Scholar 

  3. Gavrila DM (1999) The visual analysis of human movement: a survey. Comput Vis Image Underst 3(1):82–98

    Article  Google Scholar 

  4. Turk MA, Pentland AP (1991) Face recognition using Eigenfaces. In: IEE computer society conference on computer vision and pattern recognition pp 586–591

  5. Valentin D, Abdi H, O’Toole AJ, Cottrell GW (1994) Connectionist models of face processing: a survey. Pattern Recognit 27(9):1209–1230

    Article  Google Scholar 

  6. Cottrell GW, Metcalfe J (1991) Empath: Face, emotion and gender recognition using holons. In: Lippman et al (ed) Advances in neural processing systems 3. Morgan-Kaufmann Publishers Inc, San Mateo, pp 573–577

    Google Scholar 

  7. Golomb BA, Lawrence DT, Sejnowski TJ (1991) SEXNET: a neural network identifies sex from human faces. In: Lippman et al (ed) Advances in neural information processing systems 3. Morgan Kaufmann Publishers Inc, San Mateo

  8. Zhang M (1996) Face recognition using artificial neural network group-based adaptive tolerance (G.A.T.) trees. IEEE Trans Neural Netw 7(3)

  9. Bartlett MS, Hager JC, Ekman P, Sejnowski TJ (1999) Measuring facial expressions by computer image analysis. Psychophysiology 36:253–263

    Article  Google Scholar 

  10. Rosenblum M, Yacoob Y, Davis LS (1996) Human expression recognition from motion using a radial bias function network architecture. IEEE Trans Neural Netw 7(5):1121–1138

    Article  Google Scholar 

  11. Fels S, Hinton GE (1998) Glove Talk 2, a neural network interface which maps gestures to parallel formant speech synthesizer controls. IEEE Trans Neural Netw 9(1):205–212

    Article  Google Scholar 

  12. Rothwell J (2002) Artificial neural networks for psychological profiling using multichannels of nonverbal behaviour. PhD Thesis, Manchester Metropolitan University

  13. Pavidis I, Eberhardt NL, Levine JA (2002) Human behavior Seeing through the face of deception (brief communication). Nature 415:35

    Article  Google Scholar 

  14. Cohn JF, Katz GS (1998) Bimodal expression of emotion by face and voice. In: Proceedings of the sixth ACM international conference on multimedia: workshop on face/gesture recognition and their applications. ACM, New York, pp 41–44

  15. Ekman P (1988) Lying and nonverbal behavior: theoretical issues and new findings. J Nonverbal Behav 12:163–175

    Article  Google Scholar 

  16. Johnson RC (1999) Computer program recognizes facial expressions. Electronic Engineering Times, pp 51

  17. Bartlett MS, Donato G, Movellan JR, Hager JC, Ekman P, Sejnowski TJ (1999) Face image analysis for expression measurement and detection of deceit. In: Proceedings of the 6th annual joint symposium on neural computation. The Institute For Neural Computation, University of California, San Diego

  18. Iwano Y, Sukegawa H, Kasahara Y, Shirai K (1995) Extraction of speaker’s feeling using facial image and speech. In: IEEE international workshop on robot and human communication, pp 101–106

  19. Rothwell J, Bandar Z, O’Shea JD, McLean D (2006) Silent talker: a new computer-based system for the analysis of facial cues to deception. Appl Cogn Psychol (in press)

  20. Freud S (1959) Fragments of an analysis of a case of hysteria, collected papers, Vol. 3 Basic books, New York (Original work published 1905)

  21. Vrij A (1994) The impact of information and setting on detection of deception by police detectives. J Nonverbal Behav 18:117–136

    Article  Google Scholar 

  22. Stone M (1974) Cross-validatory choice and assessment of statistical predictions. J R Stat Soc Ser B 36:111–147

    MATH  Google Scholar 

  23. Hassoun MH (1995) Fundamentals of neural networks. MIT, Cambridge

    MATH  Google Scholar 

  24. McLean D (1995) Improving generalisation in continuous data domains. Ph.D. Thesis, Manchester Metropolitan University

  25. Rumelhart DE, McClelland J (1986) Parallel distributed processing. In: PDP Research Group (eds) Explorations in the micro structure of cognition Vol 1, Bradford Books, MIT, Cambridge, MA

  26. Wessels LFA, Barnard E (1992) Avoiding false local minima by proper initialization of connections. IEEE Trans Neur Netw 3(6):899–905

    Article  Google Scholar 

  27. Kolen JF, Pollack JB (1991) Back propagation is sensitive to initial conditions. In: Lippmann et al (eds) Advances in neural information processing systems 3. Morgan Kaufmann, San Mateo, pp 860–867

    Google Scholar 

  28. McLean D, Bandar Z, O’Shea J (1997) The evolution of a feed forward neural network trained under backpropagation. In: Proceedings of the third international conference on artificial neural network and genetic algorithms (ICANGA97). Springer, Berlin Heidelberg New York, pp 518–522

  29. Plaut D, Nowlan SJ, Hinton GE (1986) Experiments on learning by back propagation. Computer Science Department, Carnegie Mellon University, Pittsburgh, PA 15213

  30. Schraudolph NN, Sejnowski TJ (1996) Tempering backpropagation networks: not all weights are created equal. In: Touretzky et al (eds) Advances in neural information processing systems 8. MIT, Cambridge

    Google Scholar 

  31. Cybenko G (1989) Approximation by superpositions of a sigmoidal function. Math Control Signals Syst 2:303–314

    Article  MATH  MathSciNet  Google Scholar 

  32. Cybenko G (1998) Continuous values neural networks with two hidden layers are sufficient. Technical Report, Department of Computer Science, Tufts University, Medford, MA

  33. McCormick D, Roach A (1987) Measurement, statistics and computation. Wiley, London

    Google Scholar 

  34. Bond CF (1990) Lie detection across cultures. J Nonverbal Behav 14:189–204

    Article  Google Scholar 

  35. Feldman RS, Rime B (1991) Fundamentals of non-verbal behavior. Cambridge University Press, Cambridge

    Google Scholar 

  36. Vrij A (2000) Detecting lies and deceit: the psychology of lying and the implications for professional practice. Wiley, New York

    Google Scholar 

  37. Porter S, Woodworth M, Birt AR (2000) Truth, lies and videotape: an investigation of the ability of federal parole officers to detect deception. Law Hum Behav 24:6:643–658

    Article  Google Scholar 

  38. Ekman P, O’Sullivan M, Friesen WV, Scherer KR (1991) Invited Article: face, voice and body in detecting deceit. J Nonverbal Behav 15:125–135

    Article  Google Scholar 

  39. Ekman P (1985) Telling lies: clues to deceit in the marketplace, politics and marriage. W.W. Norton and Company, New York

    Google Scholar 

  40. Williams D (2001) How to sting the polygraph. Sting Publications. P.O. Box 720568, Norman, OK, USA

  41. Singh M, Vaid J, Sakhuja T (2000) Reading/writing vs handedness influences on line length estimation. Brain Cogn 43:398–402

    Google Scholar 

  42. Reuter-Lorenz PA, Kinsbourne M, Moscovitch M (1990) Hemispheric control of spatial attention. Brain Cogn 12:240–266

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computing and Mathematics, Manchester Metropolitan University, John Dalton Building, Chester Street, Manchester, M1 5GD, UK

    Janet Rothwell, Zuhair Bandar, James O’Shea & David McLean

Authors
  1. Janet Rothwell
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zuhair Bandar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. James O’Shea
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. David McLean
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zuhair Bandar.

Additional information

We thank those who kindly volunteered to participate in the study.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Rothwell, J., Bandar, Z., O’Shea, J. et al. Charting the behavioural state of a person using a backpropagation neural network. Neural Comput & Applic 16, 327–339 (2007). https://doi.org/10.1007/s00521-006-0055-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-006-0055-9

Keywords

Search

Navigation