Skip to main content
SpringerLink
Log in

Adaptive confidence learning for the personalization of pain intensity estimation systems

  • Original Paper
  • Published:
Evolving Systems Aims and scope Submit manuscript

Abstract

In this work, a method is presented for the continuous estimation of pain intensity based on fusion of bio-physiological and video features. Furthermore, a method is proposed for the adaptation of the system to unknown test persons based on unlabeled data. First, an analysis is presented that shows which modalities and feature sets are suited best for the task of recognizing pain levels in a person-independent setting. For this, a large set of features is extracted from the available bio-physiological channels (ECG, EMG and skin conductivity) and the video stream. We then propose a method to learn the confidence of a regression system using a multi-stage ensemble classifier. Based on the outcome of the classifier, which is realized by a neural network, confident samples are selected by the adaptation procedure. In various experiments, we show that the algorithm is able to detect highly confident samples which can be used to improve the overall performance. We furthermore discuss the current limitations of automatic pain intensity estimation—in light of the presented approach and beyond.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. Although the estimated quantity is based on the regression error, in accordance to the literature, we will call it confidence as it is a value in (0, 1] indicating how certain a classifier is about a sample. Similar techniques have for example been presented by Platt in the context of Support Vector Machines (Platt 1999).

  2. Here, only part A of the dataset is used. See Kächele et al. (2015) for details.

  3. The experiments here have been carried out using the SCL (Ledalab) features, as they are virtually as good as the early fusion, however with only 22 dimensions much more manageable than the 3154 dimensions of all the features concatenated. Further, the results here are based on standardized features, for which the baseline result for SCL (Ledalab) is 1.1122 RMSE.

References

  • Abdelwahab M, Busso C 2015) Supervised domain adaptation for emotion recognition speech. In: 2015 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2015, South Brisbane, Queensland, Australia, April 19–24, 2015, pp 5058–5062

  • AlZoubi O, Fossati D, D’Mello S, Calvo RA (2014) Affect detection from non-stationary physiological data using ensemble classifiers. Evol Syst 6(2):79–92

    Article  Google Scholar 

  • AlZoubi O, Rafael C, Stevens R (2009) Classification of eeg for affect recognition: An adaptive approach. In: Nicholson A, Li X (eds) AI 2009: Advances in Artificial Intelligence, vol 5866., Lecture Notes in Computer ScienceSpringer, Berlin Heidelberg, pp 52–61

  • Artemiadis PK, Kyriakopoulos KJ (2010) An EMG-based robot control scheme robust to time-varying EMG signal features. Trans Info Tech Biomed 14(3):582–588

    Article  Google Scholar 

  • Ashraf AB, Lucey S, Cohn JF, Chen T, Ambadar Z, Prkachin KM, Solomon PE (2009) The painful face—pain expression recognition using active appearance models. Image Vision Comput 27(12):1788–1796

    Article  Google Scholar 

  • Benedek M, Kaernbach C (2010) Decomposition of skin conductance data by means of nonnegative deconvolution. Psychophysiology 47(4):647–658

    Google Scholar 

  • Cao C, Slobounov S (2011) Application of a novel measure of EEG non-stationarity as ’Shannon- entropy of the peak frequency shifting’ for detecting residual abnormalities in concussed individuals. Clinical neurophysiology : Official Journal of the International Federation of Clinical Neurophysiology 122(7):1314–1321

    Article  Google Scholar 

  • Cauwenberghs G, Poggio T (2001) Incremental and decremental support vector machine learning. In: Advances in neural information processing systems, vol 13, pp 409–415

  • Chen W, Zhuang J, Yu W, Wang Z (2009) Measuring complexity using FuzzyEn, ApEn, and SampEn. Medl Eng Phys 31(1):61–68

    Article  Google Scholar 

  • Duviella E, Serir L, Sayed-Mouchaweh M (2013) An evolving classification approach for fault diagnosis and prognosis of a wind farm. In: Conference on Control and Fault-Tolerant Systems (SysTol) IEEE, pp 377–382

  • Glodek M, Tschechne S, Layher G, Schels M, Brosch T, Scherer S, Kächele M, Schmidt M, Neumann H, Palm G, Schwenker F (2011) Multiple classifier systems for the classification of audio-visual emotional states. In: D’Mello S, Graesser A, Schuller B, Martin JC (eds) Proceedings of the Fourth International Conference on Affective Computing and Intelligent Interaction (ACII’11) - Part II, LNCS 6975, Springer, pp 359–368

  • Hammal Z, Cohn JF (2012) Automatic detection of pain intensity. In: Proceedings of the 14th ACM International Conference on Multimodal Interaction, ICMI ’12, ACM, pp 47–52

  • He L, Jiang D, Yang L, Pei E, Wu P, Sahli H (2015) Multimodal affective dimension prediction using deep bidirectional long short-term memory recurrent neural networks. In: Proceedings of the 5th International Workshop on Audio/Visual Emotion Challenge, ACM , pp 73–80

  • Kächele M, Glodek M, Zharkov D, Meudt S, Schwenker F (2014) Fusion of audio-visual features using hierarchical classifier systems for the recognition of affective states and the state of depression. In: M. De Marsico, A. Tabbone, A. Fred (eds.) Proceedings of the International Conference on Pattern Recognition Applications and Methods (ICPRAM), SciTePress, pp 671–678

  • Kächele M, Schels M, Meudt S, Palm G, Schwenker F (2016) Revisiting the EmotiW challenge: how wild is it really? J Multimodal User Interfaces 10(2):151–162. doi:10.1007/s12193-015-0202-7

    Article  Google Scholar 

  • Kächele M, Schels M, Schwenker F (2014) Inferring depression and affect from application dependent meta knowledge. In: Proceedings of the 4th International Workshop on Audio/Visual Emotion Challenge, AVEC ’14, ACM, pp 41–48

  • Kächele M, Thiam P, Amirian M, Schwenker F, Palm G (2016) Methods for person-centered continuous pain intensity assessment from bio-physiological channels. IEEE J Sel Topics Signal Proces (99), pp 1–1

  • Kächele M, Thiam P, Amirian M, Werner P, Walter S, Schwenker F, Palm G (2015) Multimodal data fusion for person-independent, continuous estimation of pain intensity. In: Iliadis L, Jayne C (eds) Engineering Applications of Neural Networks, Communications in Computer and Information Science, vol. 517, pp. 275–285. Springer International Publishing

  • Kächele M, Thiam P, Palm G, Schwenker F, Schels M (2015) Ensemble methods for continuous affect recognition: Multi-modality, temporality, and challenges. In: Proceedings of the 5th International Workshop on Audio/Visual Emotion Challenge, AVEC ’15, ACM, pp 9–16

  • Kächele M, Werner P, Al-Hamadi A, Palm G, Walter S, Schwenker F (2015) Bio-visual fusion for person-independent recognition of pain intensity. In:  Schwenker F, Roli F, Kittler J (eds) Multiple classifier systems, Lecture notes in computer science, vol. 9132, pp 220–230. Springer International Publishing

  • Kaltwang S, Rudovic O, Pantic M (2012) Continuous pain intensity estimation from facial expressions. In: Bebis G, Boyle R, Parvin B, Koracin D, Fowlkes C, Wang S, Choi MH, Mantler S, Schulze J, Acevedo D, Mueller K, Papka M (eds) Advances in visual computing, LNCS, vol 7432. Springer, Berlin, pp 368–377

    Chapter  Google Scholar 

  • Kuncheva L (2004) Combining pattern classifiers: methods and algorithms. Wiley, Hoboken, New Jersey

  • Lucey P, Cohn JF, Prkachin KM, Solomon PE, Matthews I (2011) Painful data: the UNBC-McMaster shoulder pain expression archive database. In: IEEE International Conference on Automatic Face & Gesture Recognition and Workshops (FG 2011), IEEE, pp 57–64

  • Ludl MC, Lewandowski A, Dorffner G (2008) Adaptive machine learning in delayed feedback domains by selective relearning. Appl Artif Intell 22(6):543–557

    Article  Google Scholar 

  • Phinyomark A, Limsakul C, Phukpattaranont P (2009) A novel feature extraction for robust EMG pattern recognition. J Comput 1(1):71–80

    Google Scholar 

  • Platt JC (1999) Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. In: Advances in large margin classifiers, pp 61–74

  • Schels M, Glodek M, Meud, S, Scherer S, Schmidt M, Layher G, Tschechne S, Brosch T, Hrabal D, Walter S, Traue H, Palm G, Neumann H, Schwenker F (2013) Multi-modal classifier-fusion for the recognition of emotions, 1 edn. Coverbal synchrony in human-machine interaction. CRC Press, USA, pp 73–97

  • Schwenker F, Dietrich CR, Thiel C, Palm G (2006) Learning of decision fusion mappings for pattern recognition. Int J Artif Intell Mach Learn (AIML) 6:17–21

    Google Scholar 

  • Tkach D, Huang H, Kuiken TA (2010) Research study of stability of time-domain features for electromyographic pattern recognition. J Neuroeng Rehabil 7:21

    Article  Google Scholar 

  • Treister R, Kliger M, Zuckerman G, Aryeh IG, Eisenberg E (2012) Differentiating between heat pain intensities: the combined effect of multiple autonomic parameters. Pain 153(9):1807–1814

    Article  Google Scholar 

  • Walter S, Gruss S, Ehleiter H, Tan J, Traue H, Werner P, Al-Hamadi A, Crawcour S, Andrade A, Moreira da Silva G (2013) The BioVid heat pain database data for the advancement and systematic validation of an automated pain recognition system. In: IEEE International Conference on Cybernetics (CYBCONF), pp 128–131

  • Walter S, Gruss S, Limbrecht-Ecklundt K, Traue HC, Werner P, Al-Hamadi A. Diniz N, Silva GMd, Andrade AO (2014) Automatic pain quantification using autonomic parameters. Psychol Neurosci 7:363–380

    Article  Google Scholar 

  • Wang W, Enescu V, Sahli H (2015) Adaptive real-time emotion recognition from body movements. ACM Trans Interactive Intell Syst 5(4):18:1–18:21

    Google Scholar 

  • Werner P, Al-Hamadi A, Niese R, Walter S, Gruss S, Traue HC (2014) Automatic pain recognition from video and biomedical signals. In: International Conference on Pattern Recognition, pp 4582–4587

  • Wöllmer M, Kaiser M, Eyben F, Schuller B, Rigoll G (2013) LSTM-modeling of continuous emotions in an audiovisual affect recognition framework. Image Vision Comput 31(2):153–163

    Article  Google Scholar 

  • Yang J, Yan R, Hauptmann AG (2007) Cross-domain video concept detection using adaptive svms. In: Proceedings of the 15th ACM International Conference on Multimedia, MM ’07 , ACM pp 188–197

  • Zhang C Hamid R, Zhang Z (2008) Taylor expansion based classifier adaptation: application to person detection. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, pp 1–8

  • Zhao G, Pietikainen M (2007) Dynamic texture recognition using local binary patterns with an application to facial expressions. Pattern Analy Mach Intell IEEE Trans 29(6):915–928

    Article  Google Scholar 

  • Zhao Q, Zhang L (2005) ECG feature extraction and classification using wavelet transform and support vector machines. Int Conf Neural Netw Brain 2:1089–1092

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Neural Information Processing, Ulm University, 89069, Ulm, Germany

    Markus Kächele, Mohammadreza Amirian, Patrick Thiam, Günther Palm & Friedhelm Schwenker

  2. Institute of Information Technology, University of Magdeburg, Magdeburg, Germany

    Philipp Werner

  3. Department of Psychosomatic Medicine and Psychotherapy, Ulm University, Ulm, Germany

    Steffen Walter

Authors
  1. Markus Kächele
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohammadreza Amirian
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Patrick Thiam
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Philipp Werner
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Steffen Walter
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Günther Palm
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Friedhelm Schwenker
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Markus Kächele.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kächele, M., Amirian, M., Thiam, P. et al. Adaptive confidence learning for the personalization of pain intensity estimation systems. Evolving Systems 8, 71–83 (2017). https://doi.org/10.1007/s12530-016-9158-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12530-016-9158-4

Keywords

Search

Navigation