Skip to main content
SpringerLink
Log in

QoE assessment of interactive applications in computer networks

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Measuring end user Quality of Experience (QoE) is currently performed by subjective or objective standard methods each with its own deficiencies. The subjective quality assessment is laboratory based, costly and offline; while, the objective estimation of user satisfaction is obtained through a static manner, not directly related to end user contentment. The attempt is made here to measure user QoE based on an online, user-aware and non-intrusive method. This is investigated by identifying the measurable objective indicators of user satisfaction/dissatisfaction and assigning them to the subjective nature of QoE concept. Proposing an architectural model, the extent of modalities for implicit sensing of the user QoE is explored with respect to the real-time measurement of her/his experiences. The vocal and interactional signs of VoIP service users on their smartphone devices are applied to estimate their satisfaction/dissatisfaction levels as a case study. The obtained results are compared to the users’ self-report in order to evaluate the accuracy of this proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. (2013) AndroSensor, Android Apps on Google Play. Available: https://play.google.com/store/apps/details?id=com.fivasim.androsensor&hl=en

  2. (2013) “Measure emotion.” Facial Expression Analysis Recognition. Webpage. Available: http://www.affdex.com/

  3. Afshari S, Movahhedinia N (2014) “Non-intrusive online quality of experience assessment for voice communications,” Wireless Personal Communications, pp. 1–16, 2014/08/02 2014

  4. Bhattacharya A, Wu W, Yang Z (2012) Quality of experience evaluation of voice communication: an affect-based approach. Human-centric Comput and Inf Sci 2:1–18

    Article  Google Scholar 

  5. Bhaykar M, Yadav J, Rao KS (2013) “Speaker dependent, speaker independent and cross language emotion recognition from speech using GMM and HMM,” in Communications (NCC), 2013 National Conference on, pp. 1–5

  6. Brooks P, Hestnes B (2010) User measures of quality of experience: why being objective and quantitative is important. Netw IEEE 24:8–13

    Article  Google Scholar 

  7. Burkhardt F, Paeschke A, Rolfes M, Sendlmeier WF, Weiss B (2005) “A database of German emotional speech,” in Interspeech, pp. 1517–1520

  8. Calvo RA, D’Mello S (2010) Affect detection: an interdisciplinary review of models, methods, and their applications. Affect Comput IEEE Trans on 1:18–37

    Article  Google Scholar 

  9. Chang C-C, Lin C-J (2011) LIBSVM: a library for support vector machines. ACM Trans on Intell Syst and Technol (TIST) 2:27

    Google Scholar 

  10. Chen-Chi W, Kuan-Ta C, Yu-Chun C, Chin-Laung L (2013) Crowdsourcing multimedia QoE evaluation: a trusted framework. Multimedia IEEE Trans on 15:1121–1137

    Article  Google Scholar 

  11. Choudhury T, Consolvo S, Harrison B, Hightower J, LaMarca A, LeGrand L et al (2008) The mobile sensing platform: an embedded activity recognition system. Pervasive Comput IEEE 7:32–41

    Article  Google Scholar 

  12. Ciubotaru B, Muntean GM, Ghinea G (2009) Objective assessment of region of interest-aware adaptive multimedia streaming quality. Broadcast IEEE Trans on 55:202–212

    Article  Google Scholar 

  13. Dai Q (2011) “A survey of quality of experience” Energy-Aware Communications, pp. 146–156.

  14. De Moor K, Ketyko I, Joseph W, Deryckere T, De Marez L, Martens L et al (2010) Proposed framework for evaluating quality of experience in a mobile, testbed-oriented living lab setting. Mob Networks and Appl 15:378–391

    Article  Google Scholar 

  15. De Pessemier T, De Moor K, Joseph W, De Marez L, Martens L (2013) Quantifying the influence of rebuffering interruptions on the user’s quality of experience during mobile video watching. IEEE Trans Broadcast 59:47–61

    Article  Google Scholar 

  16. Eyben F, Weninger F, Gross F, Schuller B (2013) “Recent developments in openSMILE, the munich open-source multimedia feature extractor,” in Proceedings of the 21st ACM international conference on Multimedia, pp. 835–838

  17. Fiedler M, Hossfeld T, Phuoc T-G (2010) A generic quantitative relationship between quality of experience and quality of service. Netw IEEE 24:36–41

    Article  Google Scholar 

  18. Geerts D, De Moor K, Ketyko x I, Jacobs A, et al. (2010) “Linking an integrated framework with appropriate methods for measuring QoE,” in Quality of Multimedia Experience (QoMEX), 2010 Second International Workshop on, pp. 158–163

  19. Ghinea G, Angelides MC (2004) A user perspective of quality of service in m-commerce. Multimedia Tools and Appl 22:187–206

    Article  Google Scholar 

  20. Gulliver SR, Ghinea G (2006) Defining user perception of distributed multimedia quality. ACM Trans on Multimedia Comput, Commun, and Appl (TOMCCAP) 2:241–257

    Article  Google Scholar 

  21. Gulliver SR, Ghinea G (2007) The perceptual and attentive impact of delay and jitter in multimedia delivery. Broadcast IEEE Trans on 53:449–458

    Article  Google Scholar 

  22. Gustarini M, Ickin S, Wac K (2013) “Evaluation of challenges in human subject studies in-the-wild using subjects personal smartphones,” in Proceedings of the 2013 ACM conference on Pervasive and ubiquitous computing adjunct publication, pp. 1447–1456

  23. Hasani HR, Movahhedinia N, Ghahfarokhi BS (2011) “Evaluation of user perceived quality based on user behavior,” in Computational Intelligence and Informatics (CINTI), 2011 I.E. 12th International Symposium on, pp. 513–517

  24. Hektner JM, Schmidt JA, Csikszentmihalyi M (2007) Experience sampling method: measuring the quality of everyday life

  25. ITU-T (1996) “Recommendation ITU-T P.920: Interactive test methods for audiovisual communications,” International Telecommunication Union

  26. ITU-T (2008) Recommendation ITU-T P.10/G.100, Amendment 2, “New definitions for inclusion in Re”commendation ITU-T P.10/G.100", International Telecommunication Union

  27. ITU-T, “Recommendation P.800, Methods for subjective determination of transmission quality,” International Telecommunication Union-Telecommunication Standardisation Sector (ITU-T), 1996

  28. Jingjing Z, Ansari N (2011) On assuring end-to-end QoE in next generation networks: challenges and a possible solution. Commun Mag IEEE 49:185–191

    Article  Google Scholar 

  29. Jumisko-Pyykkö S, Weitzel M, Strohmeier D (2008) “Designing for user experience: what to expect from mobile 3D TV and video?,” in Proceedings of the 1st international conference on Designing interactive user experiences for TV and video, pp. 183–192

  30. Kane J, Scherer S, Aylett M, Morency LP, Gobl C (2013) “Speaker and language independent voice quality classification applied to unlabelled corpora of expressive speech,” in Acoustics, Speech and Signal Processing (ICASSP), 2013 I.E. International Conference on, pp. 7982–7986

  31. Ketyko I, De Moor K, Joseph W, Martens L, De Marez L (2010) “Performing QoE-measurements in an actual 3G network,” in Broadband Multimedia Systems and Broadcasting (BMSB), 2010 I.E. International Symposium on, pp. 1–6

  32. Knapp RB, Kim J, André E (2011) “Physiological signals and their use in augmenting emotion recognition for human–machine interaction,” in Emotion-Oriented Systems, ed: Springer, pp. 133–159

  33. Kuan-Ta C, Cheng-Chun T, Wei-Cheng X (2009) “OneClick: a framework for measuring network quality of experience,” in INFOCOM 2009, IEEE, pp. 702–710

  34. Kuan-Ta C, Chi-Jui C, Chen-Chi W, Yu-Chun C, Chin-Laung L (2010) Quadrant of euphoria: a crowdsourcing platform for QoE assessment. Netw IEEE 24:28–35

    Article  Google Scholar 

  35. Laghari KUR, Khan I, Crespi N (2012) “Quantitative and qualitative assessment of QoE for multimedia services in wireless environment,” in Proceedings of 4th ACM Workshop on Mobile Video in conjunction with ACM Multimedia Systems Conference 2012, pp. 7–12

  36. Lee CM, Narayanan SS (2005) Toward detecting emotions in spoken dialogs. Speech and Audio Process IEEE Trans on 13:293–303

    Article  Google Scholar 

  37. Linphone [http://www.linphone.org/]

  38. Matulin M, Mrvelj Š (2013) State-of-the-practice in evaluation of quality of experience in real-life environments. PROMET-Traffic & Transp 25:255–263

    Google Scholar 

  39. McNamara N, Kirakowski J (2011) Measuring user-satisfaction with electronic consumer products: the consumer products questionnaire. Int J of Human-Comput Stud 69:375–386

    Article  Google Scholar 

  40. Menkovski V, Exarchakos G, Liotta A, Sánchez AC (2010) Quality of experience models for multimedia streaming. Int J Mobile Comput Multimedia Commun (IJMCMC) 2:1–20

    Article  Google Scholar 

  41. Mitra K, Zaslavsky A, Åhlund C (2011) “Dynamic Bayesian networks for sequential quality of experience modelling and measurement,” Smart Spaces and Next Generation Wired/Wireless Networking, pp. 135–146

  42. Moller S, Engelbrecht KP, Kuhnel C, Wechsung I, Weiss B (2009) “A taxonomy of quality of service and quality of experience of multimodal human-machine interaction,” in Quality of Multimedia Experience, 2009. QoMEx 2009. International Workshop on, pp. 7–12

  43. Muntean G, Ghinea G, Sheehan TN (2008) Region of interest-based adaptive multimedia streaming scheme. Broadcast IEEE Trans on 54:296–303

    Article  Google Scholar 

  44. Picard RW (2003) Affective computing: challenges. Int J Human-Comput Stud 59:55–64

    Article  Google Scholar 

  45. Ping HY, Abdullah LN, Halin AA, Sulaiman PS (2013) A study of physiological signals-based emotion recognition systems. Int J Comput & Technol 11:2189–2196

    Google Scholar 

  46. Raake A (2007) Speech quality of VoIP: assessment and prediction: Wiley. com

  47. Reiter U (2011) “Overall perceived audiovisual quality-what people pay attention to,” in Consumer Electronics (ISCE), 2011 I.E. 15th International Symposium on, pp. 513–517

  48. Schatz R, Egger S, Platzer A (2011) “Poor, good enough or even better? bridging the gap between acceptability and qoe of mobile broadband data services,” in Communications (ICC), 2011 I.E. International Conference on, pp. 1–6

  49. Schatz R, Hoßfeld T, Janowski L, Egger S (2013) “From Packets to people: quality of experience as a new measurement challenge,” in Data Traffic Monitoring and Analysis, ed: Springer, pp. 219–263

  50. Serif T, Ghinea G (2008) Mobile information access in the real world: a story of three wireless devices. Comput Hum Behav 24:1385–1403

    Article  Google Scholar 

  51. Serif T, Gulliver SR, Ghinea G (2004) “Infotainment across access devices: the perceptual impact of multimedia QoS,” in Proceedings of the 2004 ACM symposium on Applied computing, pp. 1580–1585

  52. Shahin IMA (2013) “Employing both gender and emotion cues to enhance speaker identification performance in emotional talking environments,” International Journal of Speech Technology, pp. 1–11

  53. Shahin IMA (2013) “Gender-dependent emotion recognition based on HMMs and SPHMMs,” International Journal of Speech Technology, pp. 1–9

  54. Skorin-Kapov L, Varela M (2012) “A multi-dimensional view of QoE: the ARCU model,” in MIPRO, 2012 Proceedings of the 35th International Convention, pp. 662–666

  55. Song W, Tjondronegoro DW, Docherty M (2012) “Understanding user experience of mobile video: framework, measurement, and optimization,” in Mobile Multimedia: User and Technology Perspectives, D. W. Tjondronegoro, Ed., ed: INTECH Open Access Publisher, pp. 3–30

  56. Staelens N, Moens S, Van den Broeck W, Marien I, Vermeulen B, Lambert P et al (2010) Assessing quality of experience of IPTV and video on demand services in real-life environments. Broadcast IEEE Trans on 56:458–466

    Article  Google Scholar 

  57. Stankiewicz R, Jajszczyk A (2011) A survey of QoE assurance in converged networks. Comput Netw 55:1459–1473

    Article  Google Scholar 

  58. Takahashi A, Hands D, Barriac V (2008) Standardization activities in the ITU for a QoE assessment of IPTV. Commun Mag IEEE 46:78–84

    Article  Google Scholar 

  59. Wu W, Arefin A, Rivas R, Nahrstedt K, Sheppard R, Yang Z (2009) Quality of experience in distributed interactive multimedia environments: toward a theoretical framework, presented at the proceedings of the 17th ACM international conference on multimedia. Beijing, China

    Book  Google Scholar 

  60. Xu C, Zhao F, Guan J, Zhang H, Muntean G-M (2013) QoE-driven user-centric VoD services in urban multihomed P2P-based vehicular networks. Veh Technol IEEE Trans on 62:2273–2289

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Engineering Department, Faculty of Engineering, University of Isfahan, Isfahan, Iran

    Saeid Afshari & Naser Movahhedinia

Authors
  1. Saeid Afshari
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Naser Movahhedinia
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Saeid Afshari.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Afshari, S., Movahhedinia, N. QoE assessment of interactive applications in computer networks. Multimed Tools Appl 75, 903–918 (2016). https://doi.org/10.1007/s11042-014-2331-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-014-2331-5

Keywords

Search

Navigation