Skip to main content
SpringerLink
Log in

Subjective MOS model and simplified E-model enhancement for Skype associated with packet loss effects: a case using conversation-like tests with Thai users

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

This paper proposes two mathematical models that can be used to estimate VoIP quality from Skype, which is one of the most popular VoIP applications. The first model is simple, it has been developed using data from the informal interview tests called Conversation-like tests, referring to packet loss of 0 %, 5 %, 10 %, …, and 30 %. The tests have been conducted with Skype using a non ITU-T’s codec called SILK via the Internet with over 180 native Thai participants, while packet loss effects were generated using a network emulation tool. The second model is called the Enhanced Simplified E-model, this has been developed by adding the Thai Bias factor into a generic Simplified E-model, which calculates by subtracting the subjective results from the computed results using the Simplified E-model formula. After obtaining the models, they were evaluated with the Test set from 36 native Thai participants (different from the other group of participants) using Mean Absolute Percentage Error technique (MAPE). It has been found that VoIP quality measurement performance of both models are classified as excellent and provide higher reliability and accuracy than the Simplified E-model. Subjective MOS model and Enhanced Simplified E-model error reduction compared to the simplified one was at about 21.9 % and 21.2 % respectively, which is the major contribution of this work.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Arafat MY, Alam MM, Ahmed F (2014) SIP-based QoS in IP telephony. J Netw 9(12):3415–3426. doi:10.4304/jnw.9.12.3415-3426

    Google Scholar 

  2. Assem H (2013) Assessing and improving the VVoIP call quality. Thesis, National University of Ireland Maynooth

  3. Assem H, Adel M, Jennings B, Malone D, Dunne J, O’Sullivan P (2013) A generic algorithm for mid-call audio codec switching. In: Proceedings of IM 2013, Ghent, pp 1276–1281

  4. Assem H, Malone D, Dunne J, O’Sullivan P (2013) Monitoring VoIP call quality using improved simplified E-model. In: Proceedings of ICNC 2013, San Diego, pp 927–931

  5. Baharudin MAB, Quang TM, Kamiokal E (2015) Improvement of handover performance based on bio-inspired approach with received signal strength and mean opinion score. Arab J Sci Eng 40:1623–1636. doi:10.1007/s13369-015-1638-5

    Article  Google Scholar 

  6. Chu H-C, Lo C-H, Chao H-C (2013) The disclosure of and Android smartphone’s digital footprint respecting the Instant Messaging utilizing Skype and MSN. Electron Commer Res 13(3):399–410. doi:10.1007/s10660-013-9116-1

    Article  Google Scholar 

  7. Cole RG, Rosenbluth J (2001) Voice over IP performance monitoring. ACM Comput Commun Rev 31(2):9–24. doi:10.1145/505666.505669

    Article  Google Scholar 

  8. Daengsi T (2012) VoIP quality measurement: recommendation of MOS and enhanced objective measurement method for standard Thai spoken language. Dissertation, King Mongkut’s University of Technology North Bangkok

  9. Daengsi T, Khitmoh N, Wattidittachotti P (2015) VoIP quality measurement: subjective VoIP quality estimation model for G.711 and G.729 based on native Thai users. Multimed Syst. doi:10.1007/s00530-015-0468-3

    Google Scholar 

  10. Daengsi T, Wuttidittachotti P (2015) QoE modeling: a simplified e-model enhancement using subjective MOS estimation model. In: Proceedings of ICUFN 2015, Supporo, pp 386–390

  11. De Pessemier T, Stevens I, De Marez L, Martens L, Joseph W (2015) Analysis of the quality of experience of a commercial voice-over-IP service. Multimed Tools Appl 74(15):5873–5895. doi:10.1007/s11042-014-1895-4

    Article  Google Scholar 

  12. De Rango F, Tropea M, Fazio P, Marano S (2006) Overview on VoIP: subjective and objective measurement methods. Int J Comput Sci Netw Secur 6(1B):140–153

    Google Scholar 

  13. Ding L, Goubran RA (2003) Speech quality prediction in VoIP using the extended E-model. In: Proceedings of IEEE GLOBECOM, San Francisco, pp 3974–3978

  14. Gandour J, Wong D, Hutchins G (1998) Pitch processing in the human brain is influenced by language experience. NeuroReport 9:2115–2119. doi:10.1097/00001756-199806220-00038

    Article  Google Scholar 

  15. Goodness of Fit. Available from: http://uregina.ca/~gingrich/regr.pdf. Accessed Feb 2016

  16. Goudarzi M (2008) Evaluation of voice quality in 3G mobile networks. Thesis, University of Plymouth

  17. Goudarzi M, Sun L, Ifeachor EC (2011) Modelling speech quality for NB and WB SILK Codec for VoIP Applications. In: Proceedings of 5th NGMAST 2011, Cardiff, pp 42–47

  18. Hamidia M, Bakri A, Amrouche A (2015) Improving network echo cancellation in VoIP using packet loss concealment. In: Proceedings of IPAC’2015, Batna, Article No. 22. doi:10.1145/2816839.2816923

  19. Huang T-Y, Huang P, Chen K-T, Wang P-J (2010) Could Skype be more satisfying? a QoE-centric study of the FEC mechanism in an internet-scale VoIP system. IEEE Netw 24(2):42–48

    Article  Google Scholar 

  20. Huang Y-T, Neoh C-A, Lin S-Y, Shi H-Y (2013) Comparisons of prediction models of myofascial pain control after dry needling: a prospective study. Evidence-Based Complementary and Alternative Medicine. doi:10.1155/2013/478202

    Google Scholar 

  21. Ickin S, Fiedler M, Wac K, Arlos P, Temiz C, Mkocha K (2015) VLQoE: video QoE instrumentation on the smartphone. Multimed Tools Appl 74(2):381–411. doi:10.1007/s11042-014-1919-0

    Article  Google Scholar 

  22. ITU-T (1988) ITU-T Recommendation P.82, Methods for evaluation of service from the standpoint of speech transmission quality

  23. ITU-T (1996) ITU-T Recommendation P.800: Methods for subjective determination of transmission quality

  24. ITU-T (1996) ITU-T Recommendation P.800.1: Mean Opinion Score (MOS) terminology

  25. ITU-T (2006) ITU-T Recommendation P.10/G.100: vocabulary for performance and quality of service

  26. ITU-T (2007) ITU-T Recommendation P.805: subjective evaluation of conversational quality

  27. ITU-T (2008) ITU-T Recommendation G.107: the E-model: a computational model for use in transmission planning

  28. ITU-T (2011) ITU-T recommendation G.107: the E-model: a computational model for use in transmission planning

  29. Jana S, Chan A, Pande A, Mohapatra P (2015) QoE prediction model for mobile video telephony. Multimed Tools Appl. doi:10.1007/s11042-015-2711-5

    Google Scholar 

  30. Jana S, Pande A. Chan AJ, Mohapatra P (2013) Network characterization and perceptual evaluation of skype mobile videos. In: Proceedings of ICCCN 2013, Nassau, pp 1–7

  31. Jiang C, Huang P (2011) Research of monitoring VoIP voice QoS. In: Proceedings of ICICIS 2011, Hong Kong, pp 499–502

  32. Jiang W, Schulzrinne H (2002) Comparison and optimization of packet loss repair methods on VoIP perceived quality under bursty loss. In: Proceedings of NOSSDAV’02, Miami, pp 73–81

  33. Johannesson NO (1997) The ETSI computation model: a tool for transmission planning of telephone networks. IEEE Commun Mag 35(1):70–79. doi:10.1109/35.568213

    Article  Google Scholar 

  34. Karapantazis S, Pavlidou F-N (2009) VoIP: A comprehensive survey on a promising technology. Comput Netw 53(12):2050–2090. doi:10.1016/j.comnet.2009.03.010

    Article  Google Scholar 

  35. Lindeberg M, Kristiansen S, Plagemann T, Goebel V (2011) Challenges and techniques for video streaming over mobile ad hoc networks. Multimed Syst 17:51–82. doi:10.1007/s00530-010-0187-8

    Article  Google Scholar 

  36. Liu L, Sun L (2013) Performance analysis of voice call using skype. Available from: https://www.cscan.org/download/?id=924. Accessed Feb 2016

  37. Markopoulou A, Iannaccone G, Bhattacharyya S, Chuah C-N, Diot C (2004) Characterization of Failures in an IP Backbone. In: Proceedings of INFOCOM 2004. Hong Kong, pp 2307–2317

  38. Mazurczyk W, Karas M, Szczypiorski K, Janicki A (2015) YouSkyde: information hiding for Skype video traffic. Multimed Tools Appl. doi:10.1007/s11042-015-2740-0

    Google Scholar 

  39. Messenger Exposed: Investigation Report. Available from: https://webrtchacks.com/wp-content/uploads/2015/05/messenger-report.pdf. Accessed Feb 2016

  40. Michels BJ, Chang C-W (2011) Attending a presentation at a distance in real-time via Skype. TechTrends 55(1):23–27

    Article  Google Scholar 

  41. Moosikaphan K (2014) A comparative study of perceptual VoIP quality and development of subjective MOS estimation models referring to loss effects based on Thai users: Line vs Facebook. Thesis, King Mongkut’s University of Technology North Bangkok (in Thai)

  42. Narbutt M, Davis M (2005) Assessing the quality of VoIP transmission affected by playout buffer scheme. In: Proceedings of MESAQIN 2005, Prague, Paper 15

  43. National cultural dimensionshttp. Available from: http://geert-hofstede.com/national-culture.html. Accessed Feb 2016

  44. Okamoto M, Nose T, Ito A (2014) Subjective evaluation of packet loss recovery techniques for voice over IP. In: Proceedings of ICALIP 2014, Manchester, pp 497–501

  45. Psytechnics (2003) Comparison between subjective listening quality and P.862 PESQ score. Available at http://www.sageinst.com/downloads/960B/wp_sub_v_pesq.pdf. Accessed Feb 2016

  46. QoS: Quality of Service. Available from: http://www.cisco.com/web/offer/powernow/docs/netinfra/qos-cheatsheet.pdf. Accessed Feb 2016

  47. Quality Management: Troubleshooting Techniques for Voice over IP. Available from: http://assets.tequipment.net/assets/1/26/Documents/ApplicationNote-QualityManagement-TroubleshootingTechniquesForVoiceOverIP.pdf..Accessed Feb 2016

  48. Ren J, Zhang CM, Huang WC, Mao D (2010) Enhancement to E-model on standard deviation of packet delay. In: Proceedings of ICIS 2010, Chengdu, pp 256–259

  49. Ribadeneira A F (2007) An analysis of the MOS under conditions of delay, Jitter and packet loss and an analysis of the impact of introducing piggybacking and reed Solomon FEC for VOIP. Thesis, Georgia State University

  50. Rungruangthum M, Wuttidittachotti P, Daengsi T (2015) Simple MOS estimation model for skype referring to packet loss effects: development using conversation-like tests. In: Proceedings of APCC 2015 – Latest Results Workshop, Kyuto, pp 503–507

  51. Sathapornvajana S, Papasratorn B (2013) Effect of culture, Age, and language on quality of services and adoption of IP applications. Wirel Pers Commun 69(3):1067–1075. doi:10.1007/s11277-013-1066-3

    Article  Google Scholar 

  52. Sittiprapaporn W, Chindaduangratn C, Kotchabhakdi N (2004) Long-term memory traces for familiar spoken words in tonal languages as revealed by the Mismatch negativity. Songklanakarin J Sci Technol 26(6):779–786

    Google Scholar 

  53. Skype reaches 500 million installs on google play, becomes just the 4th company to do so. Available from: http://www.androidpolice.com/2015/02/26/skype-reaches-500-millions-installs-google-play-becomes-just-4th-company/. Accessed Feb 2016

  54. Skype’s incredible rise, in one image. Available from: http://blogs.wsj.com/corporate-intelligence/2014/01/15/skypes-incredible-rise-in-one-image/. Accessed Feb 2016

  55. Streijl RC, Winkler S, Hands D (2016) Mean opinion score (MOS) revisited: methods and applications limitations and alternatives. Multimed Syst 22(2):213–227. doi:10.1007/s00530-014-0446-1

    Article  Google Scholar 

  56. Sun L, Ifeachor EC (2004) New models for perceived voice quality prediction and their applications in playout buffer optimization for VoIP networks. In: Proceedings of ICC 2004, Paris, pp 1478–1483

  57. Triyason T, Kanthamanon P (2015) E-model modification for multi-languages over IP. Elektron Elektrotech 21(1):82–87. doi:10.5755/j01.eee.21.1.7612

    Google Scholar 

  58. Valin J-M, Maxwell G, Terriberry TB, Vos K (2013) High-quality, low-delay music coding in the opus codec. Available from: http://jmvalin.ca/papers/aes135_opus_celt.pdf. Accessed July 2016

  59. Vos K, Jensen S, Soerensen K (2010) SILK speech codec. Available from: https://tools.ietf.org/html/draft-vos-silk-01. Accessed July 2016

  60. Vos K, Jensen SS, Sørensen KV (2009) SILK Speech/Audio Codec. In: Proceedings of IETF 75, Stockholm. Available from: http://www.ietf.org/proceedings/75/slides/codec-3.pdf. Accessed Feb 2016

  61. Voznak M (2011) E-model modification for case of cascade codecs arrangement. Int J Math Mod Meth Appl S 5(8):1301–1309

    Google Scholar 

  62. What you need to know about Skype? Available from: https://whitepages.co.nz/pages/what-you-need-know-about-skype/. Accessed Feb 2016

  63. Wuttidittachotti P, Akapan W, Daengsi T (2015) Comparison of VoIP-QoE from Skype, LINE, Tango and viber over 3G networks in Thailand. In: Proceedings of ICUFN 2015, Supporo, pp 456–461

  64. Wuttidittachotti P, Daengsi T (2015) Quality evaluation of mobile networks using VoIP applications: a case study with Skype and LINE based-on stationary tests in Bangkok. Int J Comput Netw Inform Secur 7(12):28–41. doi:10.5815/ijcnis.2015.12.04

    Google Scholar 

  65. Wuttidittachotti P, Daengsi T (2016) VoIP-quality of experience modeling: E-model and simplified Emodel enhancement using bias factor. Multimed Tools Appl. doi:10.1007/s11042-016-3389-z

  66. Wuttidittachotti P, Khaoduang P, Daengsi T (2014) Development of a MOS estimation model for G.729 using listening-opinion tests with Thai speech referring to packet loss effects. In: Proceedings of ISCAIE 2014, Penang, pp 29–32

  67. Xiaomin L (2011) Performance evaluation of speech quality for VoIP on the internet. Thesis, National University of Singapore

  68. Zhang W, Chang Y, Liu Y, Hameed A, Tian Y (2014) Study on the QoE for VoIP networks. J Networks 9(2):515–522. doi:10.4304/jnw.9.2.515-522

    Google Scholar 

  69. Zhang H, Xie L, Byun J, Flynn P, Shim Y (2005) Packet loss burstiness and enhancement to the E-Model. In: Proceedings of SNPD/SAWN 2005. Towson University, pp 214–219

  70. Zhang X, Xu Y, Hu H, Liu Y, Guo Z, Wang Y (2012) Profiling Skype video calls: rate control and video quality. In: Proceedings of INFOCOM 2012. Orlando, pp 621–629

  71. Zhou X, Muller F, Kooij RE, Van Mieghem P (2006) Estimation of voice over IP quality in the Netherlands. In: Proceedings of IPS-MoMe 2006, Salzburg

  72. Zhou L, Yang Z, Wen Y, Wang H, Guizani M (2013) Resource allocation with incomplete information for QoE-driven multimedia communications. IEEE Trans Wirel Commun 12(8):3733–3745

    Article  Google Scholar 

  73. Zhou L, Yang Z, Wen Y, Rodrigues JJPC (2014) Distributed wireless video scheduling with delayed control information. IEEE Trans Circ Syst Video Technol 24(5):889–901

    Article  Google Scholar 

Download references

Acknowledgments

Thank you to all reviewers for very useful comments. Thank you to all participants, staff and lecturers at KMUTNB, who were involved. Particularly, thanks to Mr. Montri Rungruangthum, a master degree student in Faculty of Information Technology, KMUTNB, for conducting tests and data collection. Thanks Mr. Gary Sherriff for English editing.

Author information

Authors and Affiliations

  1. Department of Data Communication and Networking, Faculty of Information Technology, King Mongkut’s University of Technology North Bangkok, Bangkok, Thailand

    Pongpisit Wuttidittachotti

  2. Department of Enterprise Services, JADS Comm Limited, Bangkok, Thailand

    Therdpong Daengsi

Authors
  1. Pongpisit Wuttidittachotti
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Therdpong Daengsi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pongpisit Wuttidittachotti.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wuttidittachotti, P., Daengsi, T. Subjective MOS model and simplified E-model enhancement for Skype associated with packet loss effects: a case using conversation-like tests with Thai users . Multimed Tools Appl 76, 16163–16187 (2017). https://doi.org/10.1007/s11042-016-3901-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-016-3901-5

Keywords

Search

Navigation