skip to main content
10.1145/1409240.1409269acmotherconferencesArticle/Chapter ViewAbstractPublication PagesmobilehciConference Proceedingsconference-collections
research-article

Usability benchmark study of commercially available smart phones: cell phone type platform, PDA type platform and PC type platform

Authors Info & Claims
Published:02 September 2008Publication History

ABSTRACT

The upcoming of smart phones are the result of consumers' preference for high-feature products: manufacturers are lured into integration of a growing number of technologies and features to provide attractive and competitive models. At the same time usability of such complex products becomes an increasing problem. This study aims to provide designers more insight into the consequences of emerging technologies on the usability of smart phones with different platforms. We conducted a usability benchmark study and tested 4 different features on 3 smart phones with 3 different platforms (Nokia E6li (Symbian S60, cell phone type platform), HTC S710 (Windows Mobile 6, PC type platform), Palm Treo680 (Palm OS, PDA type platform)) with in total 43 subjects in a between-subjects design. The results show significant differences in usability of the smart phones for the selected functions. For all platforms, the common design guidelines for usability still apply, but are restricted by the desire to integrate more functionality to create competitive products.

References

  1. Amant, R. St., Horton, T. E. and Ritter, F. E. Model-based evaluation of cell phone menu interaction. In Proc. CHI 2004, ACM Press (2004), 343--350. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Bay, S. and Ziefle, M. Children using cellular phones: The effects of shortcomings in user interface design. Human Factors 47, 1 (2005), 158--168.Google ScholarGoogle ScholarCross RefCross Ref
  3. Brown, C. L. and Carpenter, G. S. Why is trivial important? A reasons-based account for the effects of trivial attributes on choice. Journal of Consumer Research 26, (2000), 372--385.Google ScholarGoogle ScholarCross RefCross Ref
  4. Buchanan, G., Farrant, S., Jones, M., Thimbleby, H., Marsden, G. and Pazzani, M. Improving mobile internet usability. In Proc. 10th Int. Conf. on World Wide Web, ACM Press (2001), 673--680. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Chae, M. and Kim, J. Do size and structure matter to mobile users? An empirical study of the effects of screen size, information structure, and task complexity on user activities with standard web phones. Behaviour & Information Technology 23, 3 (2004), pp 165--181.Google ScholarGoogle Scholar
  6. Cockburn, A., Gutwin, C. and Greenberg, S. A predictive model of menu performance. In Proc. CHI 2007, ACM Press (2007), 627--636. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Cooper, A. The Inmates are Running the Asylum. Macmillan Computer Publishing, Indianapolis, USA, 1999. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Cooper, Robert G. Winning at new products: accelerating the process from idea to launch. 3rd edition. Perseus Publishing, New York, 2001.Google ScholarGoogle Scholar
  9. Cordell, V. V. Consumer knowledge measures as predictors in product evaluation. Psychology & Marketing 14, 3 (1997), 241--260.Google ScholarGoogle Scholar
  10. Den Ouden, E., Lu, Y., Sonnemans, P. J. M. and Brombacher, A. C. Quality and reliability problems from a consumer's perspective: an increasing problem overlooked by businesses? Quality and Reliability Engineering International 22, 7 (2006), 821--838.Google ScholarGoogle ScholarCross RefCross Ref
  11. Dumas, J. S. and Redish, J. C. A Practical Guide to Usability Testing, Intellect Books, Portland, USA, 1999. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Flynn, L. R. and Goldsmith, R. E. A short, reliable measure of subjective knowledge. Journal of Business Research 46, (1999) 57--66.Google ScholarGoogle ScholarCross RefCross Ref
  13. Goldstein, M., Anneroth, M. and Book, R. Usability evaluation of a high-fidelity smart phone prototype: Task navigation depth affects effectiveness. In Proc. of the HCI Int. 1999, ACM Press (1999), 38--42. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Gourville, J. T. Eager sellers and stony buyers - Understanding the psychology of new-product adoption, Harvard Business Review 84, (2006), 98--106.Google ScholarGoogle Scholar
  15. Hochheiser, H. and Shneiderman, B. Performance benefits of simultaneous over sequential menus as task complexity increases. International Journal of Human-Computer Interaction 12, 2 (2000), 173--192.Google ScholarGoogle ScholarCross RefCross Ref
  16. Holleis, P., Friederike, O., Hußmann, H. and Smidt, A. Keystroke-level model for advanced mobile phone interaction. In Proc. CHI 2007, ACM Press (2007), 1505--1514. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Howell, D. C. Statistical Methods for Psychology, Fifth Edition, Thompson Learning, Pacific Grove, 2002.Google ScholarGoogle Scholar
  18. ISO 9241-11:1998 Ergonomic Requirements for Office Work with Visual Display Terminals (VDTs) - Part 11: Guidance on Usability, 1998.Google ScholarGoogle Scholar
  19. Jeong, S.-H. and Lee, K.-P. The effects of experience with a PC on the usability of a mobile product, In Proc. 6th Asian Design International Conference, 2003.Google ScholarGoogle Scholar
  20. Kiljander, H. Evolution and Usability of Mobile Phone Interaction Styles. Thesis for Doctor of Science in Technology, Helsinki University of Technology, December 2004.Google ScholarGoogle Scholar
  21. Lewis, J. R. Psychometric evaluation of an after-scenario questionnaire for computer usability studies: The ASQ. SIGCHI Bulletin 23, 1 (1991), 78--81. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Malykhina, E. Analysis: How smartphone platforms compare. InformationWeek, 20 January 2007. http://www.informationweek.com/story/showArticle.jhtml?a rticleID=196902226 (accessed August 2007).Google ScholarGoogle Scholar
  23. Malykhina, E. Smartphone buyer's guide. InformationWeek, 30 October 2006. http://www.informationweek.com/story/showArticle.jhtml?a rticleID=193402812 (accessed August 2007).Google ScholarGoogle Scholar
  24. Mohageg, M. F. and Wagner, A. Design Considerations for Information Appliances. In Information Appliances and Beyond. Interaction Design for Consumer Products, ed. Bergman, E. Academic Press, San Diego, 2000, 27--51.Google ScholarGoogle Scholar
  25. Norman, D. A. The Invisible Computer. The MIT Press, Massachusetts, 1998. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Nyberg, M., Björk, S., Goldstein, M. and Redström, J. Handheld applications design: Merging information appliances without affecting usability. In Proc. IFIP TC. 13 Conference on Human Computer Interaction. IOP Press (2001), 391--398.Google ScholarGoogle Scholar
  27. Overton, D. 'No Fault Found' returns cost the mobile industry $4.5 billion per year. July 2006. http://www.wdsglobal.com/news/whitepapers/20060717/200 60717.asp (accessed August 2007).Google ScholarGoogle Scholar
  28. Rust, R. T., Thompson, D. V., Hamilton, R. W. Defeating Feature Fatigue, Harvard Business Review 84, 2 (2006), 98--107.Google ScholarGoogle Scholar
  29. Svendsen, B. Mobile computing. In True Visions, The emergence of ambient intelligence, Aarts, E. and Encarnação, J. L. (Eds.), Berlin, Springer-Verlag, 2006.Google ScholarGoogle Scholar
  30. Tarasewich, P., Wireless devices for mobile commerce: User interface design and usability. In Mobile Commerce: Technology, Theory and Applications, ed. Mennecke, B.E and Strader, T. J. Idea Group Publishing, Hershey, 2002, 26--50. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Thompson, D. V., Hamilton, R. W. and Rust, R. T. Feature fatigue: When product capabilities become too much of a good thing. Journal of Marketing Research 42, 4 (2005), 431--442.Google ScholarGoogle ScholarCross RefCross Ref
  32. Weiss, S., Handheld Usability, John Willey & Sons, LTD, Chichester, 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Zhang, D. and Adipat, B. Challenges, methodologies, and issues in the usability testing of mobile applications. International Journal of Human-Computer Interaction 18, 3 (2005), 293--308.Google ScholarGoogle ScholarCross RefCross Ref
  34. Zheng, P. and Ni, L. M. Smart Phone & Next Generation Mobile Computing. Morgan Kaufmann Publishers, San Francisco, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Ziefle, M. and Bay, S. How to overcome disorientation in mobile phone menus: A comparison of two different types of navigation aids. Human-Computer Interaction 21, (2006), 393--433. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Ziefle, M., Bay, S. and Schwade, A. On keys' meanings and modes: The impact of different key solutions on children's efficiency using a mobile phone. Behaviour & Information Technology 25, 5 (2006), 413--431.Google ScholarGoogle Scholar
  37. Ziefle, M. and Bay, S. How older adults meet complexity: aging effects on the usability of different mobile phones. Behaviour & Information Technology 24, 5 (2005), 375--389.Google ScholarGoogle Scholar
  38. Ziefle, M., and Bay, S. Mental models of a cellular phone menu. Comparing older and younger novice users. In Proc. MobileCHI 2004, Springer-Verlag (2004), 25--37.Google ScholarGoogle ScholarCross RefCross Ref
  39. Ziefle, M. The influence of user expertise and phone complexity on performance, ease of use and learnability of different mobile phones. Behaviour & Information Technology 21, 5 (2002), 303--311Google ScholarGoogle Scholar

Index Terms

  1. Usability benchmark study of commercially available smart phones: cell phone type platform, PDA type platform and PC type platform

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in
    • Published in

      cover image ACM Other conferences
      MobileHCI '08: Proceedings of the 10th international conference on Human computer interaction with mobile devices and services
      September 2008
      568 pages
      ISBN:9781595939524
      DOI:10.1145/1409240

      Copyright © 2008 ACM

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 2 September 2008

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article

      Acceptance Rates

      Overall Acceptance Rate202of906submissions,22%

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader