Skip to main content
SpringerLink
Log in

Validation of a usability assessment instrument according to the evaluators’ perspective about the users’ performance

  • Long Paper
  • Published:
Universal Access in the Information Society Aims and scope Submit manuscript

Abstract

Technologies for ageing in place may help older adults in their homes to overcome multiple impairments and to promote their autonomy and independence. When conceptualizing technologies for ageing in place, the International Classification of Functioning, Disability and Health (ICF) is a key element due to its functioning and disability framework with consolidated concepts and terminologies. Based on the ICF conceptual framework, the article presents the ICF based Usability Scale (ICF-US) to evaluate the usability of technologies for ageing in place and reports a study aiming: (1) to validate the ICF-US to evaluate usability according to the evaluators’ perspective about the users’ performance and (2) to evaluate the utility and applicability of the ICF-US. Two observational studies evolving 184 participants were conducted to: (1) assess the validity and reliability of the ICF-US to evaluate usability according to the evaluators’ perspective about the users’ performance and (2) to verify the utility and applicability of the ICF-US. The results suggest that the ICF-US is valid and reliable and can be used in different stages of technological developments without losing its discriminatory capacity.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. World Health Organization, Growing older-Staying Well. Ageing and physical activity in everyday life, Geneve (1998)

  2. World Health Organization, Health Promotion Glossary, Genebra (1998)

  3. World Health Organization, Active ageing: a policy framework. A contribution of the World Health Organization to the second United Nations World Assembly on Aging, Genebra (2002)

  4. Connelly, K., ur Rehman Laghari, K., Mokhtari, M., Falk, T.H.: Approaches to understanding the impact of technologies for aging in place: a mini-review. Gerontology 60(3), 282–288 (2014)

    Google Scholar 

  5. Pastalan, L.A.: Aging in place: the role of housing and social supports. Haworth Press, Philadelphia (1990)

    Google Scholar 

  6. World Health Organization. International classification of functioning, disability and health: ICF. Geneva: World Health Organization (2001)

    Google Scholar 

  7. Davey, J.A.: New Zealand Institute for Research on Ageing, Accommodation options for older people in Aotearoa/New Zealand. Centre for Housing Research (2004)

  8. World Health Organization, Global Age-friendly cities: a guide, Geneva (2007)

  9. Walker, P.R.: Report from the world NGO forum on ageing and the united nations second world assembly on ageing (2002)

  10. Korhonen, I., Bardram, J.E.: Guest editorial: introduction to the special section on pervasive healthcare. IEEE Trans. Inf. Technol. Biomed. 8(3), 229–234 (2004)

    Google Scholar 

  11. Korhonen, I., Parkka, J., Van Gils, M.: Health monitoring in the home of the future. IEEE Eng. Med. Biol. Mag. 22(3), 66–73 (2003)

    Google Scholar 

  12. Boulos, M.N.K., Brewer, A.C., Karimkhani, C., Buller, D.B., Dellavalle, R.P.: Mobile medical and health apps: state of the art, concerns, regulatory control and certification. Online J. Public Health Inform. 5(3), 229 (2014)

    Google Scholar 

  13. Mosa, A.S.M., Yoo, I., Sheets, L.: A systematic review of healthcare applications for smartphones. BMC Med. Inform. Decis. Mak. 12(1), 67 (2012)

    Google Scholar 

  14. Rashidi, P., Mihailidis, A.: A survey on ambient-assisted living tools for older adults. IEEE J. Biomed. Heal. Inform. 17(3), 579–590 (2013)

    Google Scholar 

  15. Queirós, A., Silva, A., Alvarelhão, J., Rocha, N.P., Teixeira, A.: Usability, accessibility and ambient-assisted living: a systematic literature review. Int. J. UAIS 14(1), 57–66 (2015)

    Google Scholar 

  16. Suzuki, T., Murase, S., Tanaka, T., Okazawa, T.: New approach for the early detection of dementia by recording in-house activities. Telemed. e-Health 13(1), 41–44 (2007)

    Google Scholar 

  17. Or, C.K.L., Tao, D.: Does the use of consumer health information technology improve outcomes in the patient self-management of diabetes? A meta-analysis and narrative review of randomized controlled trials. Int. J. Med. Inform. 83(5), 320–329 (2014)

    Google Scholar 

  18. Farmer, A.J., McSharry, J., Rowbotham, S., McGowan, L., Ricci-Cabello, I., French, D.P.: Effects of interventions promoting monitoring of medication use and brief messaging on medication adherence for people with Type 2 diabetes: a systematic review of randomized trials. Diabet. Med. 33(5), 565–579 (2016)

    Google Scholar 

  19. Huang, Z., Tao, H., Meng, Q., Jing, L.: Management of endocrine disease: effects of telecare intervention on glycemic control in type 2 diabetes: a systematic review and meta-analysis of randomized controlled trials. Eur. J. Endocrinol. 172(3), R93–R101 (2015)

    Google Scholar 

  20. Hunt, C.W.: Technology and diabetes self-management: an integrative review. World J. Diabetes 6(2), 225 (2015)

    Google Scholar 

  21. Riazi, H., Larijani, B., Langarizadeh, M., Shahmoradi, L.: Managing diabetes mellitus using information technology: a systematic review. J. Diabetes Metab. Disord. 14, 49 (2015)

    Google Scholar 

  22. Tildesley, H.D., Po, M.D., Ross, S.A.: Internet blood glucose monitoring systems provide lasting glycemic benefit in Type 1 and 2 diabetes. Med. Clin. North Am. 99(1), 17–33 (2015)

    Google Scholar 

  23. Arambepola, C., Ricci-Cabello, I., Manikavasagam, P., Roberts, N., French, D.P., Farmer, A.: The Impact of automated brief messages promoting lifestyle changes delivered via mobile devices to people with Type 2 diabetes: a systematic literature review and meta-analysis of controlled trials. J. Med. Internet Res. 18(4), e86 (2016)

    Google Scholar 

  24. Nakamura, N., Koga, T., Iseki, H.: A meta-analysis of remote patient monitoring for chronic heart failure patients. J. Telemed. Telecare 20(1), 11–17 (2014)

    Google Scholar 

  25. Conway, A., Inglis, S.C., Clark, R.A.: Effective technologies for noninvasive remote monitoring in heart failure. Telemed. e-Health 20(6), 531–538 (2014)

    Google Scholar 

  26. Grustam, A.S., Severens, J.L., van Nijnatten, J., Koymans, R., Vrijhoef, H.J.M.: Cost-effectiveness of telehealth interventions for chronic heart failure patients: a literature review. Int. J. Technol. Assess. Health Care 30(01), 59–68 (2014)

    Google Scholar 

  27. Pedone, C., Lelli, D.: Systematic review of telemonitoring in COPD: an update. Pneumonol. Alergol. Pol. 83(6), 476–484 (2015)

    Google Scholar 

  28. Lundell, S., Holmner, Å., Rehn, B., Nyberg, A., Wadell, K.: Telehealthcare in COPD: a systematic review and meta-analysis on physical outcomes and dyspnea. Respir. Med. 109(1), 11–26 (2015)

    Google Scholar 

  29. Alcañiz, M., Botella, C., Baños, R.M., Zaragoza, I., Guixeres, J.: The Intelligent e-Therapy system: a new paradigm for telepsychology and cybertherapy. Br. J. Guid. Counc. 37(3), 287–296 (2009)

    Google Scholar 

  30. Singh, D., Kropf, J., Hanke, S., Holzinger, A.: Ambient Assisted Living Technologies from the Perspectives of Older People and Professionals, pp. 255–266. Springer, Cham (2017)

    Google Scholar 

  31. Holzinger, A., Röcker, C., Ziefle, M.: From smart health to smart hospitals. In: Smart Health: State-of-the-Art and Beyond, LNCS, Lecture Notes in Computer Science, pp. 1–20 (2015)

  32. Rosser, B.A., Eccleston, C.: Smartphone applications for pain management. J. Telemed. Telecare 17(6), 308–312 (2011)

    Google Scholar 

  33. Cruz, V.T., et al.: A rehabilitation tool designed for intensive web-based cognitive training: description and usability study. JMIR Res. Protoc. 2(2), e59 (2013)

    Google Scholar 

  34. Roehrs, A., da Costa, C.A., da Righi, R., de Oliveira, K.S.F.: Personal health records: a systematic literature review. J. Med. Internet Res. 19(1), e13 (2017)

    Google Scholar 

  35. Chen, Y.-R.R., Schulz, P.J.: The effect of information communication technology interventions on reducing social isolation in the elderly: a systematic review. J. Med. Internet Res. 18(1), e18 (2016)

    Google Scholar 

  36. Matthews, J., Win, K.T., Oinas-Kukkonen, H., Freeman, M.: Persuasive technology in mobile applications promoting physical activity: a systematic review. J. Med. Syst. 40(3), 72 (2016)

    Google Scholar 

  37. Shingleton, R.M., Palfai, T.P.: Technology-delivered adaptations of motivational interviewing for health-related behaviors: a systematic review of the current research. Patient Educ. Couns. 99(1), 17–35 (2016)

    Google Scholar 

  38. Hoppe, A.: Technological stress: mental strain of younger and older users if technology fails. Springer, Berlin (2011)

    Google Scholar 

  39. Holzinger, A., Searle, G., Wernbacher, M.: The effect of previous exposure to technology on acceptance and its importance in usability and accessibility engineering. Univers. Access Inf. Soc. 10(3), 245–260 (2011)

    Google Scholar 

  40. Holzinger, A., Schaupp, K., Eder-Halbedl, W.: An investigation on acceptance of ubiquitous devices for the elderly in a geriatric hospital environment: using the example of person tracking. In: Velasco, C., Weber, G., Barroso, J., Mohamad, Y., Paredes, H. (eds.) Computers helping people with special needs. Springer, Berlin, pp. 22–29 (2008)

    Google Scholar 

  41. Lewis, J.R.: Psychometric evaluation of the PSSUQ using data from five years of usability studies. Int. J. Hum. Comput. Interact. 14(3&4), 463–488 (2002)

    Google Scholar 

  42. Brooke, J.: SUS-A quick and dirty usability scale. In: Jordan, P., Thomas, B., Weerdmeester, B., McClelland, I. (eds.) Usability evaluation in industry, pp. 189–194. Taylor and Francis, London (1996)

    Google Scholar 

  43. Lewis, J.R.: Psychometric evaluation of the post-study system usability questionaire: the PSSUQ. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, pp. 1259–1263 (1992)

  44. Rosa, A.F., Martins, A.I., Costa, V., Queirós, A., Silva, A., Rocha, N.P.: European Portuguese validation of the post-study system usability questionnaire (PSSUQ). In: 2015 10th Iberian Conference on Information Systems and Technologies, CISTI 2015 (2015)

  45. Bangor, A., Kortum, P.T., Miller, J.T.: An empirical evaluation of the System Usability Scale. Int. J. Hum. Comput. Interact. 24(6), 574–594 (2008)

    Google Scholar 

  46. Martins, A.I., Rosa, A.F., Queirós, A., Silva, A., Rocha, N.P.: European Portuguese validation of the system usability scale (SUS). In: Procedia Computer Science, vol. 67 (2015)

  47. Fernández, L.K.: Valoración geriátrica integral. El Resid. 5(2), 55–65 (2010)

    Google Scholar 

  48. Netto, M.P., Ponte, J.: Envelhecimento: desafio na transição do século. In: Netto, M.P. (ed.) Gerontologia—a Velhice e o Envelhecimento em Visão Globalizada, 1st edn, pp. 3–12. Atheneu, São Paulo (2000)

    Google Scholar 

  49. Keller, B.K., Morton, J.L., Thomas, V.S., Potter, J.F.: The effect of visual and hearing impairments on functional status. J. Am. Geriatr. Soc. 47(11), 1319–1325 (1999)

    Google Scholar 

  50. Debevc, M., Kosec, P., Holzinger, A.: Improving multimodal web accessibility for deaf people: sign language interpreter module. Multimed. Tools Appl. 54(1), 181–199 (2011)

    Google Scholar 

  51. Di Nubila, H.B.V., Buchalla, C.M.: O papel das Classificações da OMS—CID e CIF nas definições de deficiência e incapacidade. Rev. Bras. Epidemiol. 11(2), 324–335 (2008)

    Google Scholar 

  52. Stucki, G., Reinhardt, J.D., Grimby, G., Melvin, J.: Developing ‘human functioning and rehabilitation research’ from the comprehensive perspective. J. Rehabil. Med. 39(9), 665–671 (2007)

    Google Scholar 

  53. Schneider, R., Marcolin, D., Dalacorte, R.: Avaliação funcional de idosos. Sci. Med. (Porto. Alegre) 18(1), 4–9 (2008)

    Google Scholar 

  54. Jette, A.M.: Toward a common language for function, disability, and health. Phys. Ther. 86(5), 726–734 (2006)

    Google Scholar 

  55. Nordenfelt, L.: Action theory, disability and ICF. Disabil. Rehabil. 25(18), 1075–1079 (2003)

    Google Scholar 

  56. Federici, S., Meloni, F., LoPresti, A.: International literature review on WHODAS II. Life Span Disabil. 12(1), 83–110 (2009)

    Google Scholar 

  57. Queirós, A., Alvarelhão, J., Amaro, A., Teixeira, A., Rocha, N.: The international classification of functioning, disability of health as a conceptual framework for the design, development and evaluation of AAL services for older adults. In: Workshop AAL. Rome, Italy, pp. 46–59 (2011)

  58. Queirós, A., Silva, A., Alvarelhão, J., Rocha, N.: Um modelo conceptual para o ambient assisted living. In: Teixeira, A., Queirós, A., Rocha, N. (eds.) Laboratório Vivo de Usabilidade, 1st edn, pp. 89–99. Arc Pubishing, Aveiro (2013)

    Google Scholar 

  59. Martins, A.I., Queirós, A., Silva, A.G., Rocha, N.P.: ICF based usability scale: evaluating usability according to the evaluators’ perspective about the users’ performance. In: Proceedings of the 7th International Conference on Software Development and Technologies for Enhancing Accessibility and Fighting Info-exclusion—DSAI 2016, pp. 378–383 (2016)

  60. Martins, A.I., Rosa, A.F., Queirós, A., Silva, A., Rocha, N.P.: Definition and Validation of the ICF—usability scale. Proc. Comput. Sci. 67, 132–139 (2015)

    Google Scholar 

  61. Martins, A.I., Queirós, A., Cerqueira, M.: Metodologia Living Usability Lab. In: Teixeira, A., Queirós, A., Rocha, N.P. (eds.) Laboratório Vivo de Usabilidade, 1st edn, pp. 215–233. Arc Pubishing, Aveiro (2013)

    Google Scholar 

  62. Ribeiro, V.S., Martins, A.I., Queirós, A., Silva, A.G., Rocha, N.P.: AAL@MEO: interactive digital-TV to support home care. Stud. Health Technol. Inform. 217, 1024–1029 (2015)

    Google Scholar 

  63. World MedicalAssociation: Declaration of Helsinki. Law Med Health Care A Publ Am Soc Law Med 19(3-4), 264–265 (1991)

    Google Scholar 

  64. Martins, A.I., Rosa, A.F., Queirós, A., Silva, A., Rocha, N.P.: European Portuguese validation of the system usability scale (SUS). Proc. Comput. Sci. 67, 293–300 (2015)

    Google Scholar 

  65. Ruano, L., et al.: Development of a self-administered web-based test for longitudinal cognitive assessment. Sci. Rep. 6(1), 19114 (2016)

    Google Scholar 

  66. Nokelainen, P.: An empirical assessment of pedagogical usability criteria for digital learning material with elementary school students. Educ. Technol. Soc. 9(2), 178–197 (2006)

    Google Scholar 

Download references

Acknowledgements

This study was funded by the Project QREN No. 13852 I&D «AAL4ALL—Ambient Assisted Living for All», UI 127/94—IEETA co-funded by COMPETE and FEDER.

Author information

Authors and Affiliations

  1. Institute of Electronics and Telematics Engineering of Aveiro, Campus Universitário, 3810, Aveiro, Portugal

    Ana Isabel Martins, Alexandra Queirós & Nelson Pacheco Rocha

  2. Health Sciences School, University of Aveiro, Campus Universitário, 3810, Aveiro, Portugal

    Alexandra Queirós

  3. Department of Medical Sciences, University of Aveiro, Campus Universitário, 3810, Aveiro, Portugal

    Nelson Pacheco Rocha

Authors
  1. Ana Isabel Martins
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alexandra Queirós
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nelson Pacheco Rocha
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ana Isabel Martins.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Martins, A.I., Queirós, A. & Rocha, N.P. Validation of a usability assessment instrument according to the evaluators’ perspective about the users’ performance. Univ Access Inf Soc 19, 515–525 (2020). https://doi.org/10.1007/s10209-019-00659-w

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10209-019-00659-w

Keywords

Search

Navigation