Skip to main content

Advertisement

Springer Nature Link
Log in

Development and Validation of Digital Health Technology Literacy Assessment Questionnaire

  • Mobile & Wireless Health
  • Published:
Journal of Medical Systems Aims and scope Submit manuscript

Abstract

In clinical practice, assessing digital health literacy is important to identify patients who may encounter difficulties adapting to digital health using digital technology and service. We developed the Digital Health Technology Literacy Assessment Questionnaire (DHTL-AQ) to assess the ability to use digital health technology, services, and data. The DHTL-AQ was developed in three phases. In the first phase, the conceptual framework and domains and items were generated from a systematic literature review using relevant theory and surveys. In the second phase, a cross-sectional survey with 590 adults age ≥ 18 years was conducted at an academic hospital in Seoul, Korea in January and February 2020 to test face validity of the items. Then, psychometric validation was conducted to determine the final items and cut-off scores of the DHTL-AQ. The eHealth literacy scale, the Newest Vital Sign, and 10 mobile app task ability assessments were examined to test validity. The final DHTL-AQ includes 34 items in two domains (digital functional and digital critical literacy) and 4 categories (Information and Communications Technology terms, Information and Communications Technology icons, use of an app, evaluating reliability and relevance of health information). The DHTL-AQ had excellent internal consistency (overall Cronbach’s α = 0.95; 0.87–0.94 for subtotals) and acceptable model fit (CFI = 0.821, TLI = 0.807, SRMR = 0.065, RMSEA = 0.090). The DHTL-AQ was highly correlated with task ability assessment (r = 0.7591), and moderately correlated with the eHealth literacy scale (r = 0.5265) and the Newest Vital Sign (r = 0.5929). The DHTL-AQ is a reliable and valid instrument to measure digital health technology literacy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Availability of data and material

Data available in Samsung Medical Center.

Notes

  1. Administration, U.S.F.D. What is Digital Health? Sep.22.2020; Available from: https://www.fda.gov/medical-devices/digital-health-center-excellence/what-digital-health.

  2. National Cancer Institute, Health Information National Trends Survey (HINTS) Available from: https://hints.cancer.gov/.

  3. You Can Trust Available from: https://wisconsinliteracy.org/health-literacy/programs/health-literacy-programs/health-online-finding-information-you-can-trust.html

  4. ICT, Korean Digital Literacy in 2018 Available from: https://www.nia.or.kr/site/nia_kor/ex/bbs/List.do?cbIdx=81623

  5. Kyriazos, T.A., Applied Psychometrics: Sample Size and Sample Power Considerations in Factor Analysis (EFA, CFA) and SEM in General. Psychology, 2018. 9(8). Available from: https://www.scirp.org/journal/paperinformation.aspx?paperid=86856

  6. BRAVO App 2019; Available from: http://bravoproject.or.kr/.

  7. Korean version of NVS: Kim, J. Measuring the Level of Health Literacy and Influence Factors: Targeting the Visitors of a University Hospital s Outpatient Clinic. 2011; Available from: https://www.koreascience.or.kr/article/JAKO201108161395188.page

  8. Korean Adaptation of the General Self-Efficacy Scale. 1994; Available from: http://userpage.fu-berlin.de/~health/korean.htm.

References

  1. Lee, J.-A., et al., Effective behavioral intervention strategies using mobile health applications for chronic disease management: a systematic review. BMC medical informatics and decision making, 2018. 18(1): p. 12-12. https://doi.org/10.1186/s12911-018-0591-0.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Mahmood, A., et al., Use of mobile health applications for health-promoting behavior among individuals with chronic medical conditions. Digital health, 2019. 5: p. 2055207619882181-2055207619882181. https://doi.org/10.1177/2055207619882181.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Agarwal, P., et al., Mobile App for Improved Self-Management of Type 2 Diabetes: Multicenter Pragmatic Randomized Controlled Trial. JMIR Mhealth Uhealth, 2019. 7(1): p. e10321. https://doi.org/10.2196/10321.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Dunn, P. and S. Conard, Digital health literacy in cardiovascular research. Int J Cardiol, 2018. 269: p. 274-275. https://doi.org/10.1016/j.ijcard.2018.07.011.

    Article  PubMed  Google Scholar 

  5. Price-Haywood, E.G., et al., eHealth Literacy: Patient Engagement in Identifying Strategies to Encourage Use of Patient Portals Among Older Adults. Popul Health Manag, 2017. 20(6): p. 486-494. https://doi.org/10.1089/pop.2016.0164.

    Article  PubMed  Google Scholar 

  6. Yan, K., C. Balijepalli, and E. Druyts, The Impact of Digital Therapeutics on Current Health Technology Assessment Frameworks. Front Digit Health, 2021. 3: p. 667016. https://doi.org/10.3389/fdgth.2021.667016.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Robbins, D. and P. Dunn, Digital health literacy in a person-centric world. Int J Cardiol, 2019. 290: p. 154-155. https://doi.org/10.1016/j.ijcard.2019.05.033.

    Article  PubMed  Google Scholar 

  8. Harris, K., G. Jacobs, and J. Reeder, Health Systems and Adult Basic Education: A Critical Partnership in Supporting Digital Health Literacy. Health Lit Res Pract, 2019. 3(3 Suppl): p. S33-s36. https://doi.org/10.3928/24748307-20190325-02.

    Article  PubMed  PubMed Central  Google Scholar 

  9. van der Vaart, R. and C. Drossaert, Development of the Digital Health Literacy Instrument: Measuring a Broad Spectrum of Health 1.0 and Health 2.0 Skills. J Med Internet Res, 2017. 19(1): p. e27. https://doi.org/10.2196/jmir.6709.

  10. Kayser, L., et al., A Multidimensional Tool Based on the eHealth Literacy Framework: Development and Initial Validity Testing of the eHealth Literacy Questionnaire (eHLQ). J Med Internet Res, 2018. 20(2): p. e36. https://doi.org/10.2196/jmir.8371.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Griebel, L., et al., eHealth literacy research-Quo vadis? Inform Health Soc Care, 2018. 43(4): p. 427-442. https://doi.org/10.1080/17538157.2017.1364247.

    Article  PubMed  Google Scholar 

  12. Holt, K.A., et al., Differences in the Level of Electronic Health Literacy Between Users and Nonusers of Digital Health Services: An Exploratory Survey of a Group of Medical Outpatients. Interact J Med Res, 2019. 8(2): p. e8423. https://doi.org/10.2196/ijmr.8423.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Paige, S.R., et al., Proposing a Transactional Model of eHealth Literacy: Concept Analysis. J Med Internet Res, 2018. 20(10): p. e10175. https://doi.org/10.2196/10175.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Chang, A. and P.J. Schulz, The Measurements and an Elaborated Understanding of Chinese eHealth Literacy (C-eHEALS) in Chronic Patients in China. Int J Environ Res Public Health, 2018. 15(7). https://doi.org/10.3390/ijerph15071553.

  15. Liberati, A., et al., The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. Bmj, 2009. 339: p. b2700. https://doi.org/10.1136/bmj.b2700.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Naslund, J.A., et al., Health behavior models for informing digital technology interventions for individuals with mental illness. Psychiatr Rehabil J, 2017. 40(3): p. 325-335. https://doi.org/10.1037/prj0000246.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Venkatesh, V., J.Y.L. Thong, and X. Xu, Unified Theory of Acceptance and Use of Technology: A Synthesis and the Road Ahead. J. Assoc. Inf. Syst., 2016. 17: p. 1. https://doi.org/10.17705/1jais.00428.

    Article  Google Scholar 

  18. Chung, S., B.K. Park, and E.S. Nahm, The Korean eHealth Literacy Scale (K-eHEALS): Reliability and Validity Testing in Younger Adults Recruited Online. J Med Internet Res, 2018. 20(4): p. e138. https://doi.org/10.2196/jmir.8759.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Norman, C.D. and H.A. Skinner, eHEALS: The eHealth Literacy Scale. J Med Internet Res, 2006. 8(4): p. e27. https://doi.org/10.2196/jmir.8.4.e27.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Karnoe, A., et al., Assessing Competencies Needed to Engage With Digital Health Services: Development of the eHealth Literacy Assessment Toolkit. J Med Internet Res, 2018. 20(5): p. e178. https://doi.org/10.2196/jmir.8347.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Nutbeam, D., Health literacy as a public health goal: a challenge for contemporary health education and communication strategies into the 21st century. Health Promotion International, 2000. 15(3): p. 259-267. https://doi.org/10.1093/heapro/15.3.259.

    Article  Google Scholar 

  22. Cattell, R. B. (1978). The Scientific Use of Factor Analysis in Behavioral and Life Sciences. New York: Plenum.https://doi.org/10.1007/978-1-4684-2262-7

  23. Gorsuch, R. (1983). Factor Analysis. Hillsdale, NJ: L. Erlbaum Associates.

  24. Tabachnick, B., & Fidell, L. (2013). Using Multivariate Statistics. Boston, MA: Pearson Education Inc.

  25. Comrey, A. L., & Lee, H. B. (1992). A First Course in Factor Analysis. Hillsdale, NJ: Lawrence Eribaum Associates

  26. Johanson, G.A. and G.P. Brooks, Initial Scale Development: Sample Size for Pilot Studies. Educational and Psychological Measurement, 2009. 70(3): p. 394-400. https://doi.org/10.1177/0013164409355692.

    Article  Google Scholar 

  27. Weiss, B.D., The Newest Vital Sign: Frequently Asked Questions. Health literacy research and practice, 2018. 2(3): p. e125-e127. https://doi.org/10.3928/24748307-20180530-02.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Park, Y. and H.-J. Yoon, Understanding Personal Health Record and Facilitating its Market. Healthc Inform Res, 2020. 26(3): p. 248-250. https://doi.org/10.4258/hir.2020.26.3.248.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Lee, S.M. and D. Lee, Healthcare wearable devices: an analysis of key factors for continuous use intention. Service Business, 2020. 14(4): p. 503-531. https://doi.org/10.1007/s11628-020-00428-3.

    Article  Google Scholar 

  30. Park, J., and Kyeong-Na Kim, Structural Influence of SNS Social Capital on SNS Health Information Utilization Level. The Korean Journal of Health Service Management, June 2020. 14(2): p. 1–14. https://doi.org/10.12811/kshsm.2020.14.2.001.

  31. Rasche, P., et al., Prevalence of Health App Use Among Older Adults in Germany: National Survey. JMIR mHealth and uHealth, 2018. 6(1): p. e26-e26. https://doi.org/10.2196/mhealth.8619.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Funding

This research was supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (NRF-2018R1D1A1B07047833).

Author information

Authors and Affiliations

  1. Department of Clinical Research Design and Evaluation, SAIHST, Sungkyunkwan University, Seoul, Korea

    Junghee Yoon & Juhee Cho

  2. Patient-Centered Outcomes Research Institute, Samsung Medical Center, Seoul, Korea

    Junghee Yoon

  3. Department of Digital Health, SAIHST, Sungkyunkwan University, Seoul, Korea

    Mangyeong Lee, Soo-Yong Shin & Juhee Cho

  4. Center for Clinical Epidemiology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Korea

    Mangyeong Lee & Juhee Cho

  5. Division of Hematology‐Oncology, Department of Medicine, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Korea

    Jin Seok Ahn

  6. Department of Radiation Oncology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Korea

    Dongryul Oh

  7. Center for Research Resource Standardization, Samsung Medical Center, Seoul, Korea

    Soo-Yong Shin

  8. Department of Cancer Control and Population Health, National Cancer Center Graduate School of Cancer Science and Policy, Goyang, Korea

    Yoon Jung Chang

  9. Hospice & Palliative Care Branch, National Cancer Center, Goyang, Korea

    Yoon Jung Chang

  10. Cancer Education Center, Samsung Medical Center, Seoul, Korea

    Juhee Cho

Authors
  1. Junghee Yoon
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Mangyeong Lee
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Jin Seok Ahn
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Dongryul Oh
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Soo-Yong Shin
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Yoon Jung Chang
    View author publications

    You can also search for this author inPubMed Google Scholar

  7. Juhee Cho
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

Study conception and design: All authors; data acquisition: J.H.Y and M.G.L; analysis and interpretation of results: JH.Y, MG.L, and JH.C; drafting of manuscript: JH.Y and JH.C; critical revision of manuscript: JH.Y, MG.L, and JH.C. All authors reviewed the results and approved the final version of the manuscript.

Corresponding author

Correspondence to Juhee Cho.

Ethics declarations

Ethics approval

The study was reviewed and approved by the Samsung Medical Center Institutional Review Board (2019–07-045–006).

Consent to participate

All participants provided written informed consent at the onset for this study.

Consent for publication

Not applicable.

Conflicts of interest

None declared.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article is part of the Topical Collection on Mobile & Wireless Health

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (PDF 444 KB)

Supplementary file2 (PDF 485 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yoon, J., Lee, M., Ahn, J.S. et al. Development and Validation of Digital Health Technology Literacy Assessment Questionnaire. J Med Syst 46, 13 (2022). https://doi.org/10.1007/s10916-022-01800-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10916-022-01800-8

Keywords