Skip to main content
SpringerLink
Log in

Using the TURF Framework to Design an Enhanced Dosimetry Quality Assurance Checklist in an Academic Medical Center

  • Conference paper
  • First Online:
Design, User Experience, and Usability (HCII 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14034))

Included in the following conference series:

  • 638 Accesses

Abstract

Academic radiation oncology centers have a long history of developing in-house quality assurance (QA) checklists to promote patient safety. These checklists are designed without utilizing formal human-computer interaction methods and are deployed without robust usability evaluation. We applied the Task, User, Representation, and Function (TURF) framework to identify design changes to a dosimetry QA checklist currently deployed in our institution. We found that the TURF framework provided great insights to improve the usability of QA checklists.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Marks, L.B., Jackson, M., Xie, L., et al.: The challenge of maximizing safety in radiation oncology. Pract Radiat Oncol. 1(1), 2–14 (2011). https://doi.org/10.1016/j.prro.2010.10.001

    Article  Google Scholar 

  2. Adapa, K.: Unifying evidence-based frameworks to design, develop, implement, and evaluate health information technology tools in radiation oncology (Order No. 29393143). Available from Dissertations & Theses @ University of North Carolina at Chapel Hill; ProQuest Dissertations & Theses Global (2022). (2760194888).  http://libproxy.lib.unc.edu/login?url=https://www-proquest-com.libproxy.lib.unc.edu/dissertations-theses/unifying-evidence-based-frameworks-design-develop/docview/2760194888/se-2

  3. Ford, E.C., Terezakis, S., Souranis, A., Harris, K., Gay, H., Mutic, S.: Quality control quantification (QCQ): a tool to measure the value of quality control checks in radiation oncology. Int. J, Radiat. Oncol. Biol. Phys. 84(3), e263–e269 (2012). https://doi.org/10.1016/j.ijrobp.2012.04.036

  4. Adapa, K., Mosaly, P., Yu, F., Moore, C., Das, S., Mazur, L.: Exploring association between perceived usability of dosimetry quality assurance checklist and perceived cognitive workload of dosimetrists in clinical settings. Proc. Hum. Factors Ergon. Soc. Annu. Meet. 65(1), 771–775 (2021). https://doi-org.libproxy.lib.unc.edu/10.1177/1071181321651285

  5. Tracton, G.S., Mazur, L.M., Mosaly, P., Marks, L.B., Das, S.: Developing and assessing electronic checklists for safety mindfulness, workload, and performance. Pract. Radiat. Oncol. 8(6), 458–467 (2018). https://doi.org/10.1016/j.prro.2018.05.001

  6. McGurk, R., et al.: Multi-institutional stereotactic body radiation therapy incident learning: evaluation of safety barriers using a human factors analysis and classification system. J. Patient Saf. 19(1), e18–e24 (2023). https://doi.org/10.1097/PTS.0000000000001071

  7. Adapa, K., et al.: Human-centered participatory co-design of a dosimetry-quality assurance checklist in an academic cancer center. In: Duffy, V.G. (ed.) Digital Human Modeling and Applications in Health, Safety, Ergonomics and Risk Management. Health, Operations Management, and Design. HCII 2022. LNCS, vol. 13320, pp. 3–20. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-06018-2_1

    Chapter  Google Scholar 

  8. Adapa, K..: Hhuman-, centered, participatory co-design of a dosimetry-quality assurance checklist in an academic cancer, center, ., indigital, human, modeling, and applications in health, safety, ergonomics, and risk management. health, operations management, and design. In: 13th international conference, DHM: Held as Part of the 24th HCI International Conference, HCII 2022, Virtual Event, June 26–July 1, 2022, Proceedings, Part II 2022 Jun 16, pp. 3–20. Springer International Publishing, Cham (2022). https://doi.org/10.1007/978-3-031-06018-2_1

  9. Zhang, J., Walji, M.F.: TURF: toward a unified framework of EHR usability. J. Biomed. Inform. 44(6), 1056–1067 (2011). https://doi.org/10.1016/j.jbi.2011.08.005

  10. Zhang, J., Butler. K.: UFuRT : A work-centered framework and process for design and evaluation of information systems. In: Proceedings of HCI International 2007 (2007)

    Google Scholar 

  11. Zhang. Z., Walji, M.F., Patel, V.L., Gimbel, R.W., Zhang, J.: Functional analysis of interfaces in U.S. military electronic health record system using UFuRT framework. In: AMIA Annual Symposium Proceedings 2009, pp. 730–734 (2009)

    Google Scholar 

  12. Moreira, M., Peixoto, C.: Qualitative task analysis to enhance sports characterization: a surfing case study. J. Hum. Kinet. 42, 245–257 (2014). https://doi.org/10.2478/hukin-2014-0078

  13. Knisely, B.M., Joyner, J.S., Vaughn-Cooke, M.: Cognitive task analysis and workload classification. MethodsX. 8, 101235 (2021). https://doi.org/10.1016/j.mex.2021.101235

  14. Bloom, B.S.: Taxonomy of educational objectives : The classification of educational goals. Published online, Cognitive domain (1956)

    Google Scholar 

  15. Harrow, A.J.: A Taxonomy of the Psychomotor Domain: A Guide for Developing Behavorial Objectives

    Google Scholar 

  16. Anwar, F., Sulaiman, S.: P.D.D.Dominic. Cognitive Task Analysis: A Contextual Inquiry Study on Basic Computer and Information Literacy Skills among Physicians. ISICO 2013. Published online (2013)

    Google Scholar 

  17. Chan, C.V., Kaufman, D.R.: A framework for characterizing eHealth literacy demands and barriers. J. Med. Internet Res. 13(4), e94 (2011). https://doi.org/10.2196/jmir.1750

  18. Nielsen J. Usability inspection methods. In: Plaisant, C., (ed.) Conference Companion on Human Factors in Computing Systems - CHI  1994, pp. 413–414 ACM Press (1994).:https://doi.org/10.1145/259963.260531

  19. Usability Engineering - Jakob Nielsen - Google Books

    Google Scholar 

  20. Shneiderman, B.: Designing the user interface strategies for effective human-computer interaction. SIGBIO Newsl. 9(1), 6 (1987). https://doi.org/10.1145/25065.950626

  21. Zhang, J., Johnson, T.R., Patel, V.L., Paige, D.L., Kubose, T.: Using usability heuristics to evaluate patient safety of medical devices. J. Biomed. Inform. 36(1–2), 23–30 (2003). https://doi.org/10.1016/S1532-0464(03)00060-1

  22. Parasuraman, A., Colby, C.L.: An updated and streamlined technology readiness index. J. Serv. Res. 18(1), 59–74 (2015). https://doi.org/10.1177/1094670514539730

  23. Dowding, D., Merrill, J.A.: The development of heuristics for evaluation of dashboard visualizations. Appl. Clin. Inform. 9(3), 511–518 (2018). https://doi.org/10.1055/s-0038-1666842

Download references

Acknowledgments

We thank the dosimetrists, physicists, and clinic leadership of our department for their support of our research efforts.

Author information

Authors and Affiliations

  1. Carolina Health Informatics Program, University of North Carolina, Chapel Hill, NC, USA

    Karthik Adapa & Lukasz Mazur

  2. Department of Radiation Oncology, University of North Carolina, Chapel Hill, NC, USA

    Karthik Adapa, Gregg Tracton, Ross McGurk, John Dooley, Shiva Das & Lukasz Mazur

  3. Ben Allegretti Consulting, Inc., Stafford, VA, USA

    Prithima Mosaly & Lukasz Mazur

  4. School of Information and Library Science, University of North Carolina, Chapel Hill, NC, USA

    Fei Yu

  5. Department of Medicine, School of Medicine, University of North Carolina, Chapel Hill, NC, USA

    Carlton Moore

Authors
  1. Karthik Adapa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gregg Tracton
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Prithima Mosaly
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Fei Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ross McGurk
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Carlton Moore
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. John Dooley
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Shiva Das
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Lukasz Mazur
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Karthik Adapa .

Editor information

Editors and Affiliations

  1. Aaron Marcus and Associates, Berkeley, CA, USA

    Aaron Marcus

  2. World Usability Day and Bubble Mountain Consulting, Newton Center, MA, USA

    Elizabeth Rosenzweig

  3. Southern University of Science and Technology – SUSTech, Shenzhen, China

    Marcelo M. Soares

Appendix I

Appendix I

Procedural QA task

MOSAIQ (MQ)

Taxonomy Clf. **

RayStation (RS)

Taxonomy Clf. **

QA checklist

Taxonomy Clf. **

CT level 1

CT level 2

CT level 1

CT level 2

CT level 3

CT level 1

CT level 2

DOB and Age: Check if DOB of the patient is same in both Mosaiq and RS

1.Select patient

1.1 Click on Facesheet

Perceptual ability

1. Select patient data management

1.1 Click patient information

 

Perceptual ability

1.Compare DOB information from MQ and RS

1.1 Select appropriate option

Analysis

 

1.2 Retain the patient’s DOB in working memory

Knowledge

 

1.2 Retain the patient's DOB in working memory

 

Knowledge

   

Pacemaker: Check if pacemaker is assessed as present? If pacemaker present is ROI contoured?

1. Select assessments

1.1 View simulation directive

Perceptual ability

1. Select ROI tab

1.1 Skip targets

 

Perceptual ability

1.Compare most recent pacemaker assessment in MQ with presence of ROI and contours for pacemaker in RS

1.1 Select appropriate option

Evaluation

 

1.2 Check if pacemaker has been assessed

Comprehension

 

1.2 Look for pacemaker in OARs

1.2.1 If pacemaker ROI exists, check for contours

Perceptual ability

   
 

1.3 Retain the date of assessment

Knowledge

 

1.3 Look for pacemaker in unknowns

1.3.1 If pacemaker ROI exists, check for contours

Perceptual ability

   
 

1.4 Only the most recent date of assessment is important

Evaluation

 

1.4 Retain informatiion about ROI and contours

 

Knowledge

   

Rx vs Planning note: Check if prescription is MQ is identical to plan information on RS

1. Select navigator

1.1 Click Tx plan

Perceptual ability

1. Select plan design

1.1 Select the beamset

 

Perceptual ability

1.Compare the Rx in MQ with dose, fractions, energy and modality in RS and see if dose constraints has been met

1.1 Select appropriate option

Evaluation

 

1.2 Retain the content of the note in working memory (dose, fraction, primary site, secondary site etc.)

Knowledge

 

1.2 Look for dose, fraction, energy and modality (all located in different tables) Retain the content about fraction,

 

Perceptual ability

   
    

1.3 Retain the information about fraction, dose, fraction, energy and modality (all located in different tables) in working memory

 

Knowledge

   
   

2. Select plan evaluation

2.1 Look for PTV and see if dose constraints were met

 

Perceptual ability

   

QCL Finance: IMRT/SBRT pre-auth

1. Select QCL or navigator

1.1 Add filter for pre-auth

Perceptual ability

    

1.Ensure QCL is complete, assessment is recent and completed by authorized user

1.1 Select appropriate option

Analysis

 

1.2 Double click or scan the entire breadth of list of columns

Perceptual ability

       
 

1.3 Confirm that information is complete, and date is fresh for this QCL

Analysis and Knowledge

       
  1. *CT – Cognitive Task.
  2. * * Taxonomy cf. – Taxonomy Classification.

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Adapa, K. et al. (2023). Using the TURF Framework to Design an Enhanced Dosimetry Quality Assurance Checklist in an Academic Medical Center. In: Marcus, A., Rosenzweig, E., Soares, M.M. (eds) Design, User Experience, and Usability. HCII 2023. Lecture Notes in Computer Science, vol 14034. Springer, Cham. https://doi.org/10.1007/978-3-031-35705-3_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-35705-3_18

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-35704-6

  • Online ISBN: 978-3-031-35705-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation