It's more than just use: An exploration of telemedicine use quality
Introduction
“Quality is never an accident; it is always the result of intelligent effort.”—John Ruskin
In medical informatics, the manner in which a technological intervention is used can have a significant impact on the health and well being of patients who depend on it. If we are to effectively manage technology-based medical information systems, we must understand and manage their use, especially as it applies to encounters between health care providers and patients.3 A critical and increasingly important application of medical informatics is the use of video conferencing for patient exams (see Fig. 1). In this context, video conferencing is frequently used to support a knowledge discovery process (concerning the medical condition of the patient), as well as decision-making in the form of diagnosis and recommended protocols. There is widespread interest in utilizing medical video conferencing technology as an economical method to provide expert medical service to patients in remote and awkward locations, and to address misdistributions of health care resources (i.e., facilities and medical expertise) outside major urban centers [14], [35]. There is also a growing recognition that telemedicine can facilitate the timeliness of medical care by providing access to a wider range of appropriate medical providers during the ‘golden window’ of treatment opportunity [13].
Advocates of telemedicine believe that telemedicine encounters (e.g., medical video conferencing) should be recognized as a “timely technology to facilitate health decision-making and clinical service support.” These encounters address patients' needs for communication and caring, as well as physicians' concerns for high quality clinical care, while having a positive financial impact on containing medical costs (especially after policy and reimbursement constraints are addressed by governing bodies) [27]. In broad terms, the quality of this kind of technology-based service encounter may be described as the expected level of performance and information provided by the organization, technology, employee and, to some degree, customer (as indirect user of the technology) to support the interaction and transaction success [31]. To pronounce a medical video conferencing encounter a success, decisions regarding suitable care and patient satisfaction must be supported by the effective use of the socio-technical system.
Since this form of telemedicine promises to bridge geographic distances in the provision of medical expertise, it is of great concern that mixed results have been reported in terms of utilization rates, even when external issues such as reimbursement issues and policy constraints are not major impediments (e.g., [31], [50]). These mixed results suggest that telemedicine-related research should not merely recognize constraints imposed by organizational and legislative policy, but should also look deeper into the telemedicine system as an integrated socio-technical process and product in order to assure its successful utilization. As stated by Jennett et al., “Telehealth systems can have impact at three levels: the health system level, the program level, and the patient encounter level. Each level requires different types of evaluation models” [26] (p. 364). Telemedicine researchers recognize that there is a paucity of explanatory research that predicts and facilitates the success of telemedicine encounters [45].
In our study of telemedicine encounters, we attempt to address this research gap by introducing the term use quality to address the effectiveness of the actual encounter usage. In the case of medical video conferencing, use quality spotlights the attributes of the socio-technical decision-making process of utilizing telemedicine for patient diagnosis and assessment.
The exploration of use quality focuses on work practices and methods of organizing work. Though such studies are needed as a basis for the formation of “post-bureaucratic” organizational theory, few such studies exist in the modern organizational literature. Barley and Kunda [6] call for the study of new forms of work in emerging, situated contexts to facilitate understanding of changing work patterns. Research by Jennett et al. provides recognition that telemedicine may alter the nature of work in health care [27]. However, the information systems (IS) literature lacks strong models of use quality to serve as guides for providers and researchers. The literature has not qualified use in most studies, though the IS community has known for a long time that unused systems are unsuccessful systems [34]. However, the corollary that system use produces success is not necessarily true, though many IS models and studies portray use as a proxy for, or an implied indicator of, system success [55]. The telemedicine literature has also adopted use as a proxy for success (e.g., [38], [56]). Though some researchers assert that continued use is a better indicator of success than simply use, they do not elaborate on the attributes of quality that recur in specific episodes of use that promote successful patterns and continuance of use [7].
Even the industrial and data management literatures, in which strong research efforts exist in the realm of total quality management, lack focused studies about quality in the usage stage, though considerable research exists concerning quality in the design and production stages. “The literature on improving the quality of use is vague, and specific procedures and guidelines for improving the quality of use are non-existent” [3] (p. 8).
In the telemedicine domain, use quality encompasses technology, medical procedures, decision-making and human interactions in a holistic, integrated view of the system. Medical video conferencing provides an intriguing context for exploring use quality as perceived by people in different roles in the telemedicine encounter (direct/indirect users and health care providers/consumers) via different aspects of system use (technology/human interactions). Of all the uses of medical video conferencing (e.g., education, peer consultation, patient exams), video conferencing for direct medical care, given its immediate impact on patient care, requires the highest use quality standard.
There exist no definitive references to use quality attributes based on a thorough search of IS and software engineering literatures. The telemedicine literature does address some instances of use quality-related concepts. For example, the need for training and integration into other modes of care is recognized from an organizational level, though not specified and directly addressed through the encounter experience [26], [27]. Other examples reference use in telemedicine adoption and diffusion without qualification regarding the process or standard of use (e.g., [24], [57], [60]). An interesting project in Canada by the National Telehealth Outcome Indicator Project (NTOIP) [47] has identified four indicators that are related to use quality: (1) quality of the telehealth encounter (ease of use and communication of critical health care information), (2) integration of telehealth with traditional health care, (3) the quality of the technology used and (4) user satisfaction. New to the literature is the NTOIP's recognition that the actual telemedicine user may participate in different roles (e.g., patient, provider) and have different understandings of the use process [32]. Overall, however, there is currently no high level guidance provided by medical standards, such as JACHO guidelines, that would lead to a clear, generalized understanding of the attributes of use quality in the telemedicine encounter context. Telemedicine researchers indicate that guidelines and standards are needed at a number of levels (e.g., technological, procedural, service) for telemedicine consultation for the health sector to fully embrace and diffuse telemedicine care options [26].
By citing literature references and comments made by participants in our study, we aim to provide insights into quality specifications, the effects of attribute deficiencies, and the means of addressing quality issues. In essence, we attempt to define the factors of intelligent effort (i.e., use quality) as users and technologies interact within the telemedicine system. We take a bottom-up approach from which theories within various domains may be extended or synthesized as insight deepens. We contribute to this synthesis by proposing generalized categories for specific attributes that may extend across research domains, a general assessment of constructs, and a comparative overview of patients and provider perspectives. We keep our research at the level of patient encounters and do not directly expose policy, legislative or social influences that act at the organizational level.
Section snippets
Use quality in telemedicine
The conceptualization of use quality and its role in the success of telemedicine encounters draws on the DeLone and McLean IS success model [15] (see Fig. 2). There is some concern that research efforts that reference this model are not always consistent with the spirit of the model's intent. In keeping with the spirit of the model, we heed two directives: context specification and deep probing into system use.
The discipline of quality has gone through the phases of caveat emptor, quality
Telemedicine encounter perspectives
Because of the various roles involved in, and aspects of, social and technical interactions during the encounter process, successful telemedicine system encounters require views from multiple perspectives if they are to be understood properly. Patients and providers may have different perceptions of the system and of the quality of its use. Each group may have insights into use quality attributes that may not be readily apparent to the other group. Only a small amount of research into service
Methodology
We use analytic inductive means to determine quality attributes drawn directly from respondents. Thus, we attempt to reveal implicit insights regarding issues and prescriptions for quality attributes [54]. We employ several empirical methods in this field study. Qualitative methods (e.g., focus groups and interviews) are used to elicit, code and analyze data from respondents. Quantitative methods (e.g., close-ended survey questions) are used along with open-ended survey questions to validate
Results and discussion: use quality framework
To address the first research question, we specify a framework of use quality attributes based on the union of use characteristics identified by providers and patients as critical to the success of telemedicine encounters. Use quality characteristics are categorized into three critical factors for success in telemedicine encounters: technological aptitude and ability, communication skills and orchestration (see Fig. 3). Appendix A provides a tabular description of the use quality attributes
Results and discussion: comparing patient and provider views
A summary of our findings is provided in Table 1.
To address the second research question, we compared the use quality attributes specified by providers and patients. It is interesting to note that, as the table in Appendix A demonstrates, patients identified eight of the nine use quality attributes (one attribute, adaptability, was identified only by providers). Five of the eight attributes were identified solely by patients (not by providers). Thus, we propose that use quality constructs in
Conclusion: the socio-technical nature of telemedicine encounters
We stress the need to re-characterize the construct of use to use quality when exploring systems success and when assessing use quality from the perspectives of several types of users (see Fig. 3). Our experience with both patients and providers confirms the viability of the use quality construct.
Parasuraman and Colby [43] have shown that the effectiveness of a technology service encounter depends on the use of state-of-the-art technology and the quality of technology-based interactions. Recent
Limitations and future research
As discussed earlier in the paper, the qualitative, inductive approach taken in this study may create bias in the study. The multiple methods and validation processes employed in the study seek to improve the rigor of the research effort, but cannot remove the subjectivity in interpretation inherent in the research approach. Moreover, the study subjects were drawn from one large health care provider with multiple hospitals and clinics in order to limit differences due to variations in
Acknowledgements
This research was conducted in cooperation with the Veterans Integrated Service Network 8 (VISN 8) of the Department of Veterans Affairs health care system and the INTEGRIS Telehealth Network and Rural Telemedicine Project.
Cynthia LeRouge is an Assistant Professor at St. Louis University. Her current research interests relate to health care information systems, project management and technology-mediated learning. Dr. LeRouge has held various management roles in practice in the software, healthcare, public accounting and petrochemical industries. She has published multiple research articles in journals including the International Journal of Healthcare Technology and Management, Journal, International Journal of
References (61)
“Discovering” chronic illness: using grounded theory
Social Science & Medicine
(1990)- et al.
Communication models of traditional and video-mediated health care delivery
International Journal of Medical Informatics
(2005) Measuring information output: a communication system approach
Information & Management
(1978)Impact of information quality and decision-maker quality on decision quality: a theoretical model and simulation analysis
Decision Support Systems
(1999)- et al.
An examination of end-user types
Information & Management
(1992) Patient power
Lancet
(1977)Determining information system usage: some issues and examples
Information & Management
(1993)- et al.
A theory of quality management underlying the Deming management method
Academy of Management Review
(1994) Outcomes and methods in telemedicine evaluation
Telemedicine Journal and E-Health
(2003)- et al.
Total quality control: the repair facility
Industrial Management and Data Systems
(1992)
Rating the appropriateness of coronary angiography—do practicing physicians agree with an expert panel and with each other?
New England Journal of Medicine
Approaching quality in an ever-changing world
Total Quality Management
Bringing work back in
Organizational Science
Understanding information systems continuance: an expectation–confirmation model
MIS Quarterly
From Novice to Expert: Excellence and Power in Clinical Nursing Practice
Critical service encounters—the employees viewpoint
Journal of Marketing
Videoconferencing in the provision of psychological services at a distance
Journal of Telemedicine and Telecare
User cube: a taxonomy of end users
Communications of the ACM
Mobile telemedicine system
Telemedicine and Telehealth
The DeLone and McLean model of information system success: a ten-year update
Journal of Management Information Systems
Communication patterns and technical quality of virtual visits in home care
Journal of Telemedicine and Telecare
Comments on: “what theory is not”
Administrative Science Quarterly
The paradigm of quality: evolution and revolution in the history of the discipline
A comparative study of resident, family, and administrator expectations for service quality in nursing homes
Health Care Management Review
The Discovery of Grounded Theory: Strategies for Qualitative Research
End users: who are they?
Communications of the ACM
Examining the technology acceptance model using physician acceptance of telemedicine technology
Journal of Management Information Systems
Investigations of factors affecting healthcare organization's adoption of telemedicine technology
Presence: where are we?
Cyberpsychology & Behavior
Cited by (47)
Customer relationship management systems (CRMS) in the healthcare environment: A systematic literature review
2020, Computer Standards and InterfacesCitation Excerpt :Issues such as the growing number of chronic illnesses and the ageing population; higher patient demand and expectations; and the lack of qualified medical professionals, have complicated healthcare organisations’ ability to fulfil their missions [3,4]. Health information technology (HIT, also known as e-health or medical informatics [5]), is viewed as a significant tool to achieve cost savings, efficiency, quality, and safety [6–8]. The benefits of HIT include: improved medical services and workflows, providing decision-making support and clinical information for medical professionals, expanding the quality, safety, and effectiveness of patient care, preventing medical errors; and reducing expenses, admissions, and paperwork [9–16].
Factors influencing effective use of big data: A research framework
2020, Information and ManagementCitation Excerpt :Effective use is defined in Information Systems (IS) research as the use of a system in a way that increases achievement of goals [1]. A substantial amount of research has investigated this concept [2–5]. For example, Burton-Jones and Grange [1] proposed a theory of the effective use of IS.
Technology alignment in the presence of regulatory changes: The case of meaningful use of information technology in healthcare
2018, International Journal of Medical InformaticsCitation Excerpt :Based on a review of literature focusing on effective use of IT, Burton-Jones and Grange [76] argue that the literature is in a nascent stage of development. Few studies that have explicitly studied effective use have considered social norms, experience with the technology, and comprehension [77], positive technological frames about using information system [78], awareness of functionalities available and necessary for work, skills on how to use them, better content, and effort by users [79], competence at leveraging IT [80], users’ interactions with the system [81], quality of information, system and service [82], perceived usefulness, ease of use, customization, habit, intra-group trust, and environmental uncertainty [83]. These drivers of effective use of IT are based on different definitions and are an outcome of investigations of different systems at different levels of analysis.
Anticipating mismatches of HIT investments: Developing a viability-fit model for e-health services
2015, International Journal of Medical InformaticsCitation Excerpt :Demographic, economic and societal side-effects, such as the aging of the population, the lack of qualified health workers, and higher expectations and demands of patients, local administrators or health insurers, additionally complicate the fulfillment of this mission [1]. Expectations are high that a broader adoption of health information technology (HIT) will contribute to cost reduction and sustainable or improved quality as well as generally improve social welfare [e.g.,2–4]. Electronic health (abbr.
Challenges for inter-departmental cooperation in hospitals: Results from cross-case analysis
2015, Health Policy and TechnologyCitation Excerpt :Therefore, profound knowledge about potential barriers and critical process interfaces for flawless cooperation is important to avoid redundancies and to design and guide efficient care processes. Furthermore, it is expected that a broader adoption of information technology (IT) can improve interoperability, reduce the risk of redundancies at departmental interfaces and facilitate cooperation between different stakeholders participating in the care process [13–16]. This applies to the cooperation between various medical departments as well as between different professional groups, such as physicians, nurses and hospital managers [17].
Teleconsultation in geriatrics: Impact on professional practice
2013, International Journal of Medical InformaticsCitation Excerpt :There are many reasons for these failures. The various techniques used (phone call, videophone, video conference, etc.) may constitute important limiting factors [10,11] and in this regard, video conferencing is increasingly used since it allows good-quality transmissions at an affordable cost [11–14]. However, along with technical difficulties, the impact on the organisation of care and professional practice is frequently mentioned as a possible factor jeopardising the establishment of telemedicine systems [10,15,16].
Cynthia LeRouge is an Assistant Professor at St. Louis University. Her current research interests relate to health care information systems, project management and technology-mediated learning. Dr. LeRouge has held various management roles in practice in the software, healthcare, public accounting and petrochemical industries. She has published multiple research articles in journals including the International Journal of Healthcare Technology and Management, Journal, International Journal of Human Computer Studies, Journal of Computer Information Systems, Journal of Information Technology Education, Journal of Information Systems Education and Communications of the AIS. Dr. LeRouge has a PhD in Information Systems from the University of South Florida.
Alan R. Hevner is an Eminent Scholar and Professor in the Information Systems and Decision Sciences Department in the College of Business Administration at the University of South Florida. He holds the Citigroup/Hidden River Chair of Distributed Technology. Dr. Hevner's areas of research interest include information systems development, software engineering, distributed database systems, healthcare information systems and telemedicine. He has published over 120 research papers on these topics and has consulted for a number of Fortune 500 companies. Dr. Hevner received a PhD in Computer Science from Purdue University. He has held faculty positions at the University of Maryland and the University of Minnesota. Dr. Hevner is a member of ACM, IEEE, AIS and INFORMS.
Rosann Webb Collins is Associate Professor of Information Systems and Decision Sciences at the University of South Florida. Her current research focuses on global information systems and cognitive issues in systems development. Her publications include a book, Crossing Boundaries: The Deployment of Global IT Solutions, and research articles in MIS Quarterly, Information Systems Research, IEEE Transactions on Software Engineering, The Information Society, the Journal of the American Society for Information Science, and other MIS and information science journals. Dr. Collins has consulted with numerous businesses, community organizations, libraries and educational organizations on information technology use and issues. She received a PhD in Management Information Systems from the University of Minnesota and is a member of Beta Gamma Sigma, AIS, ACM and IEEE.
- 1
Contact author. Note: A preliminary version of this paper appeared in the Proceedings of HICSS 2005.
- 2
Tel.: +1 813 974 6753/6754.