Skip to main content

Advertisement

SpringerLink
Log in

Towards a Clinical Trial Protocol to Evaluate Health Information Systems: Evaluation of a Computerized System for Monitoring Tuberculosis from a Patient Perspective in Brazil

  • Systems-Level Quality Improvement
  • Published:
Journal of Medical Systems Aims and scope Submit manuscript

Abstract

Assessment of health information systems consider different aspects of the system itself. They focus or on the professional who will use the software or on its usability or on the software engineering metrics or on financial and managerial issues. The existent approaches are very resources consuming, disconnected, and not standardized. As the software becomes more critical in the health organizations and in patients, becoming used as a medical device or a medicine, there is an urgency to identify tools and methods that can be applied in the development process. The present work is one of the steps of a broader study to identify standardized protocols to evaluate the health information systems as medicines and medical devices are evaluated by clinical trials. The goal of the present work was to evaluate the effect of the introduction of an information system for monitoring tuberculosis treatment (SISTB) in a Brazilian municipality from the patients’ perspective. The Patient Satisfaction Questionnaire and the Hospital Consumer Assessment of Healthcare Providers and Systems were answered by the patients before and after the SISTB introduction, for comparison. Patients from an outpatient clinic, formed the control group, that is, at this site was not implanted the SISTB. Descriptive statistics and mixed effects model were used for data analysis. Eighty-eight interviews were conducted in the study. The questionnaire’s results presented better averages after the system introduction but were not considered statistically significant. Therefore, it was not possible to associate system implantation with improved patient satisfaction. The HIS evaluation need be complete, the technical and managerial evaluation, the safety, the impact on the professionals and direct and/or indirect impact on patients are important. Developing the right tools and methods that can evaluate the software in its entirety, from the beginning of the development cycle with a normalized scale, are needed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. WHO (2017) Global tuberculosis report 2017

  2. WHO (2016) Global Tuberculosis Report 2016

  3. Creswell, J., Sahu, S., Sachdeva, K. S., Ditiu, L., Barreira, D., Mariandyshev, A., Mingting, C., and Pillay, Y., Tuberculosis in BRICS: Challenges and opportunities for leadership within the post-2015 agenda. Bull World Health Organ 92:459–460, 2014. https://doi.org/10.2471/BLT.13.133116.

    Article  PubMed  PubMed Central  Google Scholar 

  4. National Collaborating Centre for Chronic Conditions (UK); Centre for Clinical Practice at NICE (UK), Tuberculosis: Clinical diagnosis and Management of Tuberculosis, and measures for its prevention and control, CG 117. London: National Institute for Health and Clinical Excellence (UK), 2011.

    Google Scholar 

  5. Bedard, B. A., Younge, M., Pettit, P. A., and Mendoza, M., Using telemedicine for tuberculosis care management: A three county inter-municipal approach. J Med Syst 42:12, 2018. https://doi.org/10.1007/s10916-017-0872-7.

    Article  Google Scholar 

  6. Kopanitsa, G., A qualitative study of the barriers and opportunities for adoption of web-portals for doctors and patients in Russia. J Med Syst 41:62, 2017. https://doi.org/10.1007/s10916-017-0713-8.

    Article  PubMed  Google Scholar 

  7. Cruz-Cunha, M. M., Simoes, R., Varajão, J., and Miranda, I., Information technology supporting healthcare and social care services. J Inf Technol Res 7:41–58, 2014. https://doi.org/10.4018/jitr.2014010104.

    Article  Google Scholar 

  8. Costa, C. J., Aparicio, M., and Figueiredo, J. P., Health portal: An alternative using open source technology. Int J Web Portals 4:1–18, 2012. https://doi.org/10.4018/jwp.2012100101.

    Article  Google Scholar 

  9. Cruz-Cunha MM, Miranda IM, Gonçalves P (2013) Handbook of research on ICTs and Management Systems for Improving Efficiency in healthcare and social care. IGI Global

  10. Nabuco, O., Delivering deep health information using clinical eye. Int J Web Portals 5:28–40, 2013. https://doi.org/10.4018/jwp.2013010103.

    Article  Google Scholar 

  11. Palha, P. F., da Silva, L. M. C., Wysocki, A. D. et al., Acesso aos serviços de atenção à tuberculose: Análise da satisfação dos doentes. Rev da Esc Enferm 46:342–348, 2012. https://doi.org/10.1590/S0080-62342012000200011.

    Article  Google Scholar 

  12. Nezenega, Z. S., Gacho, Y. H., and Tafere, T. E., Patient satisfaction on tuberculosis treatment service and adherence to treatment in public health facilities of Sidama zone, South Ethiopia. BMC Health Serv Res 13:110, 2013. https://doi.org/10.1186/1472-6963-13-110.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Hung, S.-Y., Hung, W.-H., Tsai, C.-A., and Jiang, S.-C., Critical factors of hospital adoption on CRM system: Organizational and information system perspectives. Decis Support Syst 48:592–603, 2010. https://doi.org/10.1016/j.dss.2009.11.009.

    Article  Google Scholar 

  14. Kukkonen HO, Räisänen T, Hummastenniemi N (2008) Patient relationship management An Overview and Study of a Follow-Up System. US Pat App 09/953,824 22:53, 54, 62, 78

  15. Rijo RPCL (2008) Framework para a Gestão de Projectos de Sistemas de Informação de Contact Centers. Tras-os-Montes University

  16. Ammenwerth, E., Gräber, S., Herrmann, G., Bürkle, T., and König, J., Evaluation of health information systems—Problems and challenges. Int J Med Inform 71:125–135, 2003. https://doi.org/10.1016/S1386-5056(03)00131-X.

    Article  PubMed  Google Scholar 

  17. Yusof, M. M., Papazafeiropoulou, A., Paul, R. J., and Stergioulas, L. K., Investigating evaluation frameworks for health information systems. Int J Med Inform 77:377–385, 2008. https://doi.org/10.1016/j.ijmedinf.2007.08.004.

    Article  PubMed  Google Scholar 

  18. Anderson JG, Aydin CE (2005) Evaluating the Organizational Impact of Healthcare Information System, 2nd ed

  19. Hessels, A., Linda, F., Cimiotti, J. P. et al., Impact of heath information technology on the quality of. Patient Care. 3:973–982, 2016. https://doi.org/10.1016/S2215-0366(16)30284-X.Epidemiology.

    Article  Google Scholar 

  20. Koide, D., Asonuma, M., Naito, K. et al., Evaluation of electronic health records from viewpoint of patients. Stud Heal Technol Informatics 122:304–308, 2006.

    Google Scholar 

  21. Lourenço, J., Santos-Pereira, C., Rijo, R., and Cruz-Correia, R., Service level agreement of information and communication Technologies in Portuguese Hospitals. Procedia Technol 16:1397–1402, 2014. https://doi.org/10.1016/j.protcy.2014.10.158.

    Article  Google Scholar 

  22. Alkureishi, M. A., Lee, W. W., Lyons, M., Press, V. G., Imam, S., Nkansah-Amankra, A., Werner, D., and Arora, V. M., Impact of electronic medical record use on the patient–doctor relationship and communication: A systematic review. J Gen Intern Med 31:548–560, 2016. https://doi.org/10.1007/s11606-015-3582-1.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Lee, W. W., Alkureishi, M. A., Ukabiala, O., Venable, L. R., Ngooi, S. S., Staisiunas, D. D., Wroblewski, K. E., and Arora, V. M., Patient perceptions of electronic medical record use by faculty and resident physicians: A mixed methods study. J Gen Intern Med 31:1315–1322, 2016. https://doi.org/10.1007/s11606-016-3774-3.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Shield, R. R., Goldman, R. E., Anthony, D. A., Wang, N., Doyle, R. J., and Borkan, J., Gradual electronic health record implementation: New insights on physician and patient adaptation. Ann Fam Med 8:316–326, 2010. https://doi.org/10.1370/afm.1136.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Sligo, J., Gauld, R., Roberts, V., and Villa, L., A literature review for large-scale health information system project planning, implementation and evaluation. Int J Med Inform 97:86–97, 2017. https://doi.org/10.1016/j.ijmedinf.2016.09.007.

    Article  PubMed  Google Scholar 

  26. Rijo, R., Martinho, R., and Ermida, D., Developing an Enterprise architecture proof of concept in a Portuguese hospital. Procedia Comput Sci 64:1217–1225, 2015. https://doi.org/10.1016/j.procs.2015.08.511.

    Article  Google Scholar 

  27. Rijo, R. P. C. L., Crepaldi, N. Y., Bergamini, F., Rodrigues, L. M. L., de Lima, I. B., da Silva Castro Perdoná, G., and Alves, D., Impact assessment on patients’ satisfaction and healthcare professionals’ commitment of software supporting directly observed treatment, short-course: A protocol proposal. Health Informatics J, 2017. https://doi.org/10.1177/1460458217712057.

  28. Marshall G, Hays R (1994) The Patient Satisfaction Questionnaire Short-Form (PSQ-18)

  29. Thayaparan, A. J., and Mahdi, E., The patient satisfaction questionnaire short form (PSQ-18) as an adaptable, reliable, and validated tool for use in various settings. Med Educ Online 18, 2013. https://doi.org/10.3402/meo.v18i0.21747.

  30. National Quality Forum (2013) Hospital Consumer Assessment of Healthcare Providers and Systems (H-CAHPS)

  31. Cronbach, L. J., Coefficient alpha and the internal structure of tests. Psychometrika 16:297–334, 1951. https://doi.org/10.1007/BF02310555.

    Article  Google Scholar 

  32. Boonstra, A., and Broekhuis, M., Barriers to the acceptance of electronic medical records by physicians from systematic review to taxonomy and interventions. BMC Health Serv Res 10:231, 2010. https://doi.org/10.1186/1472-6963-10-231.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Kruse, C. S., Kristof, C., Jones, B., Mitchell, E., and Martinez, A., Barriers to electronic health record adoption: A systematic literature review. J Med Syst 40:252, 2016. https://doi.org/10.1007/s10916-016-0628-9.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Crepaldi, N. Y., de Lima, I. B., Vicentine, F. B., Rodrigues, L. M. L., Yamaguti, V. H., Sanches, T. L. M., Ruffino-Netto, A., Alves, D., and Rijo, R. P. C. L., Satisfaction evaluation of health professionals in the usability of software for monitoring the tuberculosis treatment. Procedia Comput Sci 121:889–896, 2017. https://doi.org/10.1016/j.procs.2017.11.115.

    Article  Google Scholar 

  35. Lafaiete, R., Motta, M., and Villa, T., Satisfação dos usuários no programa de controle da tuberculose de um município do Rio de Janeiro, Brasil. Rev Latino-Am Enferm 19:1–7, 2011.

    Article  Google Scholar 

  36. Portela, M. C., Lima, S. M. L., Brito, C., Ferreira, V. M. B., Escosteguy, C. C., and Vasconcellos, M. T. L., Tuberculosis control program and patient satisfaction, Rio de Janeiro, Brazil. Rev Saude Publica 48:497–507, 2014. https://doi.org/10.1590/S0034-8910.2014048004793.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Silva ML da, Junior LAV (2016) Manual de Certificação para Sistemas de Registro Eletrônico em Saúde

Download references

Acknowledgements

The authors would like to thank the team of the Hospital das Clínicas, and all the TB reference outpatient clinics, in the Ribeirão Preto city, that participated with a clinical team in this research and provided the conditions for this work.

Author information

Authors and Affiliations

  1. Department of Social Medicine, Ribeirão Preto Medical School, University of São Paulo, Ribeirão Preto, 14049-900, Brazil

    Nathalia Yukie Crepaldi, Fernanda Bergamini Vicentine, Lídia Maria Lourençon Rodrigues, Antonio Ruffino-Netto & Domingos Alves

  2. Intelligence Laboratory in Health, Ribeirão Preto Medical School, University of São Paulo, Ribeirão Preto, 14049-900, Brazil

    Nathalia Yukie Crepaldi, Inacia Bezerra de Lima, Tiago Lara Michelin Sanches, Domingos Alves & Rui Pedro Charters Lopes Rijo

  3. Ribeirão Preto College of Nursing, University of São Paulo, Ribeirão Preto, 14040-902, Brazil

    Inacia Bezerra de Lima

  4. School of Technology and Management, Polytechnic Institute of Leiria, 2411-901, Leiria, Portugal

    Rui Pedro Charters Lopes Rijo

  5. INESCC - Institute for Systems and Computers Engineering at Coimbra, 3000-033, Coimbra, Portugal

    Rui Pedro Charters Lopes Rijo

  6. CINTESIS - Center for Research in Health Technologies and Information Polytechnic Institute of Leiria, 4200-450, Porto, Portugal

    Rui Pedro Charters Lopes Rijo

Authors
  1. Nathalia Yukie Crepaldi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Inacia Bezerra de Lima
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fernanda Bergamini Vicentine
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lídia Maria Lourençon Rodrigues
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Tiago Lara Michelin Sanches
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Antonio Ruffino-Netto
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Domingos Alves
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Rui Pedro Charters Lopes Rijo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nathalia Yukie Crepaldi.

Ethics declarations

Funding

This study was funded by Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES, process 88881.068176 / 2014–01 and 88887.137749/2017–00), Foundation of the Brazilian Ministry of Education. This study was also funded by the Brazilian Ministry of Health (process 796767/2013).

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

In the work development, all the ethical requirements prescribed by Resolution 466/12 of the Brazil National Health Council and its complementary ones were fulfilled, being approved by the Research Ethics Committee of the Medical School of Ribeirão Preto of the University of São Paulo, Certificate of Presentation for Ethical Assessment 44813815.6.0000.5440 and 69187617.7.0000.5440.

This article does not contain any studies with animals performed by any of the authors.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Additional information

This article is part of the Topical Collection on Systems-Level Quality Improvement

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Crepaldi, N.Y., de Lima, I.B., Vicentine, F.B. et al. Towards a Clinical Trial Protocol to Evaluate Health Information Systems: Evaluation of a Computerized System for Monitoring Tuberculosis from a Patient Perspective in Brazil. J Med Syst 42, 113 (2018). https://doi.org/10.1007/s10916-018-0968-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10916-018-0968-8

Keywords

Search

Navigation