Abstract
Cross-platform development of medical applications in extended-reality (XR) head-mounted displays (HMDs) often relies on game engines with rendering capabilities currently not standardized in the context of medical visualizations. Many aspects of the visualization pipeline including the characterization of color have yet to be consistently defined across rendering models and platforms. We examined the transfer of color properties from digital objects, through the rendering and image processing steps, to the RGB values sent to the display device. Five rendering pipeline configurations within the Unity engine were evaluated using 24 digital color patches. In the second experiment, the same configurations were evaluated with a tissue slide sample image. Measurements of the change in color associated with each configuration were characterized using the CIE 1976 color difference (\({\Delta}{\text{E}}\)). We found that the distribution of \({\Delta}{\text{E}}\) for the first experiment ranges from zero, as in the case using an Unlit Shader, to 25.97, as in the case using default configurations. The default Unity configuration consistently returned the highest \({\Delta}{\text{E}}\) across all 24 colors and also the largest range of color differences. In the second experiment, \({\Delta}{\text{E}}\)E ranged from 7.49 to 34.18. The Unlit configuration resulted in the highest \({\Delta}{\text{E}}\) in three of four selected pixels in the tissue sample image. Changes in color image properties associated with texture import settings were then evaluated in a third experiment using the TG18-QC test pattern. Differences in pixel values were found in all nine of the investigated texture import settings. The findings provide an initial characterization of color transfer and a basis for future work on standardization, consistency, and optimization of color in medical XR applications.
Similar content being viewed by others
References
Valcasara N, Unreal Engine Game Development Blueprints. Packt Publishing, 2015.
Kim SL, Suk HJ, Kang JH, Jung JM, Laine TH, Westlin J, Using Unity 3D to facilitate mobile augmented reality game development, 2014 IEEE World Forum on Internet of Things (WF-IoT), 2014. https://doi.org/10.1109/WF-IoT.2014.6803110
Burley B, Physically-based shading at Disney. In ACM SIGGRAPH 2012 Courses, ACM, SIGGRAPH 2012.
Cook RL, Torrance KE, A reflectance model for computer graphics. ACM Trans. Graph. 1, 5, pp.7-24, 1982.
Walter B, Marschner SR, Li HS, Torrance KE, Microfacet models for refraction through rough surfaces, The Eurographics Association, Proceedings of the 18th Eurographics Conference on Rendering Techniques, pp.195-206, 2007.
Blinn JF, Models of light reflection for computer synthesized pictures. Proceedings of the 4th Annual Conference on Computer Graphics and Interactive Techniques, Association for Computing Machinery, New York, NY, pp.192-198, 1977. https://doi.org/10.1145/563858.563893
Schlick C, An inexpensive BRDF model for physically-based rendering, Computer Graphics Forum, 13(3), pp.223-246, 1994.
Cheng WC, Saleheen F, Badano A, Assessing color performance of whole-slide imaging scanners for digital pathology, Color Research & Application, 44(3), pp.322-334, 2019. https://doi.org/10.1002/col.22365
Samei E, Badano A, Chakraborty D, Compton K, Cornelius C, Corrigan K, Flynn MJ, Hemminger B, Hangiandreou N, Johnson J, and others, Assessment of display performance for medical imaging systems: executive summary of AAPM TG18 report, Medical Physics, 32(4), pp.1205-1225, 2005.
Unity. https://unity3d.com/. Accessed 15 Jan 2020
Wheeler G, Deng S, Toussaint N, Pushparajah K, Schnabel JA, Simpson JM, Gomez A, Virtual interaction and visualisation of 3D medical imaging data with VTK and Unity, Healthcare technology letters, 5(5), pp.148–153, 2018. https://doi.org/10.1049/htl.2018.5064
Seif M, Umeda R, Uehara Y, Higa H, A Data Conversion for Medical Training System, IEEJ 2016 Conference, 2016.
Wang R, Yao J, Wang L, Liu X, Wang H, Zheng L, A surgical training system for four medical punctures based on virtual reality and haptic feedback, 2017 IEEE Symposium on 3D User Interfaces (3DUI), pp.215-216, 2017. https://doi.org/10.1109/3DUI.2017.7893348
Peters TM, Linte CA, Yaniv Z, Williams J, Mixed and augmented reality in medicine, CRC, 2018.
Escobar-Castillejos D, Noguez J, Neri L, Magana A, Benes B, A Review of Simulators with Haptic Devices for Medical Training, Journal of Medical Systems, 40(104), 2016. https://doi.org/10.1007/s10916-016-0459-8
Unity Technologies, Unity Manual 2019.2: Standard Shader. https://docs.unity3d.com/Manual/shader-StandardShader.html. Accessed 15 Jan 2020.
Lagarde S, The High Definition Render Pipeline: Focused on visual quality. https://blogs.unity3d.com/2018/03/16/the-high-definition-render-pipeline-focused-on-visual-quality/. Accessed 15 Jan 2020.
Unity Technologies, Unity Manual 2019.2: Global Illuminance. https://docs.unity3d.com/Manual/GIIntro.html. Accessed 15 Jan 2020.
Khor WS, Baker B, Amin K, Chan A, Patel K, Woong J, Augmented and virtual reality in surgery the digital surgical environment: applications, limitations and legal pitfalls, Annals of translational medicine, 4(23), 2016.
Chen L, Day TW, Tang W, John NW, Recent developments and future challenges in medical mixed reality, 2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp.123-135, 2017.
Vávra P, Roman J, Zonča P, Ihnát P, Němec M, Kumar J, Habib N, El-Gendi A, Recent development of augmented reality in surgery: a review, Journal of healthcare engineering, 2017.
Unity Technologies, Unity Manual 2019.2: Textures. https://docs.unity3d.com/Manual/class-TextureImporter.html. Accessed 15 Jan 2020.
International Electrotechnical Commission and others, Multimedia systems and equipment-Colour measurement and management-Part 2-1: Colour management-Default RGB colour space-sRGB, IEC 61966-2-1, 1999.
Ohta N, Robertson A, Colorimetry: fundamentals and applications. John Wiley & Sons, 2016.
Unity Technologies, Unity Scripting API 2019.2: Texture Importer Platform Settings. docs.unity3d.com/ScriptReference/TextureImporterPlatformSettings.html. Accessed 15 Jan 2020.
Dille S, Fuhrmann A, Fischer G, Real-time tone mapping- An evaluation of color-accurate methods for luminance compression, Tagungsband des 22. Workshop Farbbildverarbeitung, Ilmenau, Germany, 2016.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Kim, A.S., Cheng, WC., Beams, R. et al. Color Rendering in Medical Extended-Reality Applications. J Digit Imaging 34, 16–26 (2021). https://doi.org/10.1007/s10278-020-00392-4
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10278-020-00392-4