Abstract
Pancreatic cancer has one of the worst survival rates amongst all forms of cancer because its symptoms manifest later into the progression of the disease. One of those symptoms is jaundice, the yellow discoloration of the skin and sclera due to the buildup of bilirubin in the blood. Jaundice is only recognizable to the naked eye in severe stages, but a ubiquitous test using computer vision and machine learning can detect milder forms of jaundice. We propose BiliScreen, a smartphone app that captures pictures of the eye and produces an estimate of a person's bilirubin level, even at levels normally undetectable by the human eye. We test two low-cost accessories that reduce the effects of external lighting: (1) a 3D-printed box that controls the eyes' exposure to light and (2) paper glasses with colored squares for calibration. In a 70-person clinical study, we found that BiliScreen with the box achieves a Pearson correlation coefficient of 0.89 and a mean error of -0.09 ± 2.76 mg/dl in predicting a person's bilirubin level. As a screening tool, BiliScreen identifies cases of concern with a sensitivity of 89.7% and a specificity of 96.8% with the box accessory.
Supplemental Material
Available for Download
Supplemental movie, appendix, image and software files for, BiliScreen: Smartphone-Based Scleral Jaundice Monitoring for Liver and Pancreatic Disorders
- Alireza Abdolvahabi, Brandon W. Taylor, Rebecca L. Holden, Elizabeth V Shaw, Alex Kentsis, Carlos Rodriguez-Galindo, Shizuo Mukai, and Bryan F. Shaw. Colorimetric and longitudinal analysis of leukocoria in recreational photographs of children with retinoblastoma. PloS one, 8(10):e76677, oct 2013.Google ScholarCross Ref
- American Cancer Society. Cancer Facts 8 Figures 2016. Technical report, American Cancer Society, Atlanta, GA, 2016.Google Scholar
- Andrew Bastawrous, Hillary K. Rono, Iain A. T. Livingstone, Helen A. Weiss, Stewart Jordan, Hannah Kuper, and Matthew J. Burton. Development and Validation of a Smartphone-Based Visual Acuity Test (Peek Acuity) for Clinical Practice and Community-Based Fieldwork. JAMA Ophthalmology, 133(8):930, aug 2015.Google ScholarCross Ref
- Vinod K Bhutani, Lois Johnson, and Emidio M Sivieri. Predictive ability of a predischarge hour-specific serum bilirubin for subsequent significant hyperbilirubinemia in healthy term and near-term newborns. Pediatrics, 103(1):6--14, 1999.Google ScholarCross Ref
- Giles Bond-Smith, Neal Banga, Toby M Hammond, and Charles J Imber. Pancreatic adenocarcinoma. BMJ, 344, 2012.Google Scholar
- Yuri Y Boykov and Marie-Pierre Jolly. Interactive graph cuts for optimal boundary 8 region segmentation of objects in ND images. In Proc. ICCV ’01, volume 1, pages 105--112. IEEE, 2001.Google Scholar
- Simona Crihalmeanu and Arun Ross. Multispectral scleral patterns for ocular biometric recognition. Pattern Recognition Letters, 33(14):1860--1869, 2012. Google ScholarDigital Library
- Abhijit Das, Umapada Pal, Miguel Angel Ferrer Ballester, and Michael Blumenstein. A new efficient and adaptive sclera recognition system. In 2014 IEEE Symposium on Computational Intelligence in Biometrics and Identity Management (CIBIM), pages 1--8. IEEE, dec 2014.Google ScholarCross Ref
- Abhijit Das, Umapada Pal, Miguel Angel Ferrer Ballester, and Michael Blumenstein. Sclera recognition using dense-SIFT. In International Conference on Intelligent Systems Design and Applications, ISDA, pages 74--79. IEEE, dec 2014.Google Scholar
- Lilian de Greef, Mayan Goel, Min Joon Seo, Eric C Larson, James W Stout, James A Taylor, and Shwetak N Patel. Bilicam: using mobile phones to monitor newborn jaundice. In Proc. UbiComp ’14, pages 331--342, 2014. Google ScholarDigital Library
- Maria Syl D De La Cruz, Alisa P Young, and Mack T Ruffin. Diagnosis and management of pancreatic cancer. American family physician, 89(8):626--32, apr 2014.Google Scholar
- Mario E Giardini, Iain A T Livingstone, Stewart Jordan, Nigel M Bolster, Tunde Peto, Matthew Burton, and Andrew Bastawrous. A smartphone based ophthalmoscope. Proc. EMBC ’14, 2014:2177--2180, 2014.Google ScholarCross Ref
- Dorothy M Greig, Bruce T Porteous, and Allan H Seheult. Exact maximum a posteriori estimation for binary images. Series of the Royal Statistical Society, 51(2):271--279, 1989.Google ScholarCross Ref
- G. Heath. The episclera, sclera and conjunctiva. An overview of relevant ocular anatomy. Differential Diagnosis of Ocular Disease, 9(2):36--42, 2006.Google Scholar
- Gudrun J Klinker, Steven A Shafer, and Takeo Kanade. A physical approach to color image understanding. International Journal of Computer Vision, 4(1):7--38, 1990. Google ScholarDigital Library
- LF Kozachenko and NN Leonenko. Sample estimate of the entropy of a random vector. Problemy Peredachi Informatsii, 23(2):9--16, 1987.Google Scholar
- Everett Lawson, Jason Boggess, Siddharth Khullar, Alex Olwal, Gordon Wetzstein, and Ramesh Raskar. Computational retinal imaging via binocular coupling and indirect illumination. In Proc. SIGGRAPH ’12, page 51, 2012. Google ScholarDigital Library
- Somsak Leartveravat. Transcutaneous bilirubin measurement in full term neonate by digital camera. Medical Journal of Srisaket Surin Buriram Hospitals, 24(1):105--118, 2009.Google Scholar
- Terence S Leung, Karan Kapur, Ashley Guilliam, Jade Okell, Bee Lim, Lindsay W MacDonald, and Judith Meek. Screening neonatal jaundice based on the sclera color of the eye using digital photography. Biomedical optics express, 6(11):4529--4538, 2015.Google Scholar
- Rainer Lienhart and Jochen Maydt. An extended set of Haar-like features for rapid object detection. In Proceedings. International Conference on Image Processing, volume 1, pages 0--3, 2002.Google ScholarCross Ref
- Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, volume 07-12-June, pages 3431--3440, mar 2015.Google ScholarCross Ref
- Dan L Longo, Anthony S Fauci, Dennis L Kasper, Stephen L Hauser, J Larry Jameson, and Joseph Loscalzo. Harrison’s Principles of Internal Medicine. 18th edition, 2006.Google Scholar
- Marco Marcon, Eliana Frigerio, and Stefano Tubaro. Sclera segmentation for gaze estimation and iris localization in unconstrained images. In CompIMAGE, pages 25--29, 2012.Google Scholar
- Nobuyuki Otsu. A threshold selection method from gray-level histograms. IEEE Transactions on Systems, Man, and Cybernetics, 9(1):62--66, jan 1979.Google ScholarCross Ref
- Christopher G Owen, Richard SB Newsom, Alicja R Rudnicka, Sarah A Barman, and E Goeffrey Woodward. Diabetes and the tortuosity of vessels of the bulbar conjunctiva. Ophthalmology, 115(6):e27--e32, 2008.Google ScholarCross Ref
- Christopher G Owen, Richard SB Newsom, Alicja R Rudnicka, and Tim J Ellis. Vascular response of the bulbar conjunctiva to diabetes and elevated blood pressure. Ophthalmology, 112(10):1801--1808, 2005.Google ScholarCross Ref
- Vitor F Pamplona, Ankit Mohan, Manuel M Oliveira, and Ramesh Raskar. NETRA: interactive display for estimating refractive errors and focal range. ACM transactions on graphics (TOG), 29(4):77, 2010. Google ScholarDigital Library
- Vitor F Pamplona, Erick B Passos, Jan Zizka, Manuel M Oliveira, Everett Lawson, Esteban Clua, and Ramesh Raskar. Catra: cataract probe with a lightfield display and a snap-on eyepiece for mobile phones. In Proc. SIGGRAPH ’11, pages 7--11, 2011.Google Scholar
- Danny Pascale. RGB coordinates of the Macbeth ColorChecker. Technical report, 2006.Google Scholar
- Carsten Rother, Vladimir Kolmogorov, and Andrew Blake. Grabcut: Interactive foreground extraction using iterated graph cuts. ACM Transactions on Graphics (TOG) ’04, 23(3):309--314, 2004. Google ScholarDigital Library
- Mario A Ruiz, Sammy Saab, and Leland S Rickman. The clinical detection of scleral icterus: observations of multiple examiners. Military medicine, 162(8):560--563, 1997.Google ScholarCross Ref
- Gaurav Sharma, Wencheng Wu, and Edul N. Dalal. The CIEDE2000 color-difference formula: Implementation notes, supplementary test data, and mathematical observations. Color Research and Application, 30(1):21--30, feb 2005.Google ScholarCross Ref
- DO Sillence, Alison Senn, and DM Danks. Genetic heterogeneity in osteogenesis imperfecta. Journal of medical genetics, 16(2):101--116, 1979.Google Scholar
- Audrey Vincent, Joseph Herman, Rich Schulick, Ralph H Hruban, and Michael Goggins. Pancreatic cancer. The Lancet, 378(9791):607--620, aug 2011.Google ScholarCross Ref
- Paul Viola and Michael J Jones. Rapid object detection using a boosted cascade of simple features. In Computer Vision and Pattern Recognition (CVPR), volume 1, pages 511--518, 2001.Google ScholarCross Ref
- Russell Wiesner, Erick Edwards, Richard Freeman, Ann Harper, Ray Kim, Patrick Kamath, Walter Kremers, John Lake, Todd Howard, Robert M. Merion, Robert A. Wolfe, Ruud Krom, and United Network for Organ Sharing Liver Disease Severity Score Committee. Model for end-stage liver disease (MELD) and allocation of donor livers. Gastroenterology, 124(1):91--96, jan 2003.Google ScholarCross Ref
- Stephen Wolf. Color correction matrix for digital still and video imaging systems. pages 1--40, 2003.Google Scholar
- Zhi Zhou, Eliza Yingzi Du, N. Luke Thomas, and Edward J. Delp. A New Human Identification Method: Sclera Recognition. Systems, Man and Cybernetics, Part A: Systems and Humans, IEEE Transactions on, 42(3):571--583, may 2012. Google ScholarDigital Library
Index Terms
- BiliScreen: Smartphone-Based Scleral Jaundice Monitoring for Liver and Pancreatic Disorders
Recommendations
Bilicam: using mobile phones to monitor newborn jaundice
UbiComp '14: Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous ComputingHealth sensing through smartphones has received considerable attention in recent years because of the devices' ubiquity and promise to lower the barrier for tracking medical conditions. In this paper, we focus on using smartphones to monitor newborn ...
Research and Design of Neonatal Jaundice Detector Based on Color Sensor
DMIP '21: Proceedings of the 2021 4th International Conference on Digital Medicine and Image ProcessingObjective: To solve the problem of invasive detection of neonatal jaundice and prone to infection. Methods: A neonatal jaundice detector based on color sensing was proposed. The microprocessor controls the color sensor to read the RGB color components of ...
BiliCam: using mobile phones to monitor newborn jaundice
UbiComp '14 Adjunct: Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct PublicationHealth sensing through smartphones has received considerable attention in recent years because of its promise to lower costs and provide more continuous data for tracking medical conditions. In this poster, we focus on using smartphones to sense newborn ...
Comments