Skip to main content
Springer Nature Link
Log in

Image Annotation by Eye Tracking: Accuracy and Precision of Centerlines of Obstructed Small-Bowel Segments Placed Using Eye Trackers

  • Published:
Journal of Digital Imaging Aims and scope Submit manuscript

Abstract

Small-bowel obstruction (SBO) is a common and important disease, for which machine learning tools have yet to be developed. Image annotation is a critical first step for development of such tools. This study assesses whether image annotation by eye tracking is sufficiently accurate and precise to serve as a first step in the development of machine learning tools for detection of SBO on CT. Seven subjects diagnosed with SBO by CT were included in the study. For each subject, an obstructed segment of bowel was chosen. Three observers annotated the centerline of the segment by manual fiducial placement and by visual fiducial placement using a Tobii 4c eye tracker. Each annotation was repeated three times. The distance between centerlines was calculated after alignment using dynamic time warping (DTW) and statistically compared to clinical thresholds for diagnosis of SBO. Intra-observer DTW distance between manual and visual centerlines was calculated as a measure of accuracy. These distances were 1.1 ± 0.2, 1.3 ± 0.4, and 1.8 ± 0.2 cm for the three observers and were less than 1.5 cm for two of three observers (P < 0.01). Intra- and inter-observer DTW distances between centerlines placed with each method were calculated as measures of precision. These distances were 0.6 ± 0.1 and 0.8 ± 0.2 cm for manual centerlines, 1.1 ± 0.4 and 1.9 ± 0.6 cm for visual centerlines, and were less than 3.0 cm in all cases (P < 0.01). Results suggest that eye tracking–based annotation is sufficiently accurate and precise for small-bowel centerline annotation for use in machine learning–based applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+
from $39.99 /Month
  • Starting from 10 chapters or articles per month
  • Access and download chapters and articles from more than 300k books and 2,500 journals
  • Cancel anytime
View plans

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Explore related subjects

Discover the latest articles, books and news in related subjects, suggested using machine learning.

References

  1. Alajlan N: HopDSW: An approximate dynamic space warping algorithm for fast shape matching and retrieval. Journal of King Saud University - Computer and Information Sciences 23(1): 7–14, 2011. https://doi.org/10.1016/j.jksuci.2010.01.001. http://www.sciencedirect.com/science/article/pii/S1319157810000030

    Article  Google Scholar 

  2. Allen BC, Leyendecker JR: MR enterography for assessment and management of small bowel Crohn disease. Radiol Clin N Am 52(4): 799–810, 2014. https://doi.org/10.1016/j.rcl.2014.02.001

    Article  PubMed  Google Scholar 

  3. Aquina CT, Becerra AZ, Probst CP, Xu Z, Hensley BJ, Iannuzzi JC, Noyes K, Monson JRT, Fleming FJ: Patients with adhesive small bowel obstruction should be primarily managed by a surgical team. Ann Surg 264(3): 437–447, 2016. https://doi.org/10.1097/SLA.0000000000001861

    Article  PubMed  Google Scholar 

  4. Berndt DJ, Clifford J: Using dynamic time warping to find patterns in time series.. In: Proceedings of the 3rd international conference on knowledge discovery and data mining, AAAIWS’94. AAAI Press, Seattle, 1994, pp 359–370. http://dl.acm.org/citation.cfm?id=3000850.3000887

  5. Bruining DH, Zimmermann EM, Loftus EV, Sandborn WJ, Sauer CG, Strong SA: Society of Abdominal Radiology Crohn’s Disease-Focused Panel: Consensus Recommendations for Evaluation, Interpretation, and Utilization of Computed Tomography and Magnetic Resonance Enterography in Patients With Small Bowel Crohn’s Disease. Radiology 286(3): 776–799, 2018. https://doi.org/10.1148/radiol.2018171737

    Article  PubMed  Google Scholar 

  6. Chartrand G, Cheng PM, Vorontsov E, Drozdzal M, Turcotte S, Pal CJ, Kadoury S, Tang A: Deep learning: A primer for radiologists. RadioGraphics 37(7): 2113–2131, 2017. https://doi.org/10.1148/rg.2017170077. http://pubs.rsna.org/doi/10.1148/rg.2017170077

    Article  PubMed  Google Scholar 

  7. Dale AM, Fischl B, Sereno MI: Cortical surface-based analysis. I. Segmentation and surface reconstruction. NeuroImage 9(2): 179–194, 1999. https://doi.org/10.1006/nimg.1998.0395

    Article  CAS  PubMed  Google Scholar 

  8. Ebner L, Tall M, Choudhury KR, Ly DL, Roos JE, Napel S, Rubin GD: Variations in the functional visual field for detection of lung nodules on chest computed tomography: Impact of nodule size, distance, and local lung complexity. Med Phys 44(7): 3483–3490, 2017. https://doi.org/10.1002/mp.12277

    Article  PubMed  Google Scholar 

  9. Feuerstein JD, Cheifetz AS: Crohn disease: Epidemiology, diagnosis, and management. Mayo Clin Proc 92(7): 1088–1103, 2017. https://doi.org/10.1016/j.mayocp.2017.04.010

    Article  CAS  PubMed  Google Scholar 

  10. Fischl B, Salat DH, Busa E, Albert M, Dieterich M, Haselgrove C, van der Kouwe A, Killiany R, Kennedy D, Klaveness S, Montillo A, Makris N, Rosen B, Dale AM: Whole brain segmentation: automated labeling of neuroanatomical structures in the human brain. Neuron 33(3): 341–355, 2002

    Article  CAS  PubMed  Google Scholar 

  11. Frager D, Medwid SW, Baer JW, Mollinelli B, Friedman M: CT of small-bowel obstruction: value in establishing the diagnosis and determining the degree and cause. AJR Am J Roentgenol 162(1): 37–41, 1994. https://doi.org/10.2214/ajr.162.1.8273686

    Article  CAS  PubMed  Google Scholar 

  12. Frangi AF, Niessen WJ, Vincken KL, Viergever MA: Multiscale vessel enhancement filtering.. In: Medical Image Computing and Computer-Assisted Intervention — MICCAI’98, pp 130–137, 1998

  13. Frimmel H, Näppi J, Yoshida H: Fast and robust computation of colon centerline in CT colonography. Med Phys 31(11): 3046–3056, 2004. https://doi.org/10.1118/1.1790111

    Article  PubMed  Google Scholar 

  14. Frimmel H, Näppi J, Yoshida H: Centerline-based colon segmentation for CT colonography. Med Phys 32(8): 2665–2672, 2005. https://doi.org/10.1118/1.1990288

    Article  PubMed  Google Scholar 

  15. Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. Advances In Neural Information Processing Systems, 1–9

  16. Lecun Y, Bottou L, Bengio Y, Haffner P: Gradient-based learning applied to document recognition. Proc IEEE 86(11): 2278–2324, 1998. https://doi.org/10.1109/5.726791

    Article  Google Scholar 

  17. Li Q, Sone S, Doi K: Selective enhancement filters for nodules, vessels, and airway walls in two- and three-dimensional CT scans. Med Phys 30(8): 2040–2051, 2003. https://doi.org/10.1118/1.1581411

    Article  PubMed  Google Scholar 

  18. Litjens G, Toth R, van de Ven W, Hoeks C, Kerkstra S, van Ginneken B, Vincent G, Guillard G, Birbeck N, Zhang J, Strand R, Malmberg F, Ou Y, Davatzikos C, Kirschner M, Jung F, Yuan J, Qiu W, Gao Q, Edwards PE, Maan B, van der Heijden F, Ghose S, Mitra J, Dowling J, Barratt D, Huisman H, Madabhushi A: Evaluation of prostate segmentation algorithms for MRI: The PROMISE12 challenge. Med Image Anal 18(2): 359–373, 2014. https://doi.org/10.1016/j.media.2013.12.002

    Article  PubMed  Google Scholar 

  19. Lu L, Zhang D, Li L, Zhao J: Fully automated colon segmentation for the computation of complete colon centerline in virtual colonoscopy. IEEE Trans Biomed Eng 59(4): 996–1004, 2012. https://doi.org/10.1109/TBME.2011.2182051

    Article  PubMed  Google Scholar 

  20. Magdy N, Sakr MA, Mostafa T, El-Bahnasy K (2015) Review on trajectory similarity measures. In: 2015 IEEE 7th international conference on intelligent computing and information systems (ICICIS), pp 613–619. https://doi.org/10.1109/IntelCIS.2015.7397286

  21. Maiora J, Ayerdi B, Graña M: Random forest active learning for AAA thrombus segmentation in computed tomography angiography images. Neurocomputing 126: 71–77, 2014. https://doi.org/10.1016/j.neucom.2013.01.051. http://www.sciencedirect.com/science/article/pii/S0925231213006978

    Article  Google Scholar 

  22. Maiora J, Graña M (2012) Abdominal CTA image analisys through active learning and decision random forests: Aplication to AAA segmentation. In: The 2012 international joint conference on neural networks (IJCNN), pp 1–7. https://doi.org/10.1109/IJCNN.2012.6252801

  23. Metz CT, Schaap M, Weustink AC, Mollet NR, Van Walsum T, Niessen WJ: Coronary centerline extraction from CT coronary angiography images using a minimum cost path approach. Med Phys 36(12): 5568–5579, 2009. https://doi.org/10.1118/1.3254077

    Article  CAS  PubMed  Google Scholar 

  24. Milletari F, Navab N, Ahmadi SA (2016) V-Net: Fully convolutional neural networks for volumetric medical image segmentation, 1–11. arXiv:1606.04797

  25. Mittal D, Kumar V, Saxena SC, Khandelwal N, Kalra N: Neural network based focal liver lesion diagnosis using ultrasound images. Comput Med Imaging Graph 35(4): 315–323, 2011. https://doi.org/10.1016/j.compmedimag.2011.01.007. http://www.sciencedirect.com/science/article/pii/S0895611111000188

    Article  PubMed  Google Scholar 

  26. Papadopoulos DP, Clarke ADF, Keller F, Ferrari V: Training object class detectors from eye tracking data.. In: Computer Vision – ECCV 2014, Lecture Notes in Computer Science. Springer, Cham, 2014, pp 361–376, https://doi.org/10.1007/978-3-319-10602-1_24. https://link.springer.com/chapter/10.1007/978-3-319-10602-1_24

  27. Paulson EK, Thompson WM: Review of small-bowel obstruction: the diagnosis and when to worry. Radiology 275(2): 332–342, 2015. https://doi.org/10.1148/radiol.15131519

    Article  PubMed  Google Scholar 

  28. Ronneberger O, Fischer P, Brox T (2015) U-Net: Convolutional networks for biomedical image segmentation, 1–8. arXiv:1505.04597

  29. Rubin GD, Roos JE, Tall M, Harrawood B, Bag S, Ly DL, Seaman DM, Hurwitz LM, Napel S, Roy Choudhury K: Characterizing search, recognition, and decision in the detection of lung nodules on CT scans: Elucidation with eye tracking. Radiology 274(1): 276–286, 2015. https://doi.org/10.1148/radiol.14132918. http://pubs.rsna.org/doi/10.1148/radiol.14132918

    Article  PubMed  Google Scholar 

  30. Santillan CS: MR imaging techniques of the bowel. Magn Reson Imaging Clin N Am 22(1): 1–11, 2014. https://doi.org/10.1016/j.mric.2013.07.004

    Article  PubMed  Google Scholar 

  31. Scrima A, Lubner MG, King S, Pankratz J, Kennedy G, Pickhardt PJ: Value of MDCT and clinical and laboratory data for predicting the need for surgical intervention in suspected small-bowel obstruction. AJR Am J Roentgenol 208(4): 785–793, 2017. https://doi.org/10.2214/AJR.16.16946

    Article  PubMed  Google Scholar 

  32. Silva AC, Pimenta M, Guimarães LS: Small bowel obstruction: what to look for Radiographics: A review publication of the radiological. Society of North America, Inc 29(2): 423–439, 2009. https://doi.org/10.1148/rg.292085514

    Article  Google Scholar 

  33. Spuhler C (2006) Interactive centerline finding in complex tubular structures (16697)

  34. Spuhler C, Harders M, Szekely G: Fast and robust extraction of centerlines in 3d tubular structures using a scattered-snakelet approach. SPIE Medical Imaging 6144: 614442–614442–8, 2006. https://doi.org/10.1117/12.653169. http://proceedings.spiedigitallibrary.org/proceeding.aspx?articleid=1277145

    Article  Google Scholar 

  35. Tall M, Choudhury KR, Napel S, Roos JE, Rubin GD: Accuracy of a remote eye tracker for radiologic observer studies: Effects of calibration and recording environment. Acad Radiol 19(2): 196–202, 2012. https://doi.org/10.1016/j.acra.2011.10.011. http://www.sciencedirect.com/science/article/pii/S1076633211004946

    Article  PubMed  Google Scholar 

  36. Vaughan N, Gabrys B: Comparing and combining time series trajectories using dynamic time warping. Proc Comput Sci 96: 465–474, 2016. https://doi.org/10.1016/j.procs.2016.08.106. http://www.sciencedirect.com/science/article/pii/S187705091631907X

    Article  Google Scholar 

  37. Vincent G, Guillard G, Bowes M (2012) Fully automatic segmentation of the prostate using active appearance models, Medical image computing and computer-assisted intervention – MICCAI 2012. http://promise12.grand-challenge.org/Results/displayFile?resultId=20120629193617_302_Imorphics_Results&;type=Public&;file=Imorphics.pdf

  38. Wang S, Summers RM: Machine learning and radiology. Med Image Anal 16(5): 933–951, 2012. https://doi.org/10.1016/j.media.2012.02.005

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  39. Yasaka K, Akai H, Abe O, Kiryu S: Deep learning with convolutional neural network for differentiation of liver masses at dynamic contrast-enhanced CT: A preliminary study. Radiology 286(3): 887–896, 2017. https://doi.org/10.1148/radiol.2017170706. https://pubs.rsna.org/doi/abs/10.1148/radiol.2017170706

    Article  PubMed  Google Scholar 

  40. Yu L, Yang X, Chen H, Qin J, Heng PA (2017) Volumetric convnets with mixed residual connections for automated prostate segmentation from 3d mr images, 31st AAAI conference on artificial intelligence, pp 66–72

  41. Zar JH (2014) Biostatistical analysis, 5. ed., pearson new internat. ed edn. Always learning. Pearson Education Limited, Harlow. OCLC: 862984228

  42. Zhang W, Liu J, Yao J, Louie A, Nguyen TB, Wank S, Nowinski WL, Summers RM: Mesenteric vasculature-guided small bowel segmentation on 3d CT. IEEE Trans Med Imaging 32(11): 2006–2021, 2013. https://doi.org/10.1109/TMI.2013.2271487. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4224016/

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Funding

Dr. Kang Wang was supported in part by the NIH grant T32EB005970.

Author information

Authors and Affiliations

  1. Department of Bioengineering, University of California, San Diego, CA, USA

    Alfredo Lucas

  2. Department of Radiology, University of California, San Diego, CA, USA

    Kang Wang, Cynthia Santillan, Albert Hsiao, Claude B. Sirlin & Paul M. Murphy

Authors
  1. Alfredo Lucas
    View author publications

    Search author on:PubMed Google Scholar

  2. Kang Wang
    View author publications

    Search author on:PubMed Google Scholar

  3. Cynthia Santillan
    View author publications

    Search author on:PubMed Google Scholar

  4. Albert Hsiao
    View author publications

    Search author on:PubMed Google Scholar

  5. Claude B. Sirlin
    View author publications

    Search author on:PubMed Google Scholar

  6. Paul M. Murphy
    View author publications

    Search author on:PubMed Google Scholar

Corresponding author

Correspondence to Paul M. Murphy.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lucas, A., Wang, K., Santillan, C. et al. Image Annotation by Eye Tracking: Accuracy and Precision of Centerlines of Obstructed Small-Bowel Segments Placed Using Eye Trackers. J Digit Imaging 32, 855–864 (2019). https://doi.org/10.1007/s10278-018-0169-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10278-018-0169-5

Keywords