Skip to main content

Procedurally Generated Colonoscopy and Laparoscopy Data for Improved Model Training Performance

  • Conference paper
  • First Online:
Data Engineering in Medical Imaging (DEMI 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14314))

Included in the following conference series:

  • 513 Accesses

Abstract

The use of synthetic/simulated data can greatly improve model training performance, especially in areas such as image guided surgery, where real training data can be difficult to obtain, or of limited size. Procedural generation of data allows for large datasets to be rapidly generated and automatically labelled, while also randomising relevant parameters within the simulation to provide a wide variation in models and textures used in the scene.

A method for procedural generation of both textures and geometry for IGS data is presented, using Blender Shader Graphs and Geometry Nodes, with synthetic datasets used to pre-train models for polyp detection (YoloV7) and organ segmentation (UNet), with performance evaluated on open-source datasets.

Pre-training models with synthetic data significantly improves both model performance and generalisability (i.e. performance when evaluated on other datasets). Mean DICE score across all models for liver segmentation increased by 15% (p=0.02) after pre-training on synthetic data. For polyp detection, Precision increased by 11% (p=0.002), Recall by 9% (p=0.01), mAP@.5 by 10% (p=0.01) and mAP@[.5:95] by 8% (p-0.003).

All synthetic data, as well as examples of different Shader Graph/Geometry Node operations can be downloaded at https://doi.org/10.5522/04/23843904.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Ali, S., et al.: Polypgen: a multi-center polyp detection and segmentation dataset for generalisability assessment (June 2021)

    Google Scholar 

  2. Borgli, H., et al.: Hyperkvasir, a comprehensive multi-class image and video dataset for gastrointestinal endoscopy. Sci. Data 7 (2020). https://doi.org/10.1038/s41597-020-00622-y

  3. Carstens, M., et al.: The dresden surgical anatomy dataset for abdominal organ segmentation in surgical data science. Sci. Data 10, 3 (2023). https://doi.org/10.1038/s41597-022-01719-2

    Article  Google Scholar 

  4. Funke, I., et al.: Generating large labeled data sets for laparoscopic image processing tasks using unpaired image-to-image translation. CoRR (2019)

    Google Scholar 

  5. Hong, W.Y., Kao, C.L., Kuo, Y.H., Wang, J.R., Chang, W.L., Shih, C.S.: Cholecseg8k: a semantic segmentation dataset for laparoscopic cholecystectomy based on cholec80 (Nov 2020)

    Google Scholar 

  6. Jagtap, A.D., Heinrich, M., Himstedt, M.: Automatic generation of synthetic colonoscopy videos for domain randomization (May 2022)

    Google Scholar 

  7. Li, K., et al.: Colonoscopy polyp detection and classification: dataset creation and comparative evaluations. PLoS ONE 16, e0255809 (2021). https://doi.org/10.1371/journal.pone.0255809

    Article  Google Scholar 

  8. Ma, Y., Chen, X., Cheng, K., Li, Y., Sun, B.: LDPolypVideo Benchmark: A Large-Scale Colonoscopy Video Dataset of Diverse Polyps, pp. 387–396 (2021). https://doi.org/10.1007/978-3-030-87240-3_37

  9. Moreu, E., McGuinness, K., O’Connor, N.E.: Synthetic data for unsupervised polyp segmentation (Feb 2022)

    Google Scholar 

  10. Hinterstoisser, S., Pauly, O., Heibel, H., Marek, M., Bokeloh, M.: An annotation saved is an annotation earned: using fully synthetic training for object instance detection. In: Computer Vision and Pattern Recognition (2019)

    Google Scholar 

  11. Rivoir, D., et al.: Long-term temporally consistent unpaired video translation from simulated surgical 3d data (2021). https://doi.org/10.1109/ICCV48922.2021.00333

  12. Wang, C.Y., Bochkovskiy, A., Liao, H.Y.M.: Yolov7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors (July 2022)

    Google Scholar 

  13. Wood, E., Baltrušaitis, T., Hewitt, C., Dziadzio, S., Cashman, T.J., Shotton, J.: Fake it till you make it: face analysis in the wild using synthetic data alone (2021). https://doi.org/10.1109/ICCV48922.2021.00366

  14. Yonghao Long, Siu Hin Fan, Q.D.Y.W.: Neural rendering for stereo 3d reconstruction of deformable tissues in robotic surgery

    Google Scholar 

  15. Yoon, D., et al.: Colonoscopic image synthesis with generative adversarial network for enhanced detection of sessile serrated lesions using convolutional neural network. Sci. Rep. 12, 261 (2022). https://doi.org/10.1038/s41598-021-04247-y

    Article  Google Scholar 

  16. Yoon, J., et al.: Surgical Scene Segmentation Using Semantic Image Synthesis with Virtual Surgery Environment, pp. 551–561 (2022). https://doi.org/10.1007/978-3-031-16449-1_53

Download references

Acknowledgments

This work was funded by EPSRC (EP/V052438/1), and supported by a Microsoft Azure Research Grant, and an Oracle Cloud Computing Grant.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Thomas Dowrick .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Dowrick, T., Chen, L., Ramalhinho, J., Puyal, J.GB., Clarkson, M.J. (2023). Procedurally Generated Colonoscopy and Laparoscopy Data for Improved Model Training Performance. In: Bhattarai, B., et al. Data Engineering in Medical Imaging. DEMI 2023. Lecture Notes in Computer Science, vol 14314. Springer, Cham. https://doi.org/10.1007/978-3-031-44992-5_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-44992-5_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-44991-8

  • Online ISBN: 978-3-031-44992-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics