Abstract
The use of synthetic/simulated data can greatly improve model training performance, especially in areas such as image guided surgery, where real training data can be difficult to obtain, or of limited size. Procedural generation of data allows for large datasets to be rapidly generated and automatically labelled, while also randomising relevant parameters within the simulation to provide a wide variation in models and textures used in the scene.
A method for procedural generation of both textures and geometry for IGS data is presented, using Blender Shader Graphs and Geometry Nodes, with synthetic datasets used to pre-train models for polyp detection (YoloV7) and organ segmentation (UNet), with performance evaluated on open-source datasets.
Pre-training models with synthetic data significantly improves both model performance and generalisability (i.e. performance when evaluated on other datasets). Mean DICE score across all models for liver segmentation increased by 15% (p=0.02) after pre-training on synthetic data. For polyp detection, Precision increased by 11% (p=0.002), Recall by 9% (p=0.01), mAP@.5 by 10% (p=0.01) and mAP@[.5:95] by 8% (p-0.003).
All synthetic data, as well as examples of different Shader Graph/Geometry Node operations can be downloaded at https://doi.org/10.5522/04/23843904.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Ali, S., et al.: Polypgen: a multi-center polyp detection and segmentation dataset for generalisability assessment (June 2021)
Borgli, H., et al.: Hyperkvasir, a comprehensive multi-class image and video dataset for gastrointestinal endoscopy. Sci. Data 7 (2020). https://doi.org/10.1038/s41597-020-00622-y
Carstens, M., et al.: The dresden surgical anatomy dataset for abdominal organ segmentation in surgical data science. Sci. Data 10, 3 (2023). https://doi.org/10.1038/s41597-022-01719-2
Funke, I., et al.: Generating large labeled data sets for laparoscopic image processing tasks using unpaired image-to-image translation. CoRR (2019)
Hong, W.Y., Kao, C.L., Kuo, Y.H., Wang, J.R., Chang, W.L., Shih, C.S.: Cholecseg8k: a semantic segmentation dataset for laparoscopic cholecystectomy based on cholec80 (Nov 2020)
Jagtap, A.D., Heinrich, M., Himstedt, M.: Automatic generation of synthetic colonoscopy videos for domain randomization (May 2022)
Li, K., et al.: Colonoscopy polyp detection and classification: dataset creation and comparative evaluations. PLoS ONE 16, e0255809 (2021). https://doi.org/10.1371/journal.pone.0255809
Ma, Y., Chen, X., Cheng, K., Li, Y., Sun, B.: LDPolypVideo Benchmark: A Large-Scale Colonoscopy Video Dataset of Diverse Polyps, pp. 387–396 (2021). https://doi.org/10.1007/978-3-030-87240-3_37
Moreu, E., McGuinness, K., O’Connor, N.E.: Synthetic data for unsupervised polyp segmentation (Feb 2022)
Hinterstoisser, S., Pauly, O., Heibel, H., Marek, M., Bokeloh, M.: An annotation saved is an annotation earned: using fully synthetic training for object instance detection. In: Computer Vision and Pattern Recognition (2019)
Rivoir, D., et al.: Long-term temporally consistent unpaired video translation from simulated surgical 3d data (2021). https://doi.org/10.1109/ICCV48922.2021.00333
Wang, C.Y., Bochkovskiy, A., Liao, H.Y.M.: Yolov7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors (July 2022)
Wood, E., Baltrušaitis, T., Hewitt, C., Dziadzio, S., Cashman, T.J., Shotton, J.: Fake it till you make it: face analysis in the wild using synthetic data alone (2021). https://doi.org/10.1109/ICCV48922.2021.00366
Yonghao Long, Siu Hin Fan, Q.D.Y.W.: Neural rendering for stereo 3d reconstruction of deformable tissues in robotic surgery
Yoon, D., et al.: Colonoscopic image synthesis with generative adversarial network for enhanced detection of sessile serrated lesions using convolutional neural network. Sci. Rep. 12, 261 (2022). https://doi.org/10.1038/s41598-021-04247-y
Yoon, J., et al.: Surgical Scene Segmentation Using Semantic Image Synthesis with Virtual Surgery Environment, pp. 551–561 (2022). https://doi.org/10.1007/978-3-031-16449-1_53
Acknowledgments
This work was funded by EPSRC (EP/V052438/1), and supported by a Microsoft Azure Research Grant, and an Oracle Cloud Computing Grant.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Dowrick, T., Chen, L., Ramalhinho, J., Puyal, J.GB., Clarkson, M.J. (2023). Procedurally Generated Colonoscopy and Laparoscopy Data for Improved Model Training Performance. In: Bhattarai, B., et al. Data Engineering in Medical Imaging. DEMI 2023. Lecture Notes in Computer Science, vol 14314. Springer, Cham. https://doi.org/10.1007/978-3-031-44992-5_7
Download citation
DOI: https://doi.org/10.1007/978-3-031-44992-5_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-44991-8
Online ISBN: 978-3-031-44992-5
eBook Packages: Computer ScienceComputer Science (R0)