Abstract
Cone-Beam CT (CBCT) and Intraoral Scan (IOS) are dental imaging techniques widely used for surgical planning and simulation. However, the spatial resolution of crowns is low in CBCT, and roots are not visible in IOS. We propose to take the best of both modalities: a seamless fusion of the crown from IOS and the root from CBCT into a single image in a watertight mesh, unlike prior works that compromise the resolution or simply overlay two images. The main challenges are aligning two images (registration) and fusing them (stitching) despite a large gap in the spatial resolution between two modalities. For effective registration, we propose centroid matching followed by coarse- and fine-registration based on the point-to-plane ICP method. Next, stitching of registered images is done to create a watertight mesh, for which we recursively interpolate the boundary points to seamlessly fill the gap between the registered images. Experiments show that the proposed method incurs low registration error, and the fused images are of high quality and accuracy according to the evaluation by experts.
S. Kim and Y. Choi—Equal contribution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Besl, P.J., McKay, N.D.: Method for registration of 3-d shapes. In: Sensor fusion IV: control paradigms and data structures. vol. 1611, pp. 586–606. Spie (1992)
Chung, M., Lee, J., Song, W., Song, Y., Yang, I.H., Lee, J., Shin, Y.G.: Automatic registration between dental cone-beam ct and scanned surface via deep pose regression neural networks and clustered similarities. IEEE Transactions on Medical Imaging 39(12), 3900–3909 (2020)
Cui, Z., Fang, Y., Mei, L., Zhang, B., Yu, B., Liu, J., Jiang, C., Sun, Y., Ma, L., Huang, J., et al.: A fully automatic ai system for tooth and alveolar bone segmentation from cone-beam ct images. Nature communications 13(1), 2096 (2022)
Cui, Z., Li, C., Chen, N., Wei, G., Chen, R., Zhou, Y., Shen, D., Wang, W.: Tsegnet: An efficient and accurate tooth segmentation network on 3d dental model. Medical Image Analysis 69, 101949 (2021)
Cui, Z., Li, C., Wang, W.: Toothnet: automatic tooth instance segmentation and identification from cone beam ct images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 6368–6377 (2019)
Deferm, J., Nijsink, J., Baan, F., Verhamme, L., Meijer, G., Maal, T.: Soft tissue-based registration of intraoral scan with cone beam computed tomography scan. International Journal of Oral and Maxillofacial Surgery 51(2), 263–268 (2022)
Dijkstra, E.W.: A note on two problems in connexion with graphs. In: Edsger Wybe Dijkstra: His Life, Work, and Legacy, pp. 287–290 (2022)
Ender, A., Zimmermann, M., Mehl, A.: Accuracy of complete-and partial-arch impressions of actual intraoral scanning systems in vitro. International journal of computerized dentistry 22(1), 11–19 (2019)
Ezhov, M., Gusarev, M., Golitsyna, M., Yates, J.M., Kushnerev, E., Tamimi, D., Aksoy, S., Shumilov, E., Sanders, A., Orhan, K.: Clinically applicable artificial intelligence system for dental diagnosis with cbct. Scientific reports 11(1), 15006 (2021)
Gateno, J., Xia, J., Teichgraeber, J.F., Rosen, A.: A new technique for the creation of a computerized composite skull model. Journal of oral and maxillofacial surgery 61(2), 222–227 (2003)
Hao, J., Liu, J., Li, J., Pan, W., Chen, R., Xiong, H., Sun, K., Lin, H., Liu, W., Ding, W., et al.: Ai-enabled automatic multimodal fusion of cone-beam ct and intraoral scans for intelligent 3d tooth-bone reconstruction and clinical applications. arXiv preprint arXiv:2203.05784 (2022)
Hung, K., Yeung, A.W.K., Tanaka, R., Bornstein, M.M.: Current applications, opportunities, and limitations of ai for 3d imaging in dental research and practice. International Journal of Environmental Research and Public Health 17(12), 4424 (2020)
Hyun, C.M., Bayaraa, T., Yun, H.S., Jang, T.J., Park, H.S., Seo, J.K.: Deep learning method for reducing metal artifacts in dental cone-beam ct using supplementary information from intra-oral scan. Physics in Medicine & Biology 67(17), 175007 (2022)
Jang, T.J., Yun, H.S., Hyun, C.M., Kim, J.E., Lee, S.H., Seo, J.K.: Fully automatic integration of dental cbct images and full-arch intraoral impressions with stitching error correction via individual tooth segmentation and identification. arXiv preprint arXiv:2112.01784 (2021)
Kim, S., Song, I.S., Baek, S.J.: Automatic segmentation of internal tooth structure from cbct images using hierarchical deep learning. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 703–713. Springer (2023)
Liang, Y., Qiu, L., Lu, T., Fang, Z., Tu, D., Yang, J., Shao, Y., Wang, K., Chen, X., He, L.: Oralviewer: 3d demonstration of dental surgeries for patient education with oral cavity reconstruction from a 2d panoramic x-ray. In: 26th International Conference on Intelligent User Interfaces. pp. 553–563 (2021)
Liu, J., Hao, J., Lin, H., Pan, W., Yang, J., Feng, Y., Wang, G., Li, J., Jin, Z., Zhao, Z., et al.: Deep learning-enabled 3d multimodal fusion of cone-beam ct and intraoral mesh scans for clinically applicable tooth-bone reconstruction. Patterns 4(9) (2023)
Low, K.L.: Linear least-squares optimization for point-to-plane icp surface registration. Chapel Hill, University of North Carolina 4(10), 1–3 (2004)
Pomerleau, F., Colas, F., Siegwart, R., Magnenat, S.: Comparing icp variants on real-world data sets: Open-source library and experimental protocol. Autonomous robots 34, 133–148 (2013)
Qian, J., Lu, S., Gao, Y., Tao, Y., Lin, J., Lin, H.: An automatic tooth reconstruction method based on multimodal data. Journal of Visualization 24, 205–221 (2021)
Singhal, I., Kaur, G., Neefs, D., Pathak, A.: A literature review of the future of oral medicine and radiology, oral pathology, and oral surgery in the hands of technology. Cureus 15(9) (2023)
Sukotjo, C., Schreiber, S., Li, J., Zhang, M., Chia-Chun Yuan, J., Santoso, M.: Development and student perception of virtual reality for implant surgery. Education Sciences 11(4), 176 (2021)
Swennen, G., Barth, E.L., Eulzer, C., Schutyser, F.: The use of a new 3d splint and double ct scan procedure to obtain an accurate anatomic virtual augmented model of the skull. International journal of oral and maxillofacial surgery 36(2), 146–152 (2007)
Swennen, G., Mommaerts, M., Abeloos, J., De Clercq, C., Lamoral, P., Neyt, N., Casselman, J., Schutyser, F.: A cone-beam ct based technique to augment the 3d virtual skull model with a detailed dental surface. International journal of oral and maxillofacial surgery 38(1), 48–57 (2009)
Wang, H., Minnema, J., Batenburg, K.J., Forouzanfar, T., Hu, F.J., Wu, G.: Multiclass cbct image segmentation for orthodontics with deep learning. Journal of dental research 100(9), 943–949 (2021)
Xia, J.J., Gateno, J., Teichgraeber, J.F.: New clinical protocol to evaluate craniomaxillofacial deformity and plan surgical correction. Journal of Oral and Maxillofacial Surgery 67(10), 2093–2106 (2009)
Xiong, H., Li, K., Tan, K., Feng, Y., Zhou, J.T., Hao, J., Ying, H., Wu, J., Liu, Z.: Tsegformer: 3d tooth segmentation in intraoral scans with geometry guided transformer. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 421–432. Springer (2023)
Zhao, Y., Zhang, L., Liu, Y., Meng, D., Cui, Z., Gao, C., Gao, X., Lian, C., Shen, D.: Two-stream graph convolutional network for intra-oral scanner image segmentation. IEEE Transactions on Medical Imaging 41(4), 826–835 (2021)
Zhou, X., Gan, Y., Xiong, J., Zhang, D., Zhao, Q., Xia, Z.: A method for tooth model reconstruction based on integration of multimodal images. Journal of healthcare engineering 2018 (2018)
Acknowledgements
This research was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 2022R1A5A1027646), the MSIT (Ministry of Science and ICT), Korea, under the ICT Creative Consilience program (IITP-2020-0-01819) supervised by the IITP (Institute for Information & communications Technology Planning & Evaluation), and the Korea Medical Device Development Fund grant funded by the Korea government (the Ministry of Science and ICT, the Ministry of Trade, Industry and Energy, the Ministry of Health & Welfare, the Ministry of Food and Drug Safety) (Project Number: 1711195279, RS-2021-KD000009).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Ethics declarations
Disclosure of Interests
The authors have no competing interests to declare that are relevant to the content of this article.
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Kim, S. et al. (2024). Best of Both Modalities: Fusing CBCT and Intraoral Scan Data Into a Single Tooth Image. In: Linguraru, M.G., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2024. MICCAI 2024. Lecture Notes in Computer Science, vol 15002. Springer, Cham. https://doi.org/10.1007/978-3-031-72069-7_52
Download citation
DOI: https://doi.org/10.1007/978-3-031-72069-7_52
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-72068-0
Online ISBN: 978-3-031-72069-7
eBook Packages: Computer ScienceComputer Science (R0)
