Skip to main content

Advertisement

SpringerLink
Log in

Heterogeneous data fusion and loss function design for tooth point cloud segmentation

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Tooth point cloud segmentation plays an important role in the digital dentistry, and has received much attention in the past decade. Recently, methods based on the graph neural network have made significant progress. However, the development has been hindered by two challenges: (1) the heterogeneous geometry data are analyzed separately or combined linearly which leads to a semantic gap in different streams; (2) there is mis-alignment between the loss function and evaluation metrics in the segmentation task. In this paper, a novel interacted graph network is presented that combines cues from heterogeneous geometry data by extending the graph attention architecture to propagate information among the different graphs. Moreover, in this paper, an approach is designed to search the segmentation loss function based on the computation graphs according to the evaluation metrics, and the evolution algorithm is revised to avoid potential loss and equivalent loss functions. Our method and other methods use the Shining3D Tooth Segmentation dataset, with experimental results compared in terms of accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Berman M, Triki AR, Blaschko, MB (2018) The lovász-softmax loss: a tractable surrogate for the optimization of the intersection-over-union measure in neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 4413–4421

  2. Caliva F, Iriondo C, Martinez AM et al. (2019) Distance map loss penalty term for semantic segmentation. In: Proceedings of Medical Imaging with Deep Learning, 2413–2422

  3. Cui Y, Liu X, Liu H et al (2021) Geometric attentional dynamic graph convolutional neural networks for point cloud analysis. Neurocomputing 432:300–310

    Article  Google Scholar 

  4. Cui Z, Li C, Chen N et al (2021) Tsegnet: an efficient and accurate tooth segmentation network on 3d dental model. Med Image Anal 69:101949

    Article  Google Scholar 

  5. Dong X, Yang Y (2019) One-shot neural architecture search via self-evaluated template network. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 3681–3690

  6. Elsken T, Metzen, JH, Hutter F (2019) Efficient multi-objective neural architecture search via lamarckian evolution. In: International Conference on Learning Representations, 551–562

  7. Elsken T, Metzen JH, Hutter F et al (2019) Neural architecture search: a survey. J Mach Learn Res 20(55):1–21

    MathSciNet  MATH  Google Scholar 

  8. Hao J, Liao W, Zhang Y, Peng J, Zhao Z, Chen Z, Zhou B, Feng Y, Fang B, Liu Z et al (2021) Toward clinically applicable 3-dimensional tooth segmentation via deep learning. J Dental Res 101(3):304–311

    Article  Google Scholar 

  9. He J, Wang S, Li J (2020) Tooth point cloud segmentation of dental model based on region growing. In: Proceedings of the 2nd International Conference on Artificial Intelligence and Advanced Manufacture, 489–492

  10. He X, Zhao K, Chu X (2021) Automl: a survey of the state-of-the-art. Knowledge-Based Syst 212:106622

    Article  Google Scholar 

  11. Kandasamy K, Neiswanger W, Schneider J et al (2018) Neural architecture search with bayesian optimisation and optimal transport. In: Advances in Neural Information Processing Systems, 1245–1253

  12. Kim T, Cho Y, Kim D et al (2020) Tooth segmentation of 3d scan data using generative adversarial networks. Appl Sci 10(2):490

    Article  Google Scholar 

  13. Li C, Yuan X, Lin C et al (2019) Am-lfs: Automl for loss function search. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 8410–8419

  14. Li H, Fu T, Dai J et al (2021) Autoloss-zero: Searching loss functions from scratch for generic tasks. arXiv preprint arXiv:2103.14026

  15. Li H, Sun Z, Wu Y et al (2021) Semi-supervised point cloud segmentation using self-training with label confidence prediction. Neurocomputing 437:227–237

    Article  Google Scholar 

  16. Li H, Tao C, Zhu X et al (2020) Auto seg-loss: searching metric surrogates for semantic segmentation. In: International Conference on Learning Representations, 2410–2419

  17. Li Y, Bu R, Sun M et al (2018) Pointcnn: Convolution on x-transformed points. In: Advances in Neural Information Processing Systems, 820–830

  18. Lian C, Wang L, Wu TH et al (2020) Deep multi-scale mesh feature learning for automated labeling of raw dental surfaces from 3d intraoral scanners. IEEE Trans Med Imag 39(7):2440–2450

    Article  Google Scholar 

  19. Liu C, Zoph B, Neumann M et al (2018) Progressive neural architecture search. In: Proceedings of the European Conference on Computer Vision, 19–34

  20. Liu H, Simonyan K, Yang Y (2019) Darts: Differentiable architecture search. In: International Conference on Learning Representations, 651–662

  21. Liu P, Zhang G, Wang B et al (2021) Loss function discovery for object detection via convergence-simulation driven search. In: International Conference on Learning Representations, 731–732

  22. Ma Q, Wei G, Zhou Y et al (2020) Srf-net: Spatial relationship feature network for tooth point cloud classification. Computer Graphics Forum 39(7):267–277

    Article  Google Scholar 

  23. Milletari F, Navab N, Ahmadi SA (2016) V-net: Fully convolutional neural networks for volumetric medical image segmentation. In: International Conference on 3D Vision, 565–571

  24. Paszke A, Gross S, Massa F et al (2019) Pytorch: An imperative style, high-performance deep learning library. In: Advances in Neural Information Processing Systems, 8026–8037

  25. Qin X, Zhang Z, Huang C et al (2019) Basnet: Boundary-aware salient object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 7479–7489

  26. Real E, Aggarwal A, Huang Y et al (2019) Regularized evolution for image classifier architecture search. In: Proceedings of the AAAI Conference on Artificial Intelligence, 4780–4789

  27. Real E, Moore S, Selle A et al (2017) Large-scale evolution of image classifiers. In: International Conference on Machine Learning, 2902–2911

  28. Ronneberger, O, Fischer, P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, 234–241

  29. Sun D, Pei Y, Li P et al (2020) Automatic tooth segmentation and dense correspondence of 3d dental model. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, 703–712

  30. Sun D, Pei Y, Song G et al (2020) Tooth segmentation and labeling from digital dental casts. In: IEEE International Symposium on Biomedical Imaging, 669–673

  31. Tian S, Dai N, Zhang B et al (2019) Automatic classification and segmentation of teeth on 3d dental model using hierarchical deep learning networks. IEEE Access 7:84817–84828

    Article  Google Scholar 

  32. Tian Y, Chen T, Cheng G et al (2021) Global context assisted structure-aware vehicle retrieval. IEEE Trans Intell Trans Syst 21(10):1–10

    Google Scholar 

  33. Tian Y, Cheng G, Gelernter J et al (2020) Joint temporal context exploitation and active learning for video segmentation. Pattern Recogn 100:107158

    Article  Google Scholar 

  34. Tian Y, Gelernter J, Wang X et al (2019) Traffic sign detection using a multi-scale recurrent attention network. IEEE Trans Intell Trans Syst 20(12):4466–4475

    Article  Google Scholar 

  35. Tian Y, Wang X, Wu J et al (2019) Multi-scale hierarchical residual network for dense captioning. J Artif Intell Res 64:181–196

    Article  MathSciNet  Google Scholar 

  36. Tian Y, Zhang Y, We-Gang C et al (2021) 3d tooth instance segmentation learning objectness and affinity in point cloud. ACM Trans Multimedia Comput Commun Appl 33:4780–4789

    Google Scholar 

  37. Tian Y, Zhang Y, Zhou D et al (2020) Triple attention network for video segmentation. Neurocomputing 417:202–211

    Article  Google Scholar 

  38. Veličković P, Cucurull, G, Casanova A et al (2018) Graph attention networks. In: The International Conference on Learning Representations, 1780–1789

  39. Verma N, Boyer E, Verbeek J (2018) Feastnet: Feature-steered graph convolutions for 3d shape analysis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2598–2606

  40. Wang X, Wang S, Chi C et al (2020) Loss function search for face recognition. In: International Conference on Machine Learning, 10029–10038

  41. Xie Z, Chen J, Peng B (2020) Point clouds learning with attention-based graph convolution networks. Neurocomputing 402:245–255

    Article  Google Scholar 

  42. Xu X, Liu C, Zheng Y (2018) 3d tooth segmentation and labeling using deep convolutional neural networks. IEEE Trans Vis Computer Graph 25(7):2336–2348

    Article  Google Scholar 

  43. Zanjani FG, Moin DA, Verheij B et al. (2019) Deep learning approach to semantic segmentation in 3d point cloud intra-oral scans of teeth. In: International Conference on Medical Imaging with Deep Learning, 557–571

  44. Zhang C, Song D, Huang C et al (2019) Heterogeneous graph neural network. In: Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 793–803

  45. Zhang J, Li C, Song Q et al (2020) Automatic 3d tooth segmentation using convolutional neural networks in harmonic parameter space. Graphical Models 109:101071

    Article  Google Scholar 

  46. Zhang, L, Zhao Y, Meng D et al (2021) Tsgcnet: Discriminative geometric feature learning with two-stream graph convolutional network for 3d dental model segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 6699–6708

  47. Zoph B, Le QV (2017) Neural architecture search with reinforcement learning. In: International Conference on Learning Representations, 751–762

  48. Zoph B, Vasudevan V, Shlens J et al (2018) Learning transferable architectures for scalable image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recogn, 8697–8710

Download references

Acknowledgements

The authors would like to thank AJE (www.aje.com) for its linguistic assistance during the preparation of this manuscript.

Author information

Authors and Affiliations

  1. School of Computer Science and Information Engineering, Zhejiang Gongshang University, Hangzhou, 310018, China

    Dongsheng Liu, Yan Tian, Yujie Zhang & Xun Wang

  2. Shining3D Tech. Co.,Ltd., Hangzhou, 310018, China

    Yan Tian & Yujie Zhang

  3. Information Science Department, Rutgers University, New Brunswick, 08901, USA

    Judith Gelernter

Authors
  1. Dongsheng Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yan Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yujie Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Judith Gelernter
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xun Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yan Tian.

Ethics declarations

Conflict of interest

All authors declare that they have no conflicts of interest regarding the publication of this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was supported in part by the National Natural Science Foundation of China under Grant 61972351 and 62111530300, in part by the Public Welfare Technology Research Project of Zhejiang Province under Grant LGF19G010002 and LGF20G010002, and in part by the Science and Technology Program of Zhejiang Province (Key Research and Development Plan) under Grant 2022C01005.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, D., Tian, Y., Zhang, Y. et al. Heterogeneous data fusion and loss function design for tooth point cloud segmentation. Neural Comput & Applic 34, 17371–17380 (2022). https://doi.org/10.1007/s00521-022-07379-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-022-07379-y

Keywords

Search

Navigation