Abstract
Free Point Transformer (FPT) has been proposed as a data-driven, non-rigid point set registration approach using deep neural networks. As FPT does not assume constraints based on point vicinity or correspondence, it may be trained simply and in a flexible manner by minimizing an unsupervised loss based on the Chamfer Distance. This makes FPT amenable to real-world medical imaging applications where ground-truth deformations may be infeasible to obtain, or in scenarios where only a varying degree of completeness in the point sets to be aligned is available. To test the limit of the correspondence finding ability of FPT and its dependency on training data sets, this work explores the generalizability of the FPT from well-curated non-medical data sets to medical imaging data sets. First, we train FPT on the ModelNet40 dataset to demonstrate its effectiveness and the superior registration performance of FPT over iterative and learning-based point set registration methods. Second, we demonstrate superior performance in rigid and non-rigid registration and robustness to missing data. Last, we highlight the interesting generalizability of the ModelNet-trained FPT by registering reconstructed freehand ultrasound scans of the spine and generic spine models without additional training, whereby the average difference to the ground truth curvatures is 1.3º, across 13 patients.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Aoki, Y., Goforth, H., Srivatsan, R.A., Lucey, S.: PointNetLK: robust and efficient point cloud registration using PointNet. In: The IEEE Conference on Computer Vision and Pattern Recognition, pp. 7163–7172. IEEE (2019)
Wang, Y., Solomon, J.M.: Deep closest point: learning representations for point cloud registration. In: The IEEE International Conference on Computer Vision, pp. 3523–3532. IEEE (2019)
Kurobe, A., Sekikawa, Y., Ishikawa, K., Saito, H.: CorsNet: 3D point cloud registration by deep neural network. IEEE Robot. Autom. 5(3), 3960–3966 (2020)
Wang, Y., Solomon, J.M.: PRNet: self-supervised learning for partial-to-partial registration. In: Advances in Neural Information Processing Systems 32, pp. 8814–8826 (2019)
Hansen, L., Dittmer, D., Heinrich, M.P.: Learning deformable point set registration with regularized dynamic graph CNNs for large lung motion in COPD patients. In: Zhang, D., Zhou, L., Jie, B., Liu, M. (eds.) GLMI 2019. LNCS, vol. 11849, pp. 53–61. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-35817-4_7
Wang, L., Li, X., Chen, J., Fang, Y.: Coherent point drift networks: unsupervised learning of non-rigid point set registration. arXiv preprint arXiv:1906.03039 (2019)
Wang, L., Chen, J., Li, X., Fang, I.: Non-rigid point set registration networks. arXiv preprint arXiv:1904.01428 (2019)
Maintz, J.B., Viergever, M.A.: A survey of medical image registration. Med. Image Anal. 2(5), 1–36 (1998)
Chui, H., Rangarajan, A.: A new point matching algorithm for non-rigid registration. Comput. Vis. Image Underst. 89(2–3), 114–141 (2003)
Myronenko, A., Song, X.: Point set registration: coherent point drift. IEEE Trans. Pattern Anal. Mach. Intell. 32(12), 2262–2275 (2010)
Jian, B., Vemuri, B.C.: Robust point set registration using Gaussian mixture models. IEEE Trans. Pattern Anal. Mach. Intell. 33(8), 1633–1645 (2010)
Baum, Z.M.C., Hu, Y., Barratt, D.: Multimodality biomedical image registration using free point transformer networks. In: International Conference on Medical Image Computing and Computer Assisted Interventions Advances in Simplifying Medical UltraSound Workshop, pp. 116–125 (2020)
Baum, Z.M.C., Hu, Y., Barratt, D.: Real-time multimodal image registration with partial intraoperative point-set data. Med. Image Anal. 74, 102231 (2021)
Konieczny, M.R., Senyurt, H., Krauspe, R.: Epidemiology of adolescent idiopathic scoliosis. J. Child. Orthop. 7(1), 3–9 (2012). https://doi.org/10.1007/s11832-012-0457-4
Doody, M.M., Lonstein, J.E., Stovall, M., Hacker, D.G., Luckyanov, N., Land, C.E.: Breat cancer mortality after diagnostic radiolgaphy: findings from the U.S. scoliosis cohort study. Spine 25(16), 2052–2063 (2000)
Ungi, T., et al.: Spinal curvature measurement by tracked ultrasound snapshots. Ultrasound Med. Biol. 40(2), 447–454 (2014)
Pinter, C., et al.: Real-time transverse process detection in ultrasound. In: SPIE Medical Imaging: Image-Guided Procedures, Robotic Interventions, and Modeling. SPIE (2018)
Ungi, T., et al.: Automatic spine ultrasound segmentation for scoliosis visualization and measurement. IEEE Trans. Biomed. Eng. 67(11), 3234–3241 (2020)
Qi, C.R., Su, H., Mo, K., Guibas, L.J.: PointNet: deep learning on point sets for 3D classification and segmentation. In: The IEEE Conference on Computer Vision and Pattern Recognition, pp. 652–660. IEEE (2017)
Wu, Z., Song, S., Khosla, A., Yu, F., Zhang, L., Tang, X., Xiao, J.: 3D ShapeNets: a deep representation for volumetric shapes. In: The IEEE Conference on Computer Vision and Pattern Recognition, pp. 1912–1920. IEEE (2015)
Fan, H., Su, H., Guibas, L.J.: A point set generation network for 3D object reconstruction from a single image. In: The IEEE Conference on Computer Vision and Pattern Recognition, pp. 605–613. IEEE (2017)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: The International Conference for Learning Representations. ICLR (2015)
Davis, M.H., Khotanzad, A., Flamig, D.P., Harms, S.E.: Elastic body splines: a physics based approach to coordinate transformation in medical image matching. In: IEEE Symposium on Computer-Based Medical Systems, pp. 81–88 (1995)
Besl, P.J., McKay, N.D.: A method for registration of 3-D shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 239–256 (1992)
Cobb, J.R.: Outline for the study of scoliosis. Am. Acad. Orthop. Surg. Instr. Course Lect. 5, 261–275 (1948)
Malfair, D., et al.: Radiographic evaluation of scoliosis: review. Am. J. Roentgenol. 194(3 Suppl.), S8–S22 (2010)
Acknowledgments
This work is supported by the Wellcome/EPSRC Centre for Interventional and Surgical Sciences (203145Z/16/Z). Z.M.C. Baum is supported by the Natural Sciences and Engineering Research Council of Canada Postgraduate Scholarships-Doctoral Program, and the University College London Overseas and Graduate Research Scholarships.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Baum, Z.M.C., Ungi, T., Schlenger, C., Hu, Y., Barratt, D.C. (2022). Learning Generalized Non-rigid Multimodal Biomedical Image Registration from Generic Point Set Data. In: Aylward, S., Noble, J.A., Hu, Y., Lee, SL., Baum, Z., Min, Z. (eds) Simplifying Medical Ultrasound. ASMUS 2022. Lecture Notes in Computer Science, vol 13565. Springer, Cham. https://doi.org/10.1007/978-3-031-16902-1_14
Download citation
DOI: https://doi.org/10.1007/978-3-031-16902-1_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-16901-4
Online ISBN: 978-3-031-16902-1
eBook Packages: Computer ScienceComputer Science (R0)
