Wenchong He
ABSTRACT
Over the last decade, deep learning research has achieved tremendous success in computer vision and natural language processing. The current widely successful deep learning models are largely based on convolution and pooling operations on a Euclidean plane with a regular grid (e.g., image and video data) and thus cannot be directly applied to the non-Euclidean surface. Geometric deep learning aims to fill the gap by generalizing deep learning models from a 2D Euclidean plane to a 3D geometric surface. The problem has important applications in human-computer interaction, biochemistry, and mechanical engineering, but is uniquely challenging due to the lack of a regular grid framework and the difficulties in learning geometric features on a non-Euclidean manifold. Existing works focus on generalizing deep learning models from 2D image to graphs (e.g., graph neural networks) or 3D mesh surfaces but without fully learning geometric features from a differential geometry perspective. In contrast, this paper proposes a novel geometric deep learning model called CurvaNet that integrates differential geometry with graph neural networks. The key idea is to learn direction sensitive 3D shape features through directional curvature filters. We design a U-Net like architecture with downsampling and upsampling paths based on mesh pooling and unpooling operations. Evaluation on real-world datasets shows that the proposed model outperforms several baseline methods in classification accuracy.
Supplemental Material
- Alexander Agathos, Ioannis Pratikakis, Stavros Perantonis, Nikolaos Sapidis, and Philip Azariadis. 2007. 3D mesh segmentation methodologies for CAD applications. Computer-Aided Design and Applications, Vol. 4, 6 (2007), 827--841.Google Scholar
Cross Ref
- Eman Ahmed, Alexandre Saint, Abd El Rahman Shabayek, Kseniya Cherenkova, Rig Das, Gleb Gusev, Djamila Aouada, and Bjorn Ottersten. 2018. A survey on Deep Learning Advances on Different 3D Data Representations. arXiv preprint arXiv:1808.01462 (2018).Google Scholar
- James Atwood and Don Towsley. 2016. Diffusion-convolutional neural networks. In Advances in neural information processing systems. 1993--2001.Google Scholar
- Davide Boscaini, Jonathan Masci, Emanuele Rodolà, and Michael Bronstein. 2016a. Learning shape correspondence with anisotropic convolutional neural networks. In Advances in Neural Information Processing Systems. 3189--3197.Google Scholar
- Davide Boscaini, Jonathan Masci, Emanuele Rodolà, Michael M Bronstein, and Daniel Cremers. 2016b. Anisotropic diffusion descriptors. In Computer Graphics Forum, Vol. 35. Wiley Online Library, 431--441.Google Scholar
- Michael M Bronstein, Joan Bruna, Yann LeCun, Arthur Szlam, and Pierre Vandergheynst. 2017. Geometric deep learning: going beyond euclidean data. IEEE Signal Processing Magazine, Vol. 34, 4 (2017), 18--42.Google ScholarCross Ref
- Xiaobai Chen, Aleksey Golovinskiy, and Thomas Funkhouser. 2009. A benchmark for 3D mesh segmentation. Acm transactions on graphics (tog), Vol. 28, 3 (2009), 1--12.Google Scholar
- Wei-Lin Chiang, Xuanqing Liu, Si Si, Yang Li, Samy Bengio, and Cho-Jui Hsieh. 2019. Cluster-gcn: An efficient algorithm for training deep and large graph convolutional networks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 257--266.Google ScholarDigital Library
- Yan Cui, Sebastian Schuon, Derek Chan, Sebastian Thrun, and Christian Theobalt. 2010. 3D shape scanning with a time-of-flight camera. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE, 1173--1180.Google ScholarCross Ref
- William HE Day and Herbert Edelsbrunner. 1984. Efficient algorithms for agglomerative hierarchical clustering methods. Journal of classification, Vol. 1, 1 (1984), 7--24.Google ScholarCross Ref
- Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. 2016. Convolutional neural networks on graphs with fast localized spectral filtering. In Advances in neural information processing systems. 3844--3852.Google Scholar
- Inderjit S Dhillon, Yuqiang Guan, and Brian Kulis. 2007. Weighted graph cuts without eigenvectors a multilevel approach. IEEE transactions on pattern analysis and machine intelligence, Vol. 29, 11 (2007), 1944--1957.Google ScholarDigital Library
- Felix Endres, Jürgen Hess, Jürgen Sturm, Daniel Cremers, and Wolfram Burgard. 2013. 3-D mapping with an RGB-D camera. IEEE transactions on robotics, Vol. 30, 1 (2013), 177--187.Google Scholar
- Yi Fang, Jin Xie, Guoxian Dai, Meng Wang, Fan Zhu, Tiantian Xu, and Edward Wong. 2015. 3d deep shape descriptor. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2319--2328.Google ScholarCross Ref
- Yutong Feng, Yifan Feng, Haoxuan You, Xibin Zhao, and Yue Gao. 2019. MeshNet: mesh neural network for 3D shape representation. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 8279--8286.Google ScholarDigital Library
- Matthias Fey, Jan Eric Lenssen, Frank Weichert, and Heinrich Mü ller. 2018. SplineCNN: Fast geometric deep learning with continuous B-spline kernels. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 869--877.Google ScholarCross Ref
- P Gainza, F Sverrisson, F Monti, E Rodolà, D Boscaini, MM Bronstein, and BE Correia. 2020. Deciphering interaction fingerprints from protein molecular surfaces using geometric deep learning. Nature Methods, Vol. 17, 2 (2020), 184--192.Google ScholarCross Ref
- Hongyang Gao and Shuiwang Ji. 2019. Graph U-Nets. arXiv preprint arXiv:1905.05178 (2019).Google Scholar
- Hongyang Gao, Zhengyang Wang, and Shuiwang Ji. 2018. Large-scale learnable graph convolutional networks. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1416--1424.Google ScholarDigital Library
- Michael Garland and Paul S Heckbert. 1997. Surface simplification using quadric error metrics. In Proceedings of the 24th annual conference on Computer graphics and interactive techniques. 209--216.Google ScholarDigital Library
- Kan Guo, Dongqing Zou, and Xiaowu Chen. 2015. 3d mesh labeling via deep convolutional neural networks. ACM Transactions on Graphics (TOG), Vol. 35, 1 (2015), 1--12.Google ScholarDigital Library
- Rana Hanocka, Amir Hertz, Noa Fish, Raja Giryes, Shachar Fleishman, and Daniel Cohen-Or. 2019. MeshCNN: a network with an edge. ACM Transactions on Graphics (TOG), Vol. 38, 4 (2019), 90.Google ScholarDigital Library
- Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016).Google Scholar
- Sebastian Koch, Albert Matveev, Zhongshi Jiang, Francis Williams, Alexey Artemov, Evgeny Burnaev, Marc Alexa, Denis Zorin, and Daniele Panozzo. 2019. ABC: A big CAD model dataset for geometric deep learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 9601--9611.Google ScholarCross Ref
- Yao Ma, Suhang Wang, Charu C Aggarwal, and Jiliang Tang. 2019. Graph convolutional networks with eigenpooling. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 723--731.Google ScholarDigital Library
- Jean-Luc Mari, Sophie Viseur, Sylvain Bouley, Martin-Pierre Schmidt, Jennifer Muscato, Florian Beguet, Sarah Bali, and Laurent Jorda. 2018. Robust detection of circular shapes on 3D meshes based on discrete curvatures-Application to impact craters recognition.Google Scholar
- Jonathan Masci, Davide Boscaini, Michael Bronstein, and Pierre Vandergheynst. 2015. Geodesic convolutional neural networks on riemannian manifolds. In Proceedings of the IEEE international conference on computer vision workshops. 37--45.Google ScholarDigital Library
- Takashi Matsuyama, Xiaojun Wu, Takeshi Takai, and Shohei Nobuhara. 2004. Real-time 3D shape reconstruction, dynamic 3D mesh deformation, and high fidelity visualization for 3D video. Computer Vision and Image Understanding, Vol. 96, 3 (2004), 393--434.Google ScholarDigital Library
- Federico Monti, Davide Boscaini, Jonathan Masci, Emanuele Rodola, Jan Svoboda, and Michael M Bronstein. 2017. Geometric deep learning on graphs and manifolds using mixture model cnns. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5115--5124.Google ScholarCross Ref
- Vladimir I Pavlovic, Rajeev Sharma, and Thomas S. Huang. 1997. Visual interpretation of hand gestures for human-computer interaction: A review. IEEE Transactions on pattern analysis and machine intelligence, Vol. 19, 7 (1997), 677--695.Google ScholarDigital Library
- CMPPC Rocchini, Paulo Cignoni, Claudio Montani, Paolo Pingi, and Roberto Scopigno. 2001. A low cost 3D scanner based on structured light. In Computer Graphics Forum, Vol. 20. Wiley Online Library, 299--308.Google Scholar
- Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention. Springer, 234--241.Google ScholarCross Ref
- Reihaneh Rostami, Fereshteh S Bashiri, Behrouz Rostami, and Zeyun Yu. 2019. A Survey on Data-Driven 3D Shape Descriptors. In Computer Graphics Forum, Vol. 38. Wiley Online Library, 356--393.Google Scholar
- Gérard Subsol, Bertrand Mafart, Alain Silvestre, Marie-Antoinette de Lumley, and H Delingette. 2002. 3d image processing for the study of the evolution of the shape of the human skull: presentation of the tools and preliminary results. Three-Dimensional Imaging in Paleoanthropology and Prehistoric Archaeology, Vol. 1049 (2002), 37--45.Google Scholar
- Hiromi T Tanaka, Masaki Ikeda, and Hisako Chiaki. 1998. Curvature-based face surface recognition using spherical correlation. principal directions for curved object recognition. In Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition. IEEE, 372--377.Google ScholarCross Ref
- Petar Velivc ković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks. arXiv preprint arXiv:1710.10903 (2017).Google Scholar
- Nitika Verma, Edmond Boyer, and Jakob Verbeek. 2018. Feastnet: Feature-steered graph convolutions for 3d shape analysis. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2598--2606.Google ScholarCross Ref
- Peng-Shuai Wang, Chun-Yu Sun, Yang Liu, and Xin Tong. 2018. Adaptive O-CNN: A patch-based deep representation of 3D shapes. ACM Transactions on Graphics (TOG), Vol. 37, 6 (2018), 1--11.Google ScholarDigital Library
- Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and Philip S Yu. 2019. A comprehensive survey on graph neural networks. arXiv preprint arXiv:1901.00596 (2019).Google Scholar
- Zhitao Ying, Jiaxuan You, Christopher Morris, Xiang Ren, Will Hamilton, and Jure Leskovec. 2018. Hierarchical graph representation learning with differentiable pooling. In Advances in Neural Information Processing Systems. 4800--4810.Google Scholar
- Ziwei Zhang, Peng Cui, and Wenwu Zhu. 2018. Deep learning on graphs: A survey. arXiv preprint arXiv:1812.04202 (2018).Google Scholar
- Jie Zhou, Ganqu Cui, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, and Maosong Sun. 2018. Graph neural networks: A review of methods and applications. arXiv preprint arXiv:1812.08434 (2018).Google Scholar
Index Terms
- CurvaNet: Geometric Deep Learning based on Directional Curvature for 3D Shape Analysis
Recommendations
RingNet: Geometric Deep Representation Learning for 3D Multi-domain Protein Shape Retrieval
Computational Collective IntelligenceAbstractMany applications use three-dimensional polygon meshes as geometric representation to achieve central tasks, such as 3D object retrieval and classification. However, implementing a deep learning approach dedicated to 3D meshes is a bit hard due to ...
Lines of curvature and umbilical points for implicit surfaces
This paper develops a method to analyze and compute the lines of curvature and their differential geometry defined on implicit surfaces. With our technique, we can explicitly obtain the analytic formulae of the associated geometric attributes of an ...
Concentrated curvature for mean curvature estimation in triangulated surfaces
CTIC'12: Proceedings of the 4th international conference on Computational Topology in Image ContextWe present a mathematical result that allows computing the discrete mean curvature of a polygonal surface from the so-called concentrated curvature generally used for Gaussian curvature estimation. Our result adds important value to concentrated ...
Comments