skip to main content
10.1145/3488560.3498402acmconferencesArticle/Chapter ViewAbstractPublication PageswsdmConference Proceedingsconference-collections
research-article

Geometric Inductive Matrix Completion: A Hyperbolic Approach with Unified Message Passing

Published: 15 February 2022 Publication History

Abstract

Collaborative filtering is a central task in a broad range of recommender systems. As traditional methods train latent variables for user/item individuals under a transductive setting, it requires re-training for out-of-sample inferences. Inductive matrix completion (IMC) solves this problem by learning transformation functions upon engineered features, but it sacrifices model expressiveness and highly depends on feature qualities. In this paper, we propose Geometric Inductive Matrix Completion (GIMC) by introducing hyperbolic geometry and a unified message passing scheme into this generic task. The proposed method is the earliest attempt utilizing capacious hyperbolic space to enhance the capacity of IMC. It is the first work defining continuous explicit feedback prediction within non-Euclidean space by introducing hyperbolic regression for vertex interactions. This is also the first to provide comprehensive evidence that edge semantics can significantly improve recommendations, which is ignored by previous works. The proposed method outperforms the state-of-the-art algorithms with less than 1% parameters compared to its transductive counterparts. Extensive analysis and ablation studies are conducted to reveal the design considerations and practicability for a positive impact to the research community.

Supplementary Material

MP4 File (wsdmfp211.mp4)
We present Geometric Inductive Matrix Completion (GIMC). GIMC bridges the generic sparse graph structure with hyperbolic space by introducing identity-specific vertex embeddings and hyperbolic regression for interactions. It also empirically validated the effectiveness of edge embeddings for explicit feedback. We corroborate our findings with comprehensive experiments and theoretical analysis. We hope this work reveals intuitive and novel insights for future research.

References

[1]
Charu C Aggarwal et al. 2016. Recommender Systems .Springer.
[2]
Ting Bai, Ji-Rong Wen, Jun Zhang, and Wayne Xin Zhao. 2017. A neural collaborative filtering model with interaction-based neighborhood. In CIKM .
[3]
Gary Bécigneul, Octavian-Eugen Ganea, Benson Chen, Regina Barzilay, and Tommi Jaakkola. 2020. Optimal transport graph neural networks. In ICLR .
[4]
Xuan Bi, Annie Qu, Junhui Wang, and Xiaotong Shen. 2017. A group-specific recommender system. J. Amer. Statist. Assoc. (2017).
[5]
James W Cannon, William J Floyd, Richard Kenyon, Walter R Parry, et al. 1997. Hyperbolic geometry. Flavors of geometry, Vol. 31 (1997).
[6]
Oscar Celma and Paul Lamere. 2011. Music recommendation and discovery revisited. In RecSys .
[7]
Ines Chami, Adva Wolf, Da-Cheng Juan, Frederic Sala, Sujith Ravi, and Christopher Ré. 2020. Low-Dimensional Hyperbolic Knowledge Graph Embeddings. In ACL .
[8]
Ines Chami, Rex Ying, Christopher Ré, and Jure Leskovec. 2019. Hyperbolic graph convolutional neural networks. In NeurIPS .
[9]
Hongxu Chen, Hongzhi Yin, Tong Chen, Weiqing Wang, Xue Li, and Xia Hu. 2020. Social boosted recommendation with folded bipartite network embedding. IEEE Transactions on Knowledge and Data Engineering (2020).
[10]
Hongxu Chen, Hongzhi Yin, Weiqing Wang, Hao Wang, Quoc Viet Hung Nguyen, and Xue Li. 2018. PME: projected metric embedding on heterogeneous networks for link prediction. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1177--1186.
[11]
Kai-Yang Chiang, Cho-Jui Hsieh, and Inderjit S Dhillon. 2015. Matrix Completion with Noisy Side Information. In NeurIPS .
[12]
Aaron Clauset, Cosma Rohilla Shalizi, and Mark EJ Newman. 2009. Power-law distributions in empirical data. SIAM Rev. (2009).
[13]
Leonhard Euler. 1741. Solutio problematis ad geometriam situs pertinentis. Commentarii academiae scientiarum Petropolitanae ( 1741).
[14]
Shanshan Feng, Lucas Vinh Tran, Gao Cong, Lisi Chen, Jing Li, and Fan Li. 2020. Hme: A hyperbolic metric embedding approach for next-poi recommendation. In SIGIR .
[15]
Octavian-Eugen Ganea, Gary Bécigneul, and Thomas Hofmann. 2018. Hyperbolic neural networks. In NeurIPS .
[16]
Caglar Gulcehre, Misha Denil, Mateusz Malinowski, Ali Razavi, Razvan Pascanu, Karl Moritz Hermann, Peter Battaglia, Victor Bapst, David Raposo, Adam Santoro, et al. 2018. Hyperbolic Attention Networks. In ICLR .
[17]
William L Hamilton, Rex Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In NeurIPS .
[18]
Jason Hartford, Devon Graham, Kevin Leyton-Brown, and Siamak Ravanbakhsh. 2018. Deep models of interactions across sets. In ICML .
[19]
Li He, Hongxu Chen, Dingxian Wang, Shoaib Jameel, Philip Yu, and Guandong Xu. 2021. Click-Through Rate Prediction with Multi-Modal Hypergraphs. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management . 690--699.
[20]
Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. 2020. Lightgcn: Simplifying and powering graph convolution network for recommendation. In SIGIR .
[21]
Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural collaborative filtering. In WWW .
[22]
Cheng Hsu and Cheng-Te Li. 2021. RetaGNN: Relational Temporal Attentive Graph Neural Networks for Holistic Sequential Recommendation. In WWW .
[23]
Prateek Jain and Inderjit S Dhillon. 2013. Provable inductive matrix completion. arXiv preprint arXiv:1306.0626 (2013).
[24]
Prateek Jain, Praneeth Netrapalli, and Sujay Sanghavi. 2013. Low-rank matrix completion using alternating minimization. In STOC .
[25]
Valentin Khrulkov, Leyla Mirvakhabova, Evgeniya Ustinova, Ivan Oseledets, and Victor Lempitsky. 2020. Hyperbolic image embeddings. In CVPR .
[26]
Diederik P Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In ICLR .
[27]
Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix factorization techniques for recommender systems. Computer, Vol. 42, 8 (2009), 30--37.
[28]
Dmitri Krioukov, Fragkiskos Papadopoulos, Maksim Kitsak, Amin Vahdat, and Marián Boguná. 2010. Hyperbolic geometry of complex networks. Physical Review E, Vol. 82, 3 (2010), 036106.
[29]
Dmitri Krioukov, Fragkiskos Papadopoulos, Amin Vahdat, and Marián Boguná. 2009. Curvature and temperature of complex networks. Physical Review E, Vol. 80, 3 (2009), 035101.
[30]
Anchen Li, Bo Yang, Hongxu Chen, and Guandong Xu. 2021. Hyperbolic Neural Collaborative Recommender. arXiv preprint arXiv:2104.07414 (2021).
[31]
Qi Liu, Maximilian Nickel, and Douwe Kiela. 2019. Hyperbolic graph neural networks. In NeurIPS .
[32]
Shaoteng Liu, Jingjing Chen, Liangming Pan, Chong-Wah Ngo, Tat-Seng Chua, and Yu-Gang Jiang. 2020. Hyperbolic visual embedding learning for zero-shot recognition. In CVPR .
[33]
Leyla Mirvakhabova, Evgeny Frolov, Valentin Khrulkov, Ivan Oseledets, and Alexander Tuzhilin. 2020. Performance of hyperbolic geometry models on top-N recommendation tasks. In RecSys .
[34]
Federico Monti, Michael M Bronstein, and Xavier Bresson. 2017. Geometric matrix completion with recurrent multi-graph neural networks. In NeurIPS .
[35]
Maximillian Nickel and Douwe Kiela. 2017. Poincaré embeddings for learning hierarchical representations. In NeurIPS .
[36]
Maximillian Nickel and Douwe Kiela. 2018. Learning continuous hierarchies in the lorentz model of hyperbolic geometry. In ICML .
[37]
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019. Pytorch: An imperative style, high-performance deep learning library. In NeurIPS .
[38]
Nikhil Rao, Hsiang-Fu Yu, Pradeep Ravikumar, and Inderjit S Dhillon. 2015. Collaborative Filtering with Graph Information: Consistency and Scalable Methods. In NeurIPS .
[39]
Steffen Rendle. 2010. Factorization machines. In ICDM. IEEE.
[40]
Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling. 2018. Modeling relational data with graph convolutional networks. In ESWC .
[41]
Ryohei Shimizu, Yusuke Mukuta, and Tatsuya Harada. 2021. Hyperbolic neural networks
[42]
. In ICLR .
[43]
Si Si, Kai-Yang Chiang, Cho-Jui Hsieh, Nikhil Rao, and Inderjit S Dhillon. 2016. Goal-directed inductive matrix completion. In SIGKDD .
[44]
Jianing Sun, Zhaoyue Cheng, Saba Zuberi, Felipe Pérez, and Maksims Volkovs. 2021. HGCF: Hyperbolic Graph Convolution Networks for Collaborative Filtering. In WWW .
[45]
Zequn Sun, Muhao Chen, Wei Hu, Chengming Wang, Jian Dai, and Wei Zhang. 2020. Knowledge Association with Hyperbolic Knowledge Graph Embeddings. In EMNLP .
[46]
Alexandru Tifrea, Gary Bécigneul, and Octavian-Eugen Ganea. 2018. Poincar$backslash$'e glove: Hyperbolic word embeddings. In ICLR .
[47]
Rianne van den Berg, Thomas N Kipf, and Max Welling. 2018. Graph Convolutional Matrix Completion. In SIGKDD .
[48]
Bart Vandereycken. 2013. Low-rank matrix completion by Riemannian optimization. SIAM Journal on Optimization (2013).
[49]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In NeurIPS .
[50]
Petar Velivc ković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2018. Graph attention networks. In ICLR .
[51]
Lucas Vinh Tran, Yi Tay, Shuai Zhang, Gao Cong, and Xiaoli Li. 2020. Hyperml: A boosting metric learning approach in hyperbolic space for recommender systems. In WSDM .
[52]
Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. 2019. Neural graph collaborative filtering. In SIGIR .
[53]
Jiancan Wu, Xiang Wang, Fuli Feng, Xiangnan He, Liang Chen, Jianxun Lian, and Xing Xie. 2021 a. Self-supervised graph learning for recommendation. In SIGIR .
[54]
Qitian Wu, Hengrui Zhang, Xiaofeng Gao, Junchi Yan, and Hongyuan Zha. 2021 b. Towards Open-World Recommendation: An Inductive Model-based Collaborative Filtering Approach. In ICML .
[55]
Yulei Yang and Dongsheng Li. 2020. Nenn: Incorporate node and edge features in graph neural networks. In ACML .
[56]
Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L Hamilton, and Jure Leskovec. 2018. Graph convolutional neural networks for web-scale recommender systems. In SIGKDD .
[57]
Chengkun Zhang and Junbin Gao. 2020. Hype-HAN: Hyperbolic Hierarchical Attention Network for Semantic Embedding. In IJCAI .
[58]
Muhan Zhang and Yixin Chen. 2018. Link prediction based on graph neural networks. In NeurIPS, Vol. 31. 5165--5175.
[59]
Muhan Zhang and Yixin Chen. 2020. Inductive Matrix Completion Based on Graph Neural Networks. In ICLR .
[60]
Sixiao Zhang, Hongxu Chen, Xiao Ming, Lizhen Cui, Hongzhi Yin, and Guandong Xu. 2021. Where are we in embedding spaces?. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 2223--2231.
[61]
Kai Zhong, Zhao Song, Prateek Jain, and Inderjit S Dhillon. 2019. Provable non-linear inductive matrix completion. In NeurIPS .
[62]
Zhihui Zhu, Qiuwei Li, Gongguo Tang, and Michael B Wakin. 2018. Global optimality in low-rank matrix optimization. IEEE Trans. Signal Process (2018).

Cited By

View all
  • (2024)Personalized recommendation via inductive spatiotemporal graph neural networkPattern Recognition10.1016/j.patcog.2023.109884145:COnline publication date: 1-Jan-2024
  • (2024)Hierarchical Graph Contrastive Learning for Review-Enhanced RecommendationMachine Learning and Knowledge Discovery in Databases. Research Track10.1007/978-3-031-70365-2_25(423-440)Online publication date: 22-Aug-2024
  • (2023)Geometric Matrix Completion with Collaborative Routing Between CapsulesICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP49357.2023.10095122(1-5)Online publication date: 4-Jun-2023

Index Terms

  1. Geometric Inductive Matrix Completion: A Hyperbolic Approach with Unified Message Passing

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      WSDM '22: Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining
      February 2022
      1690 pages
      ISBN:9781450391320
      DOI:10.1145/3488560
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 15 February 2022

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. graph neural networks
      2. hyperbolic space
      3. inductive matrix completion
      4. representation learning

      Qualifiers

      • Research-article

      Funding Sources

      • Autiralian Research Council Discovery Project

      Conference

      WSDM '22

      Acceptance Rates

      Overall Acceptance Rate 498 of 2,863 submissions, 17%

      Upcoming Conference

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)28
      • Downloads (Last 6 weeks)3
      Reflects downloads up to 13 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)Personalized recommendation via inductive spatiotemporal graph neural networkPattern Recognition10.1016/j.patcog.2023.109884145:COnline publication date: 1-Jan-2024
      • (2024)Hierarchical Graph Contrastive Learning for Review-Enhanced RecommendationMachine Learning and Knowledge Discovery in Databases. Research Track10.1007/978-3-031-70365-2_25(423-440)Online publication date: 22-Aug-2024
      • (2023)Geometric Matrix Completion with Collaborative Routing Between CapsulesICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP49357.2023.10095122(1-5)Online publication date: 4-Jun-2023

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media