ABSTRACT
With the rapid growth of interaction data, many clustering methods have been proposed to discover interaction patterns as prior knowledge beneficial to downstream tasks. Considering that an interaction can be seen as an action occurring among multiple objects, most existing methods model the objects and their pair-wise relations as nodes and links in graphs. However, they only model and leverage part of the information in real entire interactions, i.e., either decompose the entire interaction into several pair-wise sub-interactions for simplification, or only focus on clustering some specific types of objects, which limits the performance and explainability of clustering. To tackle this issue, we propose to Co-cluster the Interactions via Attentive Hypergraph neural network (CIAH). Particularly, with more comprehensive modeling of interactions by hypergraph, we propose an attentive hypergraph neural network to encode the entire interactions, where an attention mechanism is utilized to select important attributes for explanations. Then, we introduce a salient method to guide the attention to be more consistent with real importance of attributes, namely saliency-based consistency. Moreover, we propose a novel co-clustering method to perform a joint clustering for the representations of interactions and the corresponding distributions of attribute selection, namely cluster-based consistency. Extensive experiments demonstrate that our CIAH significantly outperforms state-of-the-art clustering methods on both public datasets and real industrial datasets.
Supplemental Material
- Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, Manjunath Kudlur, Josh Levenberg, Rajat Monga, Sherry Moore, Derek Gordon Murray, Benoit Steiner, Paul A. Tucker, Vijay Vasudevan, Pete Warden, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2016. TensorFlow: A System for Large-Scale Machine Learning. In 12th USENIX Symposium on Operating Systems Design and Implementation. USENIX Association, 265--283.Google Scholar
- Karan Aggarwal, Georgios Theocharous, and Anup B. Rao. 2020. Dynamic Clustering with Discrete Time Event Prediction. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 1501--1504.Google Scholar
- Song Bai, Feihu Zhang, and Philip HS Torr. 2021. Hypergraph convolution and hypergraph attention. Pattern Recognition , Vol. 110 (2021), 107637.Google ScholarCross Ref
- Deyu Bo, Xiao Wang, Chuan Shi, Meiqi Zhu, Emiao Lu, and Peng Cui. 2020. Structural Deep Clustering Network. In Proceedings of the Web Conference . 11.Google ScholarDigital Library
- Chantima Buaklee and Sukree Sinthupinyo. 2018. Similar Cluster recommendation of Product Purchases by Pages liked Analysis. In 10th International Conference on Electronics, Computers and Artificial Intelligence. IEEE, 1--5.Google ScholarCross Ref
- Daniel Deutch and Nave Frost. 2019. Constraints-based explanations of classifications. In ICDE. IEEE, 530--541.Google Scholar
- Yuxiao Dong, Nitesh V. Chawla, and Ananthram Swami. 2017. metapath2vec: Scalable Representation Learning for Heterogeneous Networks. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining . ACM, 135--144.Google ScholarDigital Library
- Haoyi Fan, Fengbin Zhang, Yuxuan Wei, Zuoyong Li, Changqing Zou, Yue Gao, and Qionghai Dai. 2021. Heterogeneous Hypergraph Variational Autoencoder for Link Prediction . IEEE Transactions on Pattern Analysis and Machine Intelligence (2021).Google ScholarDigital Library
- Yifan Feng, Haoxuan You, Zizhao Zhang, Rongrong Ji, and Yue Gao. 2019. Hypergraph Neural Networks. In Proceedings of the AAAI Conference on Artificial Intelligence. 3558--3565.Google ScholarDigital Library
- Mohamed H Gad-Elrab, Daria Stepanova, Trung-Kien Tran, Heike Adel, and Gerhard Weikum. 2020. ExCut: explainable embedding-based clustering over knowledge graphs. In International Semantic Web Conference. Springer, 218--237.Google ScholarDigital Library
- Aditya Grover and Jure Leskovec. 2016. node2vec: Scalable Feature Learning for Networks. In KDD. Association for Computing Machinery, 855--864.Google Scholar
- Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. 2017. DeepFM: A Factorization-Machine Based Neural Network for CTR Prediction. In Proceedings of the 26th International Joint Conference on Artificial Intelligence . AAAI Press, 1725--1731.Google ScholarCross Ref
- J. A. Hartigan and M. A. Wong. 1979. Algorithm AS 136: A K-Means Clustering Algorithm . Applied Statistics , Vol. 28, 1 (1979), 100.Google ScholarCross Ref
- G. E. Hinton. 2006. Reducing the Dimensionality of Data with Neural Networks . Science , Vol. 313, 5786 (July 2006), 504--507.Google ScholarCross Ref
- Youpeng Hu, Xunkai Li, Yujie Wang, Yixuan Wu, Yining Zhao, Chenggang Yan, Jian Yin, and Yue Gao. 2021. Adaptive Hypergraph Auto-Encoder for Relational Data Clustering . IEEE Transactions on Knowledge and Data Engineering (2021).Google ScholarCross Ref
- Ziniu Hu, Yuxiao Dong, Kuansan Wang, and Yizhou Sun. 2020. Heterogeneous Graph Transformer. In Proceedings of The Web Conference . ACM, 2704--2710.Google ScholarDigital Library
- Sarthak Jain and Byron C. Wallace. 2019. Attention is not Explanation. In NAACL. Association for Computational Linguistics, 3543--3556.Google Scholar
- Yugang Ji, Chuan Shi, Yuan Fang, Xiangnan Kong, and Mingyang Yin. 2020. Semi-supervised Co-Clustering on Attributed Heterogeneous Information Networks . Information Processing & Management , Vol. 57, 6 (2020), 102338.Google ScholarCross Ref
- Jiarui Jin, Jiarui Qin, Yuchen Fang, Kounianhua Du, Weinan Zhang, Yong Yu, Zheng Zhang, and Alexander J. Smola. 2020. An Efficient Neighborhood-based Interaction Model for Recommendation on Heterogeneous Graph. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, 75--84.Google Scholar
- Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In 3rd International Conference on Learning Representations .Google Scholar
- Scott M Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. In NIPS. 4768--4777.Google Scholar
- Xusheng Luo, Yonghua Yang, Kenny Qili Zhu, Yu Gong, and Keping Yang. 2019. Conceptualize and Infer User Needs in E-commerce. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management. ACM, 2517--2525.Google ScholarDigital Library
- Michal Moshkovitz, Sanjoy Dasgupta, Cyrus Rashtchian, and Nave Frost. 2020. Explainable k-means and k-medians clustering. In International Conference on Machine Learning. PMLR, 7055--7065.Google Scholar
- Giovanna Andrea Pinilla-De La Cruz, Rodrigo Rabetino, and Jussi Kantola. 2021. Public-Private Partnerships (PPPs) in Energy: Co-citation Analysis Using Network and Cluster Visualization. In Intelligent Human Systems Integration 2021. Springer, 460--465.Google Scholar
- Hui Qiao, Yangyang Liu, Xuewen Dong, and Di Lu. 2019. Personalized Recommendation for Cold-Start Users via Cluster-Level Latent Feature Model. In International Conference on Networking and Network Applications. 290--295.Google ScholarCross Ref
- Soumajyoti Sarkar, Mohammad Almukaynizi, Jana Shakarian, and Paulo Shakarian. 2019. Mining user interaction patterns in the darkweb to predict enterprise cyber incidents. Social Network Analysis and Mining , Vol. 9, 1 (Dec. 2019), 57.Google ScholarCross Ref
- Sofia Serrano and Noah A. Smith. 2019. Is Attention Interpretable?. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 2931--2951.Google Scholar
- Weiping Song, Chence Shi, Zhiping Xiao, Zhijian Duan, Yewen Xu, Ming Zhang, and Jian Tang. 2019. Autoint: Automatic feature interaction learning via self-attentive neural networks. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management. 1161--1170.Google ScholarDigital Library
- Mukund Sundararajan, Ankur Taly, and Qiqi Yan. 2017. Axiomatic Attribution for Deep Networks. In Proceedings of the 34th International Conference on Machine Learning, Vol. 70. PMLR, 3319--3328.Google Scholar
- Hongyan Tang, Junning Liu, Ming Zhao, and Xudong Gong. 2020. Progressive Layered Extraction (PLE): A Novel Multi-Task Learning (MTL) Model for Personalized Recommendations. In Fourteenth ACM Conference on Recommender Systems . Association for Computing Machinery, 269--278.Google ScholarDigital Library
- Loc Hoang Tran and Linh Hoang Tran. 2020. Directed hypergraph neural network. arXiv:2008.03626 [cs, stat] (Aug. 2020).Google Scholar
- Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of machine learning research , Vol. 9, 11 (2008).Google Scholar
- David S Wishart, Yannick D Feunang, An C Guo, Elvis J Lo, Ana Marcu, Jason R Grant, Tanvir Sajed, Daniel Johnson, Carin Li, Zinat Sayeeda, Nazanin Assempour, Ithayavani Iynkkaran, Yifeng Liu, Adam Maciejewski, Nicola Gale, Alex Wilson, Lucy Chin, Ryan Cummings, Diana Le, Allison Pon, Craig Knox, and Michael Wilson. 2018. DrugBank 5.0: a major update to the DrugBank database for 2018. Nucleic Acids Research , Vol. 46 (Jan. 2018), D1074--D1082.Google ScholarCross Ref
- Shiwen Wu, Wentao Zhang, Fei Sun, and Bin Cui. 2020. Graph Neural Networks in Recommender Systems: A Survey . arXiv:2011.02260 [cs] (2020).Google Scholar
- Junyuan Xie, Ross Girshick, and Ali Farhadi. 2016. Unsupervised Deep Embedding for Clustering Analysis. In Proceedings of The 33rd International Conference on Machine Learning, Vol. 48. PMLR, 478--487.Google Scholar
- Zhe Xue, Junping Du, Dawei Du, and Siwei Lyu. 2019. Deep low-rank subspace ensemble for multi-view clustering. Information Sciences , Vol. 482 (2019), 210--227.Google ScholarDigital Library
- Naganand Yadati, Madhav Nimishakavi, Prateek Yadav, Vikram Nitin, Anand Louis, and Partha Talukdar. 2019. HyperGCN: A New Method For Training Graph Convolutional Networks on Hypergraphs . In Advances in Neural Information Processing Systems 32. Curran Associates, Inc., 1511--1522.Google Scholar
- Renchi Yang, Jieming Shi, Yin Yang, Keke Huang, Shiqi Zhang, and Xiaokui Xiao. 2021. Effective and Scalable Clustering on Massive Attributed Graphs. In Proceedings of the Web Conference. ACM, Ljubljana Slovenia, 3675--3687.Google ScholarDigital Library
- Seongjun Yun, Minbyul Jeong, Raehyun Kim, Jaewoo Kang, and Hyunwoo J Kim. 2019. Graph Transformer Networks. In NIPS, Vol. 32. Curran Associates, Inc.Google Scholar
- Ruochi Zhang, Yuesong Zou, and Jian Ma. 2020. Hyper-SAGNN: a self-attention based graph neural network for hypergraphs. In International Conference on Learning Representations .Google Scholar
Index Terms
- Co-clustering Interactions via Attentive Hypergraph Neural Network
Recommendations
Efficient and Effective Attributed Hypergraph Clustering via K-Nearest Neighbor Augmentation
PACMMODHypergraphs are an omnipresent data structure used to represent high-order interactions among entities. Given a hypergraph H wherein nodes are associated with attributes, attributed hypergraph clustering (AHC) aims to partition the nodes in H into k ...
Multi-stage Hierarchical Clustering Method Based on Hypergraph
Intelligent Computing MethodologiesAbstractClustering analysis is a data analysis technique, it groups a set of data points into multiple clusters with similar data points. However, clustering of high dimensional data is still a difficult task. In order to facilitate this task, people ...
Clustering ensemble via structured hypergraph learning
AbstractClustering ensemble integrates multiple base clustering results to obtain a consensus result and thus improves the stability and robustness of the single clustering method. Since it is natural to use a hypergraph to represent the ...
Highlights- A new measurement of the quality of base clusters is proposed.
- A framework of ...
Comments