Datasets, tasks, and training methods for large-scale hypergraph learning

Abstract

Relations among multiple entities are prevalent in many fields, and hypergraphs are widely used to represent such group relations. Hence, machine learning on hypergraphs has received considerable attention, and especially much effort has been made in neural network architectures for hypergraphs (a.k.a., hypergraph neural networks). However, existing studies mostly focused on small datasets for a few single-entity-level downstream tasks and overlooked scalability issues, although most real-world group relations are large-scale. In this work, we propose new tasks, datasets, and scalable training methods for addressing these limitations. First, we introduce two pair-level hypergraph-learning tasks to formulate a wide range of real-world problems. Then, we build and publicly release two large-scale hypergraph datasets with tens of millions of nodes, rich features, and labels. After that, we propose PCL, a scalable learning method for hypergraph neural networks. To tackle scalability issues, PCL splits a given hypergraph into partitions and trains a neural network via contrastive learning. Our extensive experiments demonstrate that hypergraph neural networks can be trained for large-scale hypergraphs by PCL while outperforming 16 baseline models. Specifically, the performance is comparable, or surprisingly even better than that achieved by training hypergraph neural networks on the entire hypergraphs without partitioning.

Appendix: Additional experimental settings

1.1 Details of datasets

DBLP is a co-authorship hypergraph where nodes and hyperedges correspond to publications and authors, respectively. Each publication’s class is labeled according to its field of study. Trivago is a hotel-web search hypergraph where each node indicates each hotel and each hyperedge corresponds to a user. If a user (hyperedge) has visited the website of a particular hotel (node), the corresponding node is added to the respective user hyperedge. Furthermore, each hotel’s class is labeled based on the country in which it is located. OGBN-MAG is originally a heterogeneous graph that contains comprehensive academic information including venue, author, publication, and affiliation information. We transform this heterogeneous graph into a hypergraph as described in Sect. 4, while a label of each node (publication) indicates a published venue of the corresponding publication.

1.2 Details of graph-based baseline methods

Since graph representation models (Kipf and Welling 2017; Veličković et al. 2018) require ordinary graph structure as an input, we transform original hypergraph datasets into ordinary graph datasets by using clique expansion, where each hyperedge is replaced with a clique in the resulting graph. Formally, the clique expansion is a transformation of a given hyperedge set $\mathcal {E}$ to a clique expanded edge set $\mathcal {E}_{G} = \bigcup _{e \in \mathcal {E}} \left( {\begin{array}{c}e\\ 2\end{array}}\right)$.

Specifically, for full-graph datasets of DBLP and Trivago, we directly obtain $\mathcal {E}_{G}$ from $\mathcal {E}$, the entire hyperedge set. For full-graph datasets of OGBN-MAG, the size of the resulting clique expanded edges is too large to be loaded into the main memory. To reduce its scale, we additionally employ sampling, as described in Algorithm 2. Specifically, for each hyperedge $e'$ whose size is greater than k and for each constituent node $v \in e'$, we uniformly sample k other nodes from $e'$ (line 7) and create k edges joining v and each of the k sampled nodes. Here, we set $k=10$ for the OGBN-MAG dataset. We fail to create full-graph datasets of AMiner and MAG since clique expansion runs out of memory even with small k around 3, and thus we cannot perform experiments on them.

For partitioned-graph datasets of DBLP, Trivago, and OGBN-MAG, we apply clique expansion to the hyperedge set in each partition and use the resulting clique-expanded edge set as that of the corresponding partition. For partitioned-graph datasets of AMiner and MAG, due to the scalability issue, we apply the sampling strategy described in Algorithm 2 to each partition $\mathcal {P}_{i}$ of $\varvec{\mathcal {P}}$ (i.e., the input is $\mathcal {P}^{\mathcal {E}}_{i}$ instead of $\mathcal {E}$) and treat the resulting edge set as the edge set of the corresponding partition. Here, we set k to 10.

1.3 Details of hyperaprameter settings

We now provide detailed hyperparameter settings of representation models and training methods. The number of layers and hidden dimension of all representation models are fixed to 2 and 128, respectively.

For representation models that are trained via supervised learning methods, we train each model for 100 epochs. We tune a learning rate of each model within $\{0.01, 0.001, 0.0001\}$. For every 10 epochs, we measure the validation AP score and save the model parameters. Then, we designate the checkpoint with the highest validation AP score as the final model parameters.

For representation models that are trained via all versions of PCL, we tune the number of self-supervised learning epochs within $\{25, 50\}$, while we set a broader search space, specifically $\{20, 40, 60, 80, 100\}$, for that of other self-supervised learning methods. We tune the learning rate of the self-supervised learning within $\{0.001, 0.0001\}$ for all self-supervised learning methods. In addition, for methods that require augmentation steps, we tune the extent of node feature augmentation $p_{v}$ within $\{0.3, 0.4\}$, and the extent of topological augmentation $p_{e}$ within $\{0.3, 0.4\}$. Furthermore, for methods that require negative samples for contrastive learning, we tune the number of negative samples N within $\{1, 2\}$. The temperature parameter $\tau$ for all self-supervised learning methods, and the scalar $\lambda$ that controls the strength of inter-partition loss in PCL+PINS are both fixed to 0.5. Lastly, we train downstream task classifiers of all self-supervised learning methods with a learning rate of 0.001. We train the classifiers for 100 epochs, and for every 10 epochs, we measure the validation AP score and save the classifier parameters. Then, we designate the checkpoint with the highest validation AP score as the final classifier parameters.