Clifford Embeddings – A Generalized Approach for Embedding in Normed Algebras

Abstract

A growing number of knowledge graph embedding models exploit the characteristics of division algebras (e.g., $\mathbb {R}$, $\mathbb {C}$, $\mathbb {H}$, and $\mathbb {O}$) to learn embeddings. Yet, recent empirical results suggest that the suitability of algebras is contingent upon the knowledge graph being embedded. In this work, we tackle the challenge of selecting the algebra within which a given knowledge graph should be embedded by exploiting the fact that Clifford algebras $Cl_{p,q}$ generalize over $\mathbb {R}$, $\mathbb {C}$, $\mathbb {H}$, and $\mathbb {O}$. Our embedding approach, Keci, is the first knowledge graph embedding model that can parameterize the algebra within which it operates. With Keci, the selection of an underlying algebra becomes a part of the learning process. Specifically, Keci starts the training process by learning real-valued embeddings for entities and relations in $\mathbb {R}^m=Cl^m_{0,0}$. At each mini-batch update, Keci can steer the training process from $Cl^m_{p,q}$ to $Cl^m_{p+1,q}$ or $Cl^m_{p,q+1}$ by processing the training loss. In this way, Keci can decide the algebra within which it operates in a data-driven fashion. Consequently, Keci is a generalization of previous approaches such as DistMult, ComplEx, QuatE, and OMult. Our evaluation suggests that Keci outperforms state-of-the-art embedding approaches on seven benchmark datasets. We provide an open-source implementation of Keci, including pre-trained models, training and evaluation scripts (https://github.com/dice-group/dice-embeddings).