Improving vector space embedding of graphs through feature selection algorithms

Abstract

Graph based pattern representation offers a versatile alternative to vectorial data structures. Therefore, a growing interest in graphs can be observed in various fields. However, a serious limitation in the use of graphs is the lack of elementary mathematical operations in the graph domain, actually required in many pattern recognition algorithms. In order to overcome this limitation, the present paper proposes an embedding of a given graph population in a vector space ${\mathbb{R}}^n$. The key idea of this embedding approach is to interpret the distances of a graph g to a number of prototype graphs as numerical features of g. In previous works, the prototypes were selected beforehand with heuristic selection algorithms. In the present paper we take a more fundamental approach and regard the problem of prototype selection as a feature selection or dimensionality reduction problem, for which many methods are available. With several experiments we show the feasibility of graph embedding based on prototypes obtained from such feature selection algorithms and demonstrate their potential to outperform previous approaches.

Introduction

After decades of focusing on independent and identically distributed representation formalisms in pattern recognition, more and more effort is now rendered in various research fields on graph based representation [1]. Object representation by means of graphs is advantageous compared to vectorial approaches because of two reasons. First, graphs do not suffer from the constraint of fixed dimensionality. That is, the number of nodes and edges in a graph is not limited a priori and depends on the size and the complexity of the actual object to be modeled. Second, graphs are able to represent not only the values of object properties, i.e. features, but can be used to explicitly model relations that exist between different parts of an object.

Due to their ability to represent properties of entities and binary relations at the same time, graphs have found widespread applications in science and engineering. They are used, for instance, in bioinformatics and chemoinformatics [2], [3], [4], web content and data mining [5], [6], [7], classifying images from various fields [8], [9], [10], symbol and character recognition [11], [12], [13], and in computer network analysis [14], [15].

However, one drawback of graphs, when compared to feature vectors, is the significantly increased complexity of many algorithms. For example, the comparison of two feature vectors for identity can be accomplished in linear time with respect to the length of the two vectors. For the analogous operation on general graphs, i.e. testing two graphs for isomorphism, only exponential algorithms are known today. Another serious limitation in the use of graphs for pattern recognition tasks is the little mathematical structure in the domain of graphs. For example, computing the (weighted) sum or the product of a pair of entities (which are elementary operations required in many classification and clustering algorithms) is not possible in the domain of graphs, or is at least not defined in a standardized way. Due to these general problems in the graph domain, we observe a lack of algorithmic tools for graph based pattern recognition.

The present paper's objective is to benefit from both the universality of graphs for pattern representation and the computational convenience of vectors for pattern recognition. To this end we propose a general procedure for mapping graphs from arbitrary graph domains to a real vector space by means of functions $\phi :\mathcal{G}\to {\mathbb{R}}^n$. Based on the resulting graph maps, the considered pattern recognition task is eventually carried out. Hence, the whole arsenal of algorithmic tools readily available for vectorial data can be applied to graphs (more exactly to graph maps $\phi (g)\in {\mathbb{R}}^n$).

The presented approach for graph embedding is primarily based on the idea proposed in [16] where the dissimilarity representation for pattern recognition in conjunction with feature vectors was first introduced. In the current work we go one step further and generalize the methods described in [16] to the domain of graphs. The key idea of the novel graph embedding approach is to use the distances of an input graph g to n prototype graphs $\mathcal{P}=\{p_1,\dots ,p_n\}$ as a vectorial description of g. Apparently, the definition of the prototype set $\mathcal{P}$ is a critical issue since the graphs in $\mathcal{P}$ affect the resulting vectors. Thus, a good selection of prototypes is crucial to succeed with the algorithm to be applied in the embedding space. Commonly, the prototypes are selected from a training set $\mathcal{T}$ of existing graphs before the embedding is carried out. In previous works, this prototype selection uses some heuristics based on distances between the members of $\mathcal{T}$ [17]. In the present paper, however, a new approach is proposed where all available elements from the training set are used as prototypes in a first step, i.e. for our embedding we define $\mathcal{P}=\mathcal{T}$. Subsequently, feature subset selection algorithms are applied to the vector space embedded graphs. In other words, rather than selecting the prototypes beforehand, the embedding is carried out first and then the problem of prototype selection is reduced to a feature selection problem. Thus, by means of this more fundamental approach, we bypass the difficult problem of selecting adequate prototypes.

A preliminary version of the current paper appeared in [18]. The current paper has been significantly extended with respect to the underlying methodology and the experimental evaluation. First, two additional feature selection algorithms are applied to vector space embedded graphs. In addition, and for the sake of completeness, the results of two dimensionality reduction algorithms are also included in the experimental evaluation (originally presented in [19], [20] for the first time). The number of data sets where our embedding procedure is tested on is also considerably increased and results of an additional reference system, a similarity kernel, are added. Finally, a detailed discussion and results about the validation of the meta parameters of our approach are provided.

The remainder of this paper is organized as follows. In the next section we define basic concepts and introduce our notation. Then, in Section 3, the proposed approach for graph embedding in real vector spaces is described. The feature selection algorithms applied to the vector space embedded graphs are described in Section 4. In Section 5 we report a number of experiments and present results achieved with our embedding method. Finally, in Section 6 we summarize our work and draw conclusions.