Learning with Similarity Functions: a Tensor-Based Framework

Abstract

Machine learning algorithms are typically designed to deal with data represented as vectors. Several major applications, however, involve multi-way data, such as video sequences and multi-sensory arrays. In those cases, tensors endow a more consistent way to capture multi-modal relations, which may be lost by a conventional remapping of original data into a vector representation. This paper presents a tensor-oriented machine learning framework, and shows that the theory of learning with similarity functions provides an effective paradigm to support this framework. The proposed approach adopts a specific similarity function, which defines a measure of similarity between a pair of tensors. The performance of the tensor-based framework is evaluated on a set of complex, real-world, pattern-recognition problems. Experimental results confirm the effectiveness of the framework, which compares favorably with state-of-the-art machine learning methodologies that can accept tensors as inputs. Indeed, a formal analysis proves that the framework is more efficient than state-of-the-art methodologies also in terms of computational cost. The paper thus provides two main outcomes: (1) a theoretical framework that enables the use of tensor-oriented similarity notions and (2) a cognitively inspired notion of similarity that leads to computationally efficient predictors.

Appendix:

Unfolding is the process of reordering the elements of a tensor $\mathcal {T} \in \mathit {R}^{I_{1} \times I_{2} \times .... \times I_{N}}$ into a matrix $\mathcal {T} \in \mathit {R}^{P \times Q}$, where P × Q = I₁ × I₂ × ... × I_N .

The mode-n unfolding of $\mathcal {T}$, $\mathcal {T}_{(n)}$, defines a specific instance of such a procedure [20]. Accordingly, $\mathcal {T}_{(n)} \in \mathit {R}^{I_{n} \times I_{1}I_{2} I_{n-1}I_{n + 1}...I_{N}}$ and the reordering process follows the scheme exemplified in Fig. 9. In the example, a third-order tensor $\mathcal {T}$ is unfolded in the matrices $\mathcal {T}_{(1)}$, $\mathcal {T}_{(2)}$, and $\mathcal {T}_{(3)}$. The mode-1 unfolding $\mathcal {T}_{(1)}$ is obtained by first partitioning the original tensor according to the slices $\mathcal {T} (:,:,i3)$. Then, the eventual matrix is created by placing side-by-side the slices in the original order. Likewise, $\mathcal {T}_{(2)}$ and $\mathcal {T}_{(3)}$ are obtained by partitioning the original tensor according to the slices $\mathcal {T} (:,i_{2},:)$ and $\mathcal {T} (i_{1},:,i3)$, respectively. Such scheme can be easily extended to a generic tensor of order N.

Fig. 9

Unfolding of a third-order tensor