Tensorial Neural Networks and Its Application in Longitudinal Network Data Analysis

Abstract

The traditional neural networks are only able to process vectorial data, resulting in the loss of spatial information in high-dimensional structural data when vectorising data. The matrix neural networks (MatNet), a new approach, is only capable of capturing structural information on the first and the second dimension/mode of matrix data. Although the state-of-the-art method multilinear tensor regression (MLTR) manages to capture the linear relational information in high dimensions, the possible nonlinear relationships within multidimensional data may be ignored. To analyse both linear and nonlinear relationships among each mode of the multidimensional relational data, a new model, named tensorial neural networks, is proposed. Within the tensorial neural networks, the hidden layers are in high-dimensions rather than one dimension or two dimensions. The backpropagation algorithm for tensorial neural networks is derived and provided. The performance of the new approach is assessed in analysing longitudinal network data which contains weekly international relationships among 25 countries from 2004 to mid-2014 from World-Wide Integrated Crisis Early Warning System. In other words, the application of this newly proposed method, tensorial neural networks, is on international relationship study in this paper. The dependencies among the international relationship data are generally reciprocity and transitivity which are also the interests of the research.