Abstract:
Neural networks are becoming more and more important for intelligent communications and their theoretical research has become a top priority. Loss surfaces are crucial to...View moreMetadata
Abstract:
Neural networks are becoming more and more important for intelligent communications and their theoretical research has become a top priority. Loss surfaces are crucial to understand and improve performance in neural networks. In this paper, the Hessian matrix of second order optimization method is analyzed through the analytical framework of random matrix theory (RMT) in order to understand the geometry of loss surfaces. The limited spectrum distribution, extreme eigenvalue distribution, and standard condition number (SCN) of Hessian matrix are analyzed to understand their asymptotic characteristics. Moreover, the relationships among the extreme eigenvalue distribution, SCN, and the convergence of loss surfaces are investigated. The above analyses give insight into utilizing RMT to analyze the neural network theory.
Date of Conference: 20-24 May 2019
Date Added to IEEE Xplore: 15 July 2019
ISBN Information: