Yanli Yang
ABSTRACT
Artificial neural network is an important force to promote the development of artificial intelligence, but it usually needs to be trained before use. The initial weight given randomly is the most widely used method when training neural networks. However, randomly given initial weights are independent of the samples. A weight initialization method associated with samples for deep feedforward neural network (DFFNN) is proposed. The initial weights set by this method are a combination of the original weights given randomly and the weights obtained by the first epoch training. The initial weights not only have random characteristics, but are also closely related to the samples to be trained. The proposed method is tested with the bearing data provided by the Case Western Reserve University (CWRU) Bearing Data Center. The testing results show that the proposed method can accelerate the training of DFFNN to some extent.
- Hinton G. E. and Salakhutdinov R. R. 2006. Reducing the dimensionality of data with neural networks, Science 313, 504--507.Google Scholar
Cross Ref
- Wang L. N., Yang Y., Min R. Q., and Chakradhar S. 2017. Accelerating deep neural network training with inconsistent stochastic gradient descent, Neural Networks 93, 219--229.Google ScholarCross Ref
- Nur A. S., Radzi N. H. M., and Ibrahim A. O. 2014. Artificial neural network weight optimization: A review, TELKOMNIKA Indonesian Journal of Electrical Engineering 12, 6897--6902.Google ScholarCross Ref
- Combes R. T. D., Pezeshki M., Shabanian S., Courville A., and Bengio Yoshua, 2018. On the learning dynamics of deep neural networks, arXiv: 1809.06848v1.Google Scholar
- Ramos E. Z., Nakakuni M., and Yfantis E. 2017. Quantitative measures to evaluate neural network weight initialization strategies, in IEEE 7th Annual Computing and Communication Workshop and Conference (CCWC), Las Vegas, USA, 2017.Google Scholar
- Kumar S. K., On weight initialization in deep neural networks, arXiv:1704.08863v2, 2017.Google Scholar
- Daniely A., Frostig R., and Singer Y. 2016. Toward deeper understanding of neural networks: The power of initialization and a dual view on expressivity, in 30th Conference on Neural Information Processing Systems (NIPS 2016), Barcelona, Spain, 2016, pp. 1--9.Google Scholar
- Lee A., Geem Z. W., and Suh K. D., 2016. Determination of optimal initial weights of an artificial neural network by using the harmony search algorithm: Application to breakwater armor stones, Applied Sciences 6, 1--17.Google ScholarCross Ref
- Schmidhuber J. 2015. Deep learning in neural networks: An overview, Neural Networks 61, 85--117.Google ScholarDigital Library
- Case Western Reserve University Bearing Data Center Website. <http://csegroups.case.edu/bearingdatacenter/home>.Google Scholar
- DeepLearnToolbox website. <https://github.com/rasmusbergpalm/DeepLearnToolbox>.Google Scholar
- Yang Y. L., Fu P. Y., and He Y. C. 2018. Bearing faults automatic classification based on deep learning, IEEE Access 6, 71540--71554.Google ScholarCross Ref
- Yang Y. L. and Fu P. Y. 2018. Rolling-element bearing fault data automatic clustering based on wavelet and deep neural network, Shock and Vibration 2018, Article ID 3047830, 11 pages.Google Scholar
Index Terms
- A Weight Initialization Method Associated with Samples for Deep Feedforward Neural Network
Recommendations
A review on weight initialization strategies for neural networks
AbstractOver the past few years, neural networks have exhibited remarkable results for various applications in machine learning and computer vision. Weight initialization is a significant step employed before training any neural network. The weights of a ...
Mutual information based weight initialization method for sigmoidal feedforward neural networks
When a sigmoidal feedforward neural network (SFNN) is trained by the gradient-based algorithms, the quality of the overall learning process strongly depends on the initial weights. To improve the algorithm stability and avoid local minima, a Mutual ...
Weight and bias initialization routines for Sigmoidal Feedforward Network
AbstractThe success of the Sigmoidal Feedforward Networks in the solution of complex learning task can be attributed to their Universal Approximation Property. These networks are trained using non-linear iterative optimization method (of first-order or ...
Comments