Loading [a11y]/accessibility-menu.js
Enhanced recurrent network training | IEEE Conference Publication | IEEE Xplore

Enhanced recurrent network training


Abstract:

In this paper, we introduce new, more efficient, methods for training recurrent neural networks (RNNs). These methods are based on a new understanding of the error surfac...Show More

Abstract:

In this paper, we introduce new, more efficient, methods for training recurrent neural networks (RNNs). These methods are based on a new understanding of the error surfaces of RNNs that has been developed in recent years. These error surfaces contain spurious valleys that disrupt the search for global minima. The spurious valleys are caused by instabilities in the networks, which become more pronounced with increased prediction horizons. The new methods described in this paper increase the prediction horizons in a principled way that enables the search algorithms to avoid the spurious valleys. The paper also presents a new method for determining when an RNN is extrapolating. When an RNN operates outside the region spanned by the training set, adequate performance cannot be guaranteed. The new method presented in this paper accurately predicts poor performance well before its onset.
Date of Conference: 12-17 July 2015
Date Added to IEEE Xplore: 01 October 2015
ISBN Information:

ISSN Information:

Conference Location: Killarney, Ireland

Contact IEEE to Subscribe

References

References is not available for this document.