Abstract
The adaptation of individual learning rates is important for many learning tasks, particularly in the case of nonstationary learning environments. Sutton has presented with the Incremental Delta Bar Delta algorithm a versatile method for many tasks. However, this algorithm was formulated only for linear models. A straightforward generalization to nonlinear models is possible, but we show in this work that it poses some obstacles, namely the stability of the learning algorithm. We propose a new self-regulation of the model’s activation which ensures stability. Our algorithm shows better performance than other approaches on a nonstationary benchmark task. Furthermore we show how to derive this algorithm from basic loss functions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Almeida, L., Langlois, T., Amaral, J.D.: On-line step size adaptation. Technical Report RT07/97, INESC. 9 Rua Alves Redol, Lisboa, Portugal (1997)
Bagheri, S., Thill, M., Koch, P., Konen, W.: Online adaptable learning rates for the game Connect-4. Submitted to IEEE Trans. on Computational Intelligence and AI in Games (2014)
Dabney, W., Barto, A.G.: Adaptive step-size for online temporal difference learning. In: 26th AAAI Conference on Artificial Intelligence (2012)
Jacobs, R.A.: Increased rates of convergence through learning rate adaptation. Neural Networks 1(4), 295–307 (1988)
Koop, A.: Investigating experience: Temporal coherence and empirical knowledge representation. Master’s thesis, University of Alberta, Canada (2008)
Li, C., Ye, Y., Miao, Q., Shen, H.-L.: Kimel: A kernel incremental metalearning algorithm. Signal Processing 93(6), 1586–1596 (2013); Special issue on Machine Learning in Intelligent Image Processing
Mahmood, A.R., Sutton, R.S., Degris, T., Pilarski, P.M.: Tuning-free step-size adaptation. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2121–2124. IEEE (2012)
Schraudolph, N.N.: Online local gain adaptation for multi-layer perceptrons. Technical Report IDSIA-09-98, Istituto Dalle Molle Di Studi Sull Intelligenza Artificiale (IDSIA), Switzerland (1998)
Schraudolph, N.N.: Online learning with adaptive local step sizes. In: Neural Nets WIRN Vietri 1999, pp. 151–156. Springer (1999)
Sutton, R.S.: Adapting bias by gradient descent: An incremental version of delta-bar-delta. In: Swartout, W.R. (ed.) 10th AAAI Conference on Artificial Intelligence, pp. 171–176 (1992)
Sutton, R.S.: Gain adaptation beats least squares. In: 7th Yale Workshop on Adaptive and Learning Systems, pp. 161–166 (1992)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Konen, W., Koch, P. (2014). Adaptation in Nonlinear Learning Models for Nonstationary Tasks. In: Bartz-Beielstein, T., Branke, J., Filipič, B., Smith, J. (eds) Parallel Problem Solving from Nature – PPSN XIII. PPSN 2014. Lecture Notes in Computer Science, vol 8672. Springer, Cham. https://doi.org/10.1007/978-3-319-10762-2_29
Download citation
DOI: https://doi.org/10.1007/978-3-319-10762-2_29
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-10761-5
Online ISBN: 978-3-319-10762-2
eBook Packages: Computer ScienceComputer Science (R0)