Abstract
There exist optimization problems with the target objective, which is to be optimized, and several extra objectives. The extra objectives may or may not be helpful in optimization process in terms of the number of objective evaluations necessary to reach an optimum of the target objective.
OneMax+ZeroMax is a previously proposed benchmark optimization problem where the target objective is OneMax and a single extra objective is ZeroMax, which is equal to the number of zero bits in the bit vector. This is an example of a problem where extra objectives are not good, and objective selection methods should ignore the extra objectives. The EA+RL method is a method which selects objectives to be optimized by evolutionary algorithms (EA) using reinforcement learning (RL). Previously it was shown that it runs in \(\varTheta (N \log N)\) on OneMax+ZeroMax when configured to use the randomized local search algorithm and the Q-learning algorithm with the greedy exploration strategy.
We present the runtime analysis for the case when the \((1+1)\)-EA algorithm is used. It is shown that the expected running time is at most \(3.12 e N \log N\).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Brockhoff, D., Friedrich, T., Hebbinghaus, N., Klein, C., Neumann, F., Zitzler, E.: On the effects of adding objectives to plateau functions. Trans. Evol. Comput. 13(3), 591–603 (2009)
Buzdalov, M., Buzdalova, A., Shalyto, A.: A first step towards the runtime analysis of evolutionary algorithm adjusted with reinforcement learning. In: Proceedings of the International Conference on Machine Learning and Applications, vol. 1, pp. 203–208. IEEE Computer Society (2013)
Buzdalova, A., Buzdalov, M.: Increasing efficiency of evolutionary algorithms by choosing between auxiliary fitness functions with reinforcement learning. In: Proceedings of the International Conference on Machine Learning and Applications, vol. 1, pp. 150–155 (2012)
Hajek, B.: Hitting-time and occupation-time bounds implied by drift analysis with applications. Adv. Appl. Probab. 14(3), 502–525 (1982)
Handl, J., Lovell, S.C., Knowles, J.D.: Multiobjectivization by decomposition of scalar cost functions. In: Rudolph, G., Jansen, T., Lucas, S., Poloni, C., Beume, N. (eds.) PPSN 2008. LNCS, vol. 5199, pp. 31–40. Springer, Heidelberg (2008)
Jensen, M.T.: Helper-objectives: using multi-objective evolutionary algorithms for single-objective optimisation: evolutionary computation combinatorial optimization. J. Math. Model. Algorithms 3(4), 323–347 (2004)
Knowles, J.D., Watson, R.A., Corne, D.W.: Reducing local optima in single-objective problems by multi-objectivization. In: Zitzler, E., Deb, K., Thiele, L., Coello Coello, C.A., Corne, D.W. (eds.) EMO 2001. LNCS, vol. 1993, pp. 269–283. Springer, Heidelberg (2001)
Lochtefeld, D.F., Ciarallo, F.W.: Helper-objective optimization strategies for the job-shop scheduling problem. Appl. Soft Comput. 11(6), 4161–4174 (2011)
Neumann, F., Wegener, I.: Can single-objective optimization profit from multiobjective optimization? In: Knowles, J., Corne, D., Deb, K., Chair, D.R. (eds.) Multiobjective Problem Solving from Nature. Natural Computing Series, pp. 115–130. Springer, Heidelberg (2008)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
Witt, C.: Optimizing linear functions with randomized search heuristics - the robustness of mutation. In: Proceedings of the 29th Annual Symposium on Theoretical Aspects of Computer Science, pp. 420–431 (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Antipov, D., Buzdalov, M., Doerr, B. (2015). Runtime Analysis of \((1+1)\) Evolutionary Algorithm Controlled with Q-learning Using Greedy Exploration Strategy on OneMax+ZeroMax Problem. In: Ochoa, G., Chicano, F. (eds) Evolutionary Computation in Combinatorial Optimization. EvoCOP 2015. Lecture Notes in Computer Science(), vol 9026. Springer, Cham. https://doi.org/10.1007/978-3-319-16468-7_14
Download citation
DOI: https://doi.org/10.1007/978-3-319-16468-7_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-16467-0
Online ISBN: 978-3-319-16468-7
eBook Packages: Computer ScienceComputer Science (R0)