Abstract
Recent studies show that evolutionary optimizers are effective tools in solving real-world problem with complex and competing specifications. As more advanced multiobjective evolutionary optimizers (MOEO) are being designed and proposed, the issue of performance assessment has become increasingly important. While performance assessment could be done via theoretical and empirical approach, the latter is more practical and effective and has been adopted as the de facto approach in the evolutionary multiobjective optimization community. However, researches pertinent to empirical study have focused mainly on its individual components like test metrics and functions, there are limited discussions on the overall adequacy of empirical test in substantiating their statements made on the performance and behavior of the evaluated algorithm. As such, this paper aims to provide a holistic perspective towards the empirical investigation of MOEO and present a conceptual framework, which researchers could consider in the design and implementation of MOEO experimental study. This framework comprises of a structural algorithmic development plan and a general theory of adequacy in the context of evolutionary multiobjective optimization.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bosman, P.A.N., Thierens, D.: The balance between proximity and diversity in multiobjective evolutionary algorithms. IEEE Transactions on Evolutionary Computation 7(2), 174–188 (2003)
He, J., Yao, X.: Towards an analytic framework for analyzing the computation time of evolutionary algorithms. Artificial Intelligence 145, 59–97 (2003)
Veldhuizen, D.A.V., Lamont, G.B.: Multiobjective Evolutionary Algorithm Test Suites. In: ACM Symposium on Applied Computing, pp. 351–357. ACM Press, New York (1999)
Okabe, T.: Evolutionary Multi-Objective Optimization-On the Distribution of Offspring in Parameter and Fitness Space. PhD thesis, Technische Fakultaet der Universitaet Bielefeld, Shaker Verlag (2004)
Deb, K., Thiele, L., Laumanns, M., Zitzler, E.: Scalable MultiObjective Optimization Test Problems. In: Congress on Evolutionary Computation, vol. 1, pp. 825–830 (2002)
Knowles, J.D., Corne, D.W.: On metrics for comparing non-dominated sets. In: Congress on Evolutionary Computation Conference, pp. 711–716 (2002)
Hansen, M.P., Jaszkiewicz, A.: Evaluating the quality of approximations to the non-dominated set. Technical Report IMM-REP1998 -7, Technical University of Denmark (1998)
Zitzler, E., Thiele, L., Laumanns, M., Fonseca, C.M., Fonseca, V.G.: Performance assessment of multiobjective optimizers: an analysis and review. IEEE Transactions of Evolutionary Computation 7(2), 117–132 (2003)
Goodenough, J.B., Gerhart, S.L.: Towards a theory of test data selection. IEEE Transactions on Software Engineering 1(2), 156–173 (1975)
Weyuker, E.J.: Axiomatizing software test data adequacy. IEEE Transactions on Software Engineering 12(12), 1128–1138 (1986)
Teoh, E.J., Chiam, S.C., Goh, C.K., Tan, K.C.: Adapting evolutionary dynamics of variation for multi-objective optimization. In: IEEE Congress on Evolutionary Computation, vol. 2, pp. 1290–1297. IEEE, Los Alamitos (2005)
Khare, V., Yao, X., Deb, K.: Performance scaling of multi-objective evolutionary algorithms. In: Fonseca, C.M., Fleming, P.J., Zitzler, E., Deb, K., Thiele, L. (eds.) EMO 2003. LNCS, vol. 2632, pp. 376–390. Springer, Heidelberg (2003)
Hughes, E.J.: Evolutionary many-objective optimization: many once or one many? In: IEEE Congress on Evolutionary Computation, vol. 1, pp. 222–227. IEEE, Los Alamitos (2005)
Huband, S., Barone, L., While, R.L., Hingston, P.: A Scalable Multi-objective Test Problem Toolkit. In: Coello Coello, C.A., Hernández Aguirre, A., Zitzler, E. (eds.) EMO 2005. LNCS, vol. 3410, pp. 280–295. Springer, Heidelberg (2005)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer Berlin Heidelberg
About this paper
Cite this paper
Chiam, S.C., Goh, C.K., Tan, K.C. (2007). Adequacy of Empirical Performance Assessment for Multiobjective Evolutionary Optimizer. In: Obayashi, S., Deb, K., Poloni, C., Hiroyasu, T., Murata, T. (eds) Evolutionary Multi-Criterion Optimization. EMO 2007. Lecture Notes in Computer Science, vol 4403. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-70928-2_66
Download citation
DOI: https://doi.org/10.1007/978-3-540-70928-2_66
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-70927-5
Online ISBN: 978-3-540-70928-2
eBook Packages: Computer ScienceComputer Science (R0)