Jacopo Panerati
Giovanni Beltrame
Skip Abstract Section
Abstract
This article presents a detailed overview and the experimental comparison of 15 multi-objective design-space exploration (DSE) algorithms for high-level design. These algorithms are collected from recent literature and include heuristic, evolutionary, and statistical methods. To provide a fair comparison, the algorithms are classified according to the approach used and examined against a large set of metrics. In particular, the effectiveness of each algorithm was evaluated for the optimization of a multiprocessor platform, considering initial setup effort, rate of convergence, scalability, and quality of the resulting optimization. Our experiments are performed with statistical rigor, using a set of very diverse benchmark applications (a video converter, a parallel compression algorithm, and a fast Fourier transformation algorithm) to take a large spectrum of realistic workloads into account. Our results provide insights on the effort required to apply each algorithm to a target design space, the number of simulations it requires, its accuracy, and its precision. These insights are used to draw guidelines for the choice of DSE algorithms according to the type and size of design space to be optimized.
- G. Beltrame, L. Fossati, and D. Sciuto. 2009. ReSP: A nonintrusive transaction-level reflective MPSoC simulation platform for design space exploration. IEEE Trans. Comput. Aid. Des. Integr. Circuits Syst. 28, 12, 1857--1869. Google Scholar
Digital Library
- G. Beltrame, L. Fossati, and D. Sciuto. 2010. Decision-theoretic design space exploration of multiprocessor platforms. IEEE Trans. Comput. Aid. Des. Integr. Circuits Syst. 29, 7, 1083--1095. DOI: http://dx.doi.org/10.1109/TCAD.2010.2049053 Google ScholarDigital Library
- G. Beltrame and G. Nicolescu. 2011. A multi-objective decision-theoretic exploration algorithm for platform-based design. In Proceedings of the Design, Automation and Test in Europe (DATE).Google Scholar
- Carlos A. Coello Coello. 2000. An updated survey of GA-based multiobjective optimization techniques. ACM Comput. Surv. 32, 2, 109--143. DOI: http://dx.doi.org/10.1145/358923.358929 Google ScholarDigital Library
- Carlos A. Coello Coello. 1999. A comprehensive survey of evolutionary-based multiobjective optimization techniques. Knowl. Inform. Syst. 1, 3, 269--308.Google ScholarDigital Library
- Piotr Czyzak and Adrezej Jaszkiewicz. 1998. Pareto simulated annealinga metaheuristic technique for multiple-objective combinatorial optimization. J. Multi-Criteria Decision Anal. 7, 1, 34--47. DOI: http://dx.doi.org/10.1002/(SICI)1099-1360(199801)7:1<34::AID-MCDA161>3.0.CO;2-6Google ScholarCross Ref
- Kalyanmoy Deb and Tushar Goel. 2001. Controlled elitist non-dominated sorting genetic algorithms for better convergence. In Evolutionary Multi-Criterion Optimization, Eckart Zitzler, Lothar Thiele, Kalyanmoy Deb, Carlos Artemio Coello Coello, and David Corne (Eds.), Lecture Notes in Computer Science, Vol. 1993, Springer Berlin, 67--81. DOI: http://dx.doi.org/10.1007/3-540-44719-9\_5 Google ScholarDigital Library
- Cagkan Erbas. 2006. System-Level Modelling and Design Space Exploration for Multiprocessor Embedded System-on-Chip Architectures. Amsterdam University Press.Google Scholar
- Carlos M. Fonseca and Peter J. Fleming. 1995. An overview of evolutionary algorithms in multiobjective optimization. Evol. Comput. 3, 1, 1--16. DOI: http://dx.doi.org/10.1162/evco.1995.3.1.1 Google ScholarDigital Library
- William Fornaciari, Donatella Sciuto, Cristina Silvano, and Vittorio Zaccaria. 2002. A sensitivity-based design space exploration methodology for embedded systems. Des. Autom. Embed. Syst. 7, 1, 7--33. DOI: http://dx.doi.org/{10.1023/A:1019791213967}Google ScholarDigital Library
- T. Givargis, F. Vahid, and J. Henkel. 2001. System-level exploration for Pareto-optimal configurations in parameterized systems-on-a-chip. In Proceedings of the IEEE/ACM International Conference on Computer Aided Design (ICCAD'01). 25--30. DOI: http://dx.doi.org/{10.1109/ICCAD.2001.968593} Google ScholarDigital Library
- M. D. Hill and M. R. Marty. 2008. Amdahl's Law in the multicore era. Computer 41, 7, 33--38. Google ScholarDigital Library
- H. Ishibuchi and T. Murata. 1996. Multi-objective genetic local search algorithm. In Proceedings of the IEEE International Conference on Evolutionary Computation. 119--124. DOI: http://dx.doi.org/10.1109/ICEC.1996.542345Google ScholarCross Ref
- H. Ishibuchi and T. Murata. 1998. A multi-objective genetic local search algorithm and its application to flowshop scheduling. IEEE Tran. Syst. Man Cybernet. Part C: Appli. Rev. 28, 3, 392--403. DOI: http://dx.doi.org/10.1109/5326.704576 Google ScholarDigital Library
- Stanley Jaddoe and Andy D Pimentel. 2008. Signature-based calibration of analytical system-level performance models. In Proceedings of the 8th International Workshop on Embedded Computer Systems: Architectures, Modeling, and Simulation (SAMOS'08). Lecture Notes in Computer Science, Vol. 5114, Springer-Verlag, Berlin, 268--278. Google ScholarDigital Library
- Andrzej Jaszkiewicz. 2004. A comparative study of multiple-objective metaheuristics on the bi-objective set covering problem and the pareto memetic algorithm. Ann. Oper. Res. 131, 1--4, 135--158. DOI: http://dx.doi.org/10.1023/B:ANOR.0000039516.50069.5bGoogle ScholarCross Ref
- Andrzej Jaszkiewicz and Grzegorz Dbrowski. 2005. MOMH: Multiple objective meta heuristics. http://home.gna.org/momh/.Google Scholar
- Leslie Pack Kaelbling, Michael L. Littman, and Andrew P. Moore. 1996. Reinforcement learning: A survey. J. Artif. Intell. Res. 4, 237--285. http://citeseer.ist.psu.edu/kaelbling96reinforcement.html. Google ScholarCross Ref
- J. Kennedy and R. Eberhart. 1995. Particle swarm optimization. In Proceedings of the IEEE International Conference on Neural Networks, Vol. 4. 1942--1948. DOI: http://dx.doi.org/10.1109/ICNN.1995.488968Google Scholar
- Martin Lukasiewycz, Michael Glay, Christian Haubelt, and Jurgen Teich. 2008. Efficient symbolic multi-objective design space exploration. In Proceedings of the Asia and South Pacific Design Automation Conference (ASP-DAC'08). IEEE Computer Society Press, 691--696. Google ScholarDigital Library
- Giovanni Mariani, Aleksandar Brankovic, Gianluca Palermo, Jovana Jovic, Vittorio Zaccaria, and Cristina Silvano. 2010. A correlation-based design space exploration methodology for multi-processor systems-on-chip. In Proceedings of the 47th ACM/IEEE Design Automation Conference (DAC). 120--125. Google ScholarDigital Library
- R. T. Marler and J. S. Arora. 2004. Survey of multi-objective optimization methods for engineering. Struct. Multidisciplinary Optim. 26, 6, 369--395. DOI: http://dx.doi.org/10.1007/s00158-003-0368-6Google ScholarCross Ref
- G. Martin. 2006. Overview of the MPSoC design challenge. In Proceedings of the 43rd ACM/IEEE Design Automation Conference. 274--279. DOI: http://dx.doi.org/{10.1109/DAC.2006.229245} Google ScholarDigital Library
- Kaisa Miettinen and Marko M. Mkel. 2002. On scalarizing functions in multiobjective optimization. OR Spectrum 24, 2, 193--213. DOI: http://dx.doi.org/10.1007/s00291-001-0092-9Google ScholarCross Ref
- S. Mohanty, V. K. Prasanna, S. Neema, and J. Davis. 2002. Rapid design space exploration of heterogeneous embedded systems using symbolic search and multi-granular simulation. SIGPLAN Not. 37, 7, 18--27. Google ScholarDigital Library
- T. Okabe, Y. Jin, and B. Sendhoff. 2003. A critical survey of performance indices for multi-objective optimisation. In Proceedings of the Congress on Evolutionary Computation (CEC'03). Vol. 2. 878--885. DOI: http://dx.doi.org/10.1109/CEC.2003.1299759Google Scholar
- G. Palermo, C. Silvano, and V. Zaccaria. 2008. Discrete particle swarm optimization for multi-objective design space exploration. In Proceedings of the 11th EUROMICRO Conference on Digital System Design Architectures, Methods and Tools (DSD'08). 641--644. DOI: http://dx.doi.org/10.1109/DSD.2008.21 Google ScholarDigital Library
- G. Palermo, C. Silvano, and V. Zaccaria. 2009. ReSPIR: A response surface-based pareto iterative refinement for application-specific design space exploration. IEEE Trans. Comput. Aid. Des. Integr. Circuits Syst. 28, 12, 1816--1829. DOI: http://dx.doi.org/10.1109/TCAD.2009.2028681 Google ScholarDigital Library
- Maurizio Palesi and Tony Givargis. 2002. Multi-objective design space exploration using genetic algorithms. In Proceedings of the 10th International Symposium on Hardware/Software Codesign (CODES'02). ACM, 67--72. DOI: http://dx.doi.org/10.1145/774789.774804 Google ScholarDigital Library
- P. G. Paulin and J. P. Knight. 1989. Algorithms for high-level synthesis. IEEE Des. Test Comput. 6, 6, 18--31. Google ScholarDigital Library
- A. D. Pimentel, C. Erbas, and S. Polstra. 2006. A systematic approach to exploring embedded system architectures at multiple abstraction levels. IEEE Trans. Computers 55, 2, 99--112. DOI: http://dx.doi.org/{10.1109/TC.2006.16} Google ScholarDigital Library
- Stuart J. Russell and Peter Norvig. 1995. Artificial Intelligence: A Modern Approach 1st Ed. Prentice Hall. Google ScholarDigital Library
- Paolo Serafini. 1994. Simulated annealing for multi objective optimization problems. In Multiple Criteria Decision Making, G. H. Tzeng, H. F. Wang, U. P. Wen, and P. L. Yu (Eds.), Springer, New York, 283--292. DOI: http://dx.doi.org/10.1007/978-1-4612-2666-6\_29Google Scholar
- D. Sheldon, F. Vahid, and S. Lonardi. 2007. Soft-core processor customization using the design of experiments paradigm. In Proceedings of the DATE Conference. 1--6. Google ScholarDigital Library
- S. N. Sivanandam and S. N. Deepa. 2007. Introduction to Genetic Algorithms 1st Ed. Springer, Inc. Google ScholarDigital Library
- N. Srinivas and Kalyanmoy Deb. 1994. Muiltiobjective optimization using nondominated sorting in genetic algorithms. Evol. Comput. 2, 3, 221--248. DOI: http://dx.doi.org/10.1162/evco.1994.2.3.221 Google ScholarDigital Library
- Toktam Taghavi and Andy D. Pimentel. 2011. Design metrics and visualization techniques for analyzing the performance of MOEAs in DSE. In Proceedings of the International Conference on Embedded Computer Systems: Architectures Modeling, and Simulation (ICSAMOS). 67--76.Google Scholar
- E. L. Ulungu, J. Teghem, P. H. Fortemps, and D. Tuyttens. 1999. MOSA method: A tool for solving multiobjective combinatorial optimization problems. J. Multi-Criteria Decision Anal. 8, 4, 221--236. DOI: http://dx.doi.org/10.1002/(SICI)1099-1360(199907)8:4{221::AID-MCDA247}3.0.CO;2-OGoogle ScholarCross Ref
- E. L. Ulungu and J. Teghem. 1994. Multi-objective combinatorial optimization problems: A survey. J. Multi-Criteria Decision Anal. 3, 2, 83--104. DOI: http://dx.doi.org/10.1002/mcda.4020030204Google ScholarCross Ref
- Vittorio Zaccaria, Gianluca Palermo, F. Castro, Cristina Silvano, and Giovanni Mariani. 2010. Multicube explorer: An open source framework for design space exploration of chip multi-processors. In Proceedings of the Workshop on Parallel Programming and Run-time Management Techniques for Many-core Architectures.Google Scholar
- E. Zitzler, K. Deb, and L. Thiele. 1999a. Comparison of Multiobjective Evolutionary Algorithms: Empirical Results (Revised Version). TIK Report 70, Computer Engineering and Networks Laboratory (TIK), ETH Zurich.Google Scholar
- E. Zitzler, K. Deb, and L. Thiele. 1999b. Comparison of multiobjective evolutionary algorithms on test functions of different difficulty. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO'99): Bird-of-a-feather Workshop on Multi-criterion Optimization.Google Scholar
- E. Zitzler and L. Thiele. 1999. Multiobjective evolutionary algorithms: A comparative case study and the strength Pareto approach. IEEE Trans. Evol. Comput. 3, 4, 257--271. DOI: http://dx.doi.org/10.1109/4235.797969 Google ScholarDigital Library
Index Terms
- A comparative evaluation of multi-objective exploration algorithms for high-level design
Recommendations
A Multi-objective Genetic Algorithm for Design Space Exploration in High-Level Synthesis
ISVLSI '08: Proceedings of the 2008 IEEE Computer Society Annual Symposium on VLSIThis paper presents a methodology for design space exploration (DSE) in high-level synthesis (HLS), based on a multi-objective genetic algorithm. Since all high-level synthesis sub-tasks are notoriously NP-complete and interdependent and the design ...
Solving multi-objective optimization problems using self-adaptive harmony search algorithms
AbstractIn recent years, there have been many multi-objective evolutionary algorithms proposed to solve multi-objective optimization problems. These evolutionary algorithms generate many solutions for iterations and move to the true Pareto optimal region ...
A comparative study of constrained multi-objective evolutionary algorithms on constrained multi-objective optimization problems
2017 IEEE Congress on Evolutionary Computation (CEC)Solving constrained multi-objective optimization problems is a difficult task, it needs to simultaneously optimize multiple conflicting objectives and a number of constraints. This paper first reviews a number of popular constrained multi-objective ...
Comments