Benchmarking algorithm portfolio construction methods

A portfolio is a set of algorithms, which run concurrently or interchangeably, whose aim is to improve performance by avoiding a bad selection of a single algorithm. Despite its high error tolerance, a carefully constructed portfolio, i.e., the smallest set of complementary algorithms, is expected to perform better than an arbitrarily constructed one. In this paper, we benchmark five algorithm portfolio construction methods, using as benchmark problems the ASLib scenarios, under a cross-validation regime. We examine the performance of each portfolio in terms of its riskiness, i.e., the existence of unsolved problems on the test set, and its robustness, i.e., the existence of an algorithm that solves most instances. The results demonstrate that two of these methods produce portfolios with the lowest risk, albeit with different levels of robustness.