Abstract
In solving a multi-objective optimization problem by scalarization techniques, solutions to a scalarized problem are, in general, weakly efficient rather than efficient to the original problem. Thus, it is crucial to understand what condition ensures that all weakly efficient solutions are efficient. In this paper, we give a condition to verify when efficiency coincides with weak efficiency, provided that the free disposal hull of a given set is convex. By using this characterization, we obtain various applications to multi-objective optimization problems under some convex conditions. We also apply the main theorem to the least absolute shrinkage and selection operator (LASSO) and show that for a multi-objective version of LASSO, all weakly efficient solutions are efficient. Numerical simulation demonstrates that this equivalence is helpful to accelerate the hyper-parameter search for LASSO.
Similar content being viewed by others
Notes
It aligns to the fact that R-package glmnet examines 100 hyper-parameters by default.
References
Aeberhard, S., Coomans, D., de Vel, O.: Comparison of classifiers in high dimensional settings. Tech. Rep. 92-02, Dept. of Computer Science and Dept. of Mathematics and Statistics, James Cook University of North Queensland (1992)
Al-Mezel, S.A.R., Al-Solamy, F.R.M., Ansari, Q.H.: Fixed Point Theory, Variational Analysis, and Optimization. CRC Press, Boca Raton (2014)
Benoist, J., Popovici, N.: The structure of the efficient frontier of finite-dimensional completely-shaded sets. J. Math. Anal. Appl. 250(1), 98–117 (2000). https://doi.org/10.1006/jmaa.2000.6960
Buza, K.: Feedback prediction for blogs. In: Spiliopoulou, M., Schmidt-Thieme, B., Janning, R. (eds.) Data Analysis, Machine Learning and Knowledge Discovery. Studies in Classification, Data Analysis, and Knowledge Organization, pp. 145–152. Springer, Berlin (2014). https://doi.org/10.1007/978-3-319-01595-8
Cassotti, M., Ballabio, D., Todeschini, R., Consonni, V.: A similarity-based QSAR model for predicting acute toxicity towards the fathead minnow (Pimephales promelas). SAR QSAR Environ. Res. 26, 217–243 (2015). https://doi.org/10.1080/1062936X.2015.1018938
Coelho, F., Costa, M., Verleysen, M., Braga, A.P.: LASSO multi-objective learning algorithm for feature selection. Soft. Comput. 24, 13209–13217 (2020). https://doi.org/10.1007/s00500-020-04734-w
Cortez, P., Morais, A.: A data mining approach to predict forest fires using meteorological data. In: Neves, J., Santos, M.F., Machado, J. (eds.) Proceedings of the 13th EPIA 2007—Portuguese Conference on Artificial Intelligence, New Trends in Artificial Intelligence, pp. 512–523. APPIA, Guimarães (2007). http://www3.dsi.uminho.pt/pcortez/fires.pdf
Das, I., Dennis, J.E.: Normal-boundary intersection: a new method for generating the Pareto surface in nonlinear multicriteria optimization problems. SIAM J. Optim. 8(3), 631–657 (1998). https://doi.org/10.1137/S1052623496307510
Dua, D., Graff, C.: UCI machine learning repository (2017). http://archive.ics.uci.edu/ml
Ehrgott, M., Nickel, S.: On the number of criteria needed to decide Pareto optimality. Math. Methods Oper. Res. 55(3), 329–345 (2002). https://doi.org/10.1007/s001860200207
Eichfelder, G.: Adaptive Scalarization Methods in Multiobjective Optimization. Springer, Berlin (2008)
Friedman, J., Hastie, T., Tibshirani, R.: Sparse inverse covariance estimation with the graphical LASSO. Biostatistics 9(3), 432–441 (2007). https://doi.org/10.1093/biostatistics/kxm045
Gil, D., Girela, J.L., De Juan, J., Gomez-Torres, M.J., Johnsson, M.: Predicting seminal quality with artificial intelligence methods. Expert Syst. Appl. 39(16), 12564–12573 (2012). https://doi.org/10.1016/j.eswa.2012.05.028
Graf, F., Kriegel, H.P., Schubert, M., Pölsterl, S., Cavallaro, A.: 2D image registration in CT images using radial image descriptors. In: Fichtinger, G., Martel, A., Peters, T. (eds.) Medical Image Computing and Computer-Assisted Intervention—MICCAI 2011, pp. 607–614. Springer, Berlin (2011)
Hebiri, M., van de Geer, S.: The smooth-lasso and other \(\ell _1 + \ell _2\)-penalized methods. Electron. J. Stat. 5, 1184–1226 (2011). https://doi.org/10.1214/11-EJS638
Kobayashi, K., Hamada, N., Sannai, A., Tanaka, A., Bannai, K., Sugiyama, M.: Bézier simplex fitting: describing Pareto fronts of simplicial problems with small samples in multi-objective optimization. In: Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence, AAAI-19, vol. 33, pp. 2304–2313. AAAI Press (2019). https://doi.org/10.1609/aaai.v33i01.33012304
La Torre, D., Popovici, N.: Arcwise cone-quasiconvex multicriteria optimization. Oper. Res. Lett. 38(2), 143–146 (2010). https://doi.org/10.1016/j.orl.2009.11.003
Lowe, T., Thisse, J.F., Ward, J., Wendell, R.: On efficient solutions to multiple objective mathematical programs. Manag. Sci. 30(11), 1346–1349 (1984). https://doi.org/10.1287/mnsc.30.11.1346
Luc, D.: Theory of Vector Optimization. Lecture Notes in Economics and Mathematical Systems, vol. 319. Springer, Berlin (1989)
Malivert, C., Boissard, N.: Structure of efficient sets for strictly quasi convex objectives. J. Convex Anal. 1, 143–150 (1994)
Matous̆ek, J.: Lectures on Discrete Geometry, Graduate Texts in Mathematics, vol. 212. Springer, New York (2002)
Messac, A., Mattson, C.A.: Normal constraint method with guarantee of even representation of complete Pareto frontier. AIAA J. 42(10), 2101–2111 (2004)
Miettinen, K.: Nonlinear Multiobjective Optimization, International Series in Operations Research & Management Science, vol. 12. Springer, Berlin (1999)
Nesterov, Y.: Introductory Lectures on Convex Optimization: A Basic Course. Kluwer Academic Publishers (2004)
Ortigosa, I., Lopez, R., Garcia, J.: A neural networks approach to residuary resistance of sailing yachts prediction. In: Proceedings of the International Conference on Marine Engineering MARINE 2007 (2007)
Pascoletti, A., Serafini, P.: Scalarizing vector optimization problems. J. Optim. Theory Appl. 42, 499–524 (1984). https://doi.org/10.1007/BF00934564
Popovici, N.: Pareto reducible multicriteria optimization problems. Optimization 54(3), 253–263 (2005). https://doi.org/10.1080/02331930500096213
Popovici, N.: Structure of efficient sets in lexicographic quasiconvex multicriteria optimization. Oper. Res. Lett. 34(2), 142–148 (2006). https://doi.org/10.1016/j.orl.2005.03.003
Popovici, N.: Involving the Helly number in Pareto reducibility. Oper. Res. Lett. 36(2), 173–176 (2008)
Rafiei, M.H., Adeli, H.: A novel machine learning model for estimation of sale prices of real estate units. J. Constr. Eng. Manag. 142(2), 04015066 (2016). https://doi.org/10.1061/(ASCE)CO.1943-7862.0001047
Sato, H.: Inverted PBI in MOEA/D and its impact on the search performance on multi and many-objective optimization. In: Proceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation, GECCO ’14, pp. 645–652. ACM, New York (2014). https://doi.org/10.1145/2576768.2598297
Tanaka, A., Sannai, A., Kobayashi, K., Hamada, N.: Asymptotic risk of Bézier simplex fitting. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 2416–2424 (2020). https://doi.org/10.1609/aaai.v34i03.5622. https://ojs.aaai.org/index.php/AAAI/article/view/5622
Tibshirani, R.: Regression shrinkage and selection via the LASSO. J. R. Stat. Soc. Ser. B (Methodol.) 58(1), 267–288 (1996)
Tibshirani, R., Saunders, M., Rosset, S., Zhu, J., Knight, K.: Sparsity and smoothness via the fused LASSO. J. R. Stat. Soc. Ser. B (Stat. Methodol.) 67(1), 91–108 (2005). https://doi.org/10.1111/j.1467-9868.2005.00490.x
Tibshirani, R.J.: The LASSO problem and uniqueness. Electron. J. Stat. 7, 1456–1490 (2013)
Ward, J.: Structure of efficient sets for convex objectives. Math. Oper. Res. 14(2), 249–257 (1989)
Yuan, M., Lin, Y.: Model selection and estimation in regression with grouped variables. J. R. Stat. Soc. Ser. B (Stat. Methodol.) 68(1), 49–67 (2006). https://doi.org/10.1111/j.1467-9868.2005.00532.x
Zhang, Q., Li, H.: MOEA/D: a multiobjective evolutionary algorithm based on decomposition. IEEE Trans. Evol. Comput. 11(6), 712–731 (2007). https://doi.org/10.1109/TEVC.2007.892759
Zou, H., Hastie, T.: Regularization and variable selection via the elastic net. J. R. Stat. Soc. Ser. B (Stat. Methodol.) 67(2), 301–320 (2005)
Acknowledgements
The authors are grateful to the reviewers for their comments and suggestions. We are also grateful to Kenta Hayano, Yutaro Kabata, and Hiroshi Teramoto for their kind comments. Shunsuke Ichiki was supported by JSPS KAKENHI Grant Numbers JP21K13786, JP19J00650 and JP17H06128. This work is based on the discussions at 2018 IMI Joint Use Research Program, Short-term Joint Research “multi-objective optimization and singularity theory: Classification of Pareto point singularities” in Kyushu University.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Alfredo N. Iusem.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Hamada, N., Ichiki, S. Free Disposal Hull Condition to Verify When Efficiency Coincides with Weak Efficiency. J Optim Theory Appl 192, 248–270 (2022). https://doi.org/10.1007/s10957-021-01961-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10957-021-01961-5