Abstract
Akaike’s information criterion (AIC) is a measure of the quality of a statistical model for a given set of data. We can determine the best statistical model for a particular data set by the minimization based on the AIC. Since it is difficult to find the best statistical model from a set of candidates by this minimization in practice, stepwise methods, which are local search algorithms, are commonly used to find a better statistical model though it may not be the best.
We formulate this AIC minimization as a mixed integer nonlinear programming problem and propose a method to find the best statistical model. In particular, we propose ways to find lower and upper bounds and a branching rule for this minimization. We then combine them with SCIP, which is a mathematical optimization software and a branch-and-bound framework. We show that the proposed method can provide the best statistical model based on AIC for small-sized or medium-sized benchmark data sets in UCI Machine Learning Repository. Furthermore, we show that this method can find good quality solutions for large-sized benchmark data sets.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Achterberg, T.: SCIP solving constraint integer programs. Math. Prog. Comp. 1(1), 1–41 (2009)
Akaike, H.: A new look at the statistical model identification. IEEE Trans. Autom. Control 19(6), 716–723 (1974)
Bertsimas, D., King, A., Mazumder, R.: Best subset selection via a modern optimization lens. Ann. Stat. 44(2), 813–852 (2016)
Bertsimas, D., Shioda, R.: Algorithm for cardinality-constrained quadratic optimization. Comput. Optim. Appl. 43, 1–22 (2009)
Guyon, I., Elisseeff, A.: An introduction to variable and feature selection. J. Mach. Learn. Res. 3, 1157–1182 (2003)
IBM ILOG CPLEX Optimizer 12.6.2, IBM ILOG (2015)
Miyashiro, R., Takano, Y.: Mixed integer second-order cone programming formulations for variable selection. Eur. J. Oper. Res. 247, 721–731 (2015)
SCIP: Solving Constraint Integer Programs. http://scip.zib.de/
Shinano, Y., Achterberg, T., Berthold, T., Heinz, S., Koch, T.: ParaSCIP: A parallel extension of SCIP. In: Bischof, C., Hegering, H.-G., Nagel, W.E., Wittum, G. (eds.) Competence in High Performance Computing 2010, pp. 135–148. Springer, Heidelberg (2012)
UCI Machine Learning Repository. http://archive.ics.uci.edu/ml/
Vigerske, S., Gleixner, A.: SCIP: Global Optimization of Mixed-Integer Nonlinear Programs in a Branch-and-Cut Framework. ZIB-Report 16–24, Zuse Institute Berlin, May 2016
Acknowledgements
The second author was supported by JSPS KAKENHI Grant Numbers 26400203. We would like to thank the anonymous referees for providing significant comments on the presentation of the manuscript.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Kimura, K., Waki, H. (2016). Mixed Integer Nonlinear Program for Minimization of Akaike’s Information Criterion. In: Greuel, GM., Koch, T., Paule, P., Sommese, A. (eds) Mathematical Software – ICMS 2016. ICMS 2016. Lecture Notes in Computer Science(), vol 9725. Springer, Cham. https://doi.org/10.1007/978-3-319-42432-3_36
Download citation
DOI: https://doi.org/10.1007/978-3-319-42432-3_36
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-42431-6
Online ISBN: 978-3-319-42432-3
eBook Packages: Computer ScienceComputer Science (R0)