Loading [a11y]/accessibility-menu.js
Simulation selection for empirical model comparison | IEEE Conference Publication | IEEE Xplore

Simulation selection for empirical model comparison


Abstract:

We propose an efficient statistical method for the empirical model comparison, which is typically referred to as a simulation procedure to evaluate multiple statistical l...Show More

Abstract:

We propose an efficient statistical method for the empirical model comparison, which is typically referred to as a simulation procedure to evaluate multiple statistical learning algorithms. First, we use experimental designs to appropriately construct the training and test sets for estimating the empirical performances of these models using the mean square errors. Second, we apply the idea of Bayesian fully sequential ranking and selection to optimally allocate the simulation budget according to the value of information. To make the procedure computationally tractable, we assume a normal-Wishart prior distribution, and propose a new approximation scheme for the posterior distribution by matching it with a normal-Wishart distribution using the first-order moment. Numerical experiments are conducted to show the superiority of the proposed approach on empirical model comparison problems.
Date of Conference: 06-09 December 2015
Date Added to IEEE Xplore: 18 February 2016
ISBN Information:
Electronic ISSN: 1558-4305
Conference Location: Huntington Beach, CA, USA