We formulate mixed integer programming (MIP) models to obtain approximate solutions to finite horizon stochastic inventory models. These deterministic formulations of necessity make a number of simplifying assumptions, but their special structure permits very short model solution times under a range of experimental conditions. We evaluate the performance of these models using simulation optimization to estimate the true optimal solutions. Computational experiments identify several demand and cost scenarios in which the MIP models yield near-optimal solutions, and other cases where they fail, suggesting directions for future research.