We develop a general-purpose algorithm using a Bayesian optimization framework for the efficient refinement of object proposals. While recent research has achieved substantial progress for object localization and related objectives in computer vision, current state-of-the-art object localization procedures are nevertheless encumbered by inefficiency and inaccuracy. We present a novel, computationally efficient method for refining inaccurate bounding-box proposals for a target object using Bayesian optimization. Offline, image features from a convolutional neural network are used to train a model to predict an object proposal's offset distance from a target object. Online, this model is used in a Bayesian active search to improve inaccurate object proposals. In experiments, we compare our approach to a state-of-the-art bounding-box regression method for localization refinement of pedestrian object proposals.