Statistical appearance models are valuable tools in medical image segmentation. Current methods elegantly incorporate global shape and appearance, but cannot cope with local appearance variations and rely on an assumption of Gaussian gray value distribution. Furthermore, initialization near the optimal solution is required. We propose a shape inference method that is based on pixel classification, so that local and non-linear intensity variations are dealt with naturally, while a global shape model ensures a consistent segmentation. Optimization by stochastic sampling removes the need for accurate initialization. The method is demonstrated on vertebra segmentation in spine radiographs. Segmentation errors are below 2 mm in 88 out of 91 cases, with an average error of 1.4 mm.