Conditional random fields (CRF) have been introduced to remote sensing image classification recently to integrate contextual information into remote sensing classification. It employs the spatial property on both pixel's spectral data and labels. However, this leads to a large number of model parameters to train. In this letter, the training efficiency is improved by modifying the conventional CRF model. At the same time, a class boundary constraint is imposed into this framework to avoid over correction. The advantages of the developed method are demonstrated in the experimental results using real remotely sensed data.