Tracking planar patches is a subject of interest in computer vision. We present a method to estimate, in real-time, the position and orientation of a previously viewed planar patch. The algorithm is based on minimising the sum of squared differences (SSD) between a selected set of pixels obtained from a previously stored image of the patch and the current image of it. First a linear model for projectively tracking a planar patch is introduced; then, a procedure to accelerate the tracking process is presented. This procedure is based on using a small set of informative points in the tracking process. In experiments, we show the gain in performance and compare different procedures for selecting the set of points used in tracking.