This paper presents a new method for categorical classification. A method called two-dimensional continuous dynamic programming (2DCDP) is adopted to optimally capture the corresponding pixels within nonlinearly matched areas in an input image and a reference image representing an object without advance segmentation procedure. Then an image can be converted into a direction pattern which is made by matching pixels between a reference image and an input image. Finally, the category of the test image is deemed to be that which has the strongest correlation with the learning images. Experimental results show that the proposed method achieves a competitive performance on the Caltech 101 image dataset.