We propose a two-stage classification framework for image recognition which conjunctively uses parametric and non-parametric approaches. In the first stage, input images are classified using a bag-of-features (BoF) based method with a multi-class classifier. The results are categorized into two groups by our confidence analysis: highly confident and less confident. The images with less confidence are re-classified in the second stage using our inclined local naive bayes nearest neighbor (IL-NBNN). In the original local NBNN, the similarity between the input image and its k-NN classes are calculated aiming at higher discriminability and computational efficiency. Our IL-NBNN virtually calculates the similarity between all the classes efficiently by incorporating the confidence order obtained in the first stage. As a result, efficient and accurate image classification has been achieved with very small extra cost. The experiments using the three image datasets show the validity of our proposed algorithm.