In this paper, we propose a novel approach for online handwriting recognition (HR) based on hidden Markov model (HMM). In a conventional HMM-based HR system, the input test sample is recognized by first measuring the log-likelihood score from each class-specific HMM, and then the class with the highest score is assigned as the recognized class. It is observed that, for a given test sample, the difference in log-likelihood scores of top-2 outputs (classes) is often less for faithful classification. The problem intensifies for those scripts that have a large set of similar shape characters such as the Indic script. To address this problem, first, we analyze the HMM states corresponding to the top-2 classes and identify a subset of states that most discriminate the two classes. Afterwards, the final recognition among the two classes is carried out by comparing the log-likelihood scores of these chosen states. Since the proposed methodology focuses only on the most discriminative states of the two classes, therefore it enhances the classification confidence as well as overall recognition accuracy with least added complexity. The proposal is demonstrated for character and limited vocabulary word recognition tasks and evaluated on the locally collected Assamese character and word databases. The experimental results are promising over the conventional HMM-based HR system.