In this paper, a pose-invariant hand shape recognition method based on the geometry of the fingers is proposed. Firstly, inspired by the segmentation method presented by Yoruk et al., we conduct a novel improvement on the segmentation for extracting the region of the fingers when the hand is in a natural pose. Secondly, Fourier descriptors and finger area functions are employed to extract the finger boundary curve features and region areas, respectively. Finally, score-level fusion based on a weighted sum is used to obtain matching results. Because the finger segmentation strategy and the feature extraction method are both rotation and translation invariant, the proposed method is more suitable for a naturally posed hand. Experiments using the Bogazici University Hand database show that the proposed method can achieve an equal error rate of 0.0369 for all data and 0.0273 for samples with an intragroup angle deviation of less than 45°. Thus, the proposed method is suitable for real-world applications.