Imbalanced data classification, which is a common and important problem in various fields related to the detection of anomaly, failure, and risk, has been studied intensively. Conventional methods are based on sampling, misclassification costs, or ensemble of classifiers, and many of them are heuristic and task dependent. Aiming at a higher classification performance with the solution of such problems, we propose a confusion-matrix-based kernel logistic regression (CM-KLOGR). After pretraining with a cross-entropy error function, CM-KLOGR retrains its parameters using the harmonic mean of the various evaluation criteria, including sensitivity, positive predictive value, and others, which are derived from a confusion matrix. CM-KLOGR inherits the advantages of kernel logistic regression, and it has the potential to raise the values of all the evaluation criteria in a well-balanced way. This paper presents the concept and formulation of CM-KLOGR, accompanied by the results of an exploratory experiment using an imbalanced biomedical dataset.