We study the adaptive control problems of a class of discrete-time partially observed Markov decision processes whose transition kernels are parameterized by a unknown vector. Given a sequence of parameter estimates converging to the true value with probability 1, we propose an adaptive control policy and show that under some conditions this policy is self-optimizing in the long-run average sense.