Posterior Tracking Algorithm for Multi-objective Classification Bandits

Abstract

A classification bandit problem is a kind of pure exploration K-armed bandit problems in which a given set of K arms must be judged whether it contains at least L good arms or not with probability at least $1-\delta$ for a given positive integer $L(\le K)$ and $\delta >0$, by drawing as small number of arms as possible, where an arm is good if and only if its expected reward $\mu$ is at least a given threshold $\xi$. To apply algorithms for this problem to more diverse real-world problems, we extend the problem with one dimensional rewards to that with multi-dimensional rewards by defining good arms as arms whose i-th dimensional expected reward $\mu _i$ is at least given threshold $\xi _i$ for all the dimensions i. We also extend P-Tracking algorithm, which is reported to be a best performer for the original one-dimensional-reward problem, to that for our multi-dimensional-reward problem. Our results using numerical simulations demonstrate the superiority of the extended P-Tracking algorithm in sample efficiency compared to extended other existing algorithms.