We investigate the opportunistic scheduling problem where a server opportunistically serves multiple classes of users under time varying multi-state Markovian channels. The aim of the server is to find an optimal policy minimizing the average waiting cost of users. Mathematically, the problem can be cast to a restless bandit one, and a pivot to solve restless bandit by index policy is to establish indexability. We mathematically propose a set of sufficient conditions on channel state transition matrix, and consequently, the index policy is feasible. Our work consists of a small step toward solving the opportunistic scheduling problem in its generic form involving multi- state Markovian channels and multi-class users.