This paper considers the grant-free random access scheme in IoT networks with a massive number of devices. By embedding the data symbols in the signature sequences, joint device activity detection, and data decoding can be achieved, which, however, significantly increases the computational complexity. Coordinate descent algorithms, with a low per-iteration complexity, have been employed to solve the detection problem, but previous works typically employ a random coordinate selection policy which leads to slow convergence. This paper develops a bandit based strategy, i.e., bandit sampling, to speed up the convergence of coordinate descent. We exploit a multi-armed bandit algorithm to learn which coordinates will result in more aggressive descent of the objective function. Both convergence rate analysis and simulation results are provided to show that the proposed algorithm enjoys a faster convergence rate with a lower time complexity compared with the state-of-the-art algorithm.