Considering the continuance feature of the spectrum idle state observed from our spectrum measurements, in this paper, we propose a novel SKipping bAsed specTrum sEnsing (SKATE) scheme for cognitive radio (CR) transmissions. Instead of sensing at each time slot, we skip the sensing process in some following slots if a band is captured in an idle state, so that the sensing cost could be reduced significantly. For the STAKE scheme, how to determine the number of slots to skip is the key, where more slots could bring more benefits, but also with a higher probability to cause collisions. Since the perfect knowledge on spectrum environment might be hardly obtainable in practice, to enable the device to predict the spectrum state effectively and achieve the optimal joint decision on both band selection and skipped slots, we develop a reinforcement learning (RL) based solution for the SKATE scheme to make it learn the optimal strategy asymptotically. Both real and simulated spectrum data are adopted to demonstrate the effectiveness of the proposed RL based SKATE scheme.