Dynamic radio resource allocation to network slices in mobile networks is challenging due to the diverse requirements of RAN slices and the dynamic environment of wireless networks. Reinforcement learning (RL) has been successfully applied to solve different network resource allocation problems where an agent learns how to choose the best action from the interactions with the environment. This survey studies the state-of-the-art RL approaches that address radio resource management in radio access network slicing. To this end, we first categorize different problem definitions based on the network environment. Then we explain how each environment can be modeled as a Markov decision process and what RL algorithms can be used to solve them. In addition, we discuss the challenges present in existing works and suggest strategies to address them.