A novel quantum-inspired reinforcement learning (QiRL) algorithm is proposed for decision-making of Markovian state transition. The QiRL algorithm adopts a probabilistic action selection policy to better balance the tradeoff between exploration and exploitation, which is inspired by the collapse phenomenon in quantum measurement. Several simulated experiments of Markovian state transition demonstrate that QiRL is more robust to learning rates and initial states than traditional reinforcement learning. The QiRL approach provides an effective method for complex decision-making problems.