The present paper proposes a dynamic spectrum access policy for multi-hop cognitive radio networks (CRNs), where the transmission in each hop suffers a delay waiting for the communication channel to become available. Recognizing the energy constraints, we assume that each secondary user (SU) in the network is powered by a battery with finite initial energy. We develop an energy-efficient policy for CRNs using the Markov decision process, which searches for spectrum opportunities without a common communication channel and assigns each sensor's decision to every time slot. We first consider a single-sensor scenario. Due to the intermittent activation of the sensor, achieving the optimal sensing schedule requires excessive complexity and is computationally intractable, owing to the fact that the state space of the Markov decision process evolves exponentially with time variance. In order to overcome this difficulty, we propose a state-reduced suboptimal policy by relaxing the constrained state space, i.e., assuming that the electrical energy of a node is infinite, because this state-reduced suboptimal approach can substantially reduce the complexity of decision-making for CRNs. We then analyze the performance of the proposed policy and compare it with the optimal solution. Furthermore, we verify the performance of this spectrum access policy under real conditions in which the electrical energy of a node is finite. The proposed spectrum access policy uses the dynamic information of each channel. We prove that this schedule is a good approximation for the true optimal schedule, which is impractical to obtain. According to our theoretical analysis, the proposed policy has less complexity but comparable performance. It is proved that when the operating time of the CRN is sufficiently long, the data reception rate on the sink node side will converge to the optimal rate with probability 1. Based on the results for the single-sensor scenario, the proposed schedule is extended to a multi-hop CRN. The proposed schedule can achieve synchronization between transmitter and receiver without relying on a common control channel, and also has near-optimal performance. The performance of the proposed spectrum access policy is confirmed through simulation.