In the multiuser cooperative communication, a core problem is how to efficiently find an appropriate cooperative relay node with lower power consumption and minimal payment when the number of relay nodes is large. This is referred to as optimal relay selection problem, which characterize the desired tradeoff between the probing cost for establishing cooperative transmission and the reward via the cooperative diversity. In this paper, we establish optimal strategies for the homogenous relay node and heterogenous relay node by using optimal stopping theory with a finite horizon. Furthermore, for the homogenous and heterogenous case, the optimal strategies exhibit a pure SNR threshold structure and reward threshold structure, respectively. Therefore, they are simple to be implemented. Theoretical analysis and numerical results indicate that the proposed optimal relay selection strategies can reduce the number of probing candidate relays and improve the transmission rate with lower energy consumption and minimal payment.