In this paper, we study the power allocation problem in a relay assisted multi-band underlay cognitive radio network. Such a network allows unlicensed users (secondary users) to access the spectrum bands under a transmission power constraint. Due to the concave increasing property of logarithm function, it is not always wise for secondary users to expend all the transmission power in one band if their aim is to maximize achievable data rate. In particular, we study a scenario where two secondary users and a half-duplexing relay exist with two available bands. The two users choose different bands for direct data transmission and use the other band for relay transmission. By properly allocating the power on two bands, each user may be able to increase its total achievable data rate while satisfying the power constraint. We formulate the power allocation problem as a non-cooperative game and investigate its Nash equilibria. We prove the power allocation game is a supermodular game and that Nash equilibria exist. We further find the best response function of users and propose a best response update algorithm to solve the corresponding dynamic game. Numerical results show the overall performance in terms of achievable rates is improved through our proposed transmission scheme and power allocation algorithm. Our proposed algorithm also shows satisfactory performance in terms of convergence speed.