Efficient design of cognitive radios requires secondary users implementing adaptive resource allocation schemes that exploit knowledge of the channel state information (CSI), and limit interference to the primary system. In this paper, stochastic resource allocation algorithms are developed for underlay cognitive radios to maximize the sum-rate of secondary users while adhering to “average power” and “probability of interference” constraints. The latter guarantee that during most of the time the power interfering with primary receivers stays below a certain pre-specified level. Although the resultant optimization problem is non-convex, it exhibits zero-duality gap and can be efficiently solved. The optimal schemes are a function of the link quality of the secondary network, the activity of the primary users, and the Lagrange multipliers associated with the considered constraints. The focus is on developing algorithms that: i) employ stochastic approximation tools to estimate the multipliers; and ii) are able to cope with imperfections present in the CSI of the primary network.