The introduction of cognitive radio enables systems awareness of their environment and internal state, fulfills the need of dynamic spectrum access for higher spectrum utilization. However, at the same time, the use of cognitive radios further complicates the security problems in wireless networks and introduces additional challenges for a counter measure. Due to the intelligence of the attackers, many of the attacks may be stealthy by nature. In this paper, we propose a cross-layer model for stealthy attacks detection with stealthy jammer as a prime example. Quickest detection technique is adopted and embedded in the proposed framework since the attacks usually happen at unknown time and are unpredictable due to the lack of prior knowledge of the attackers. A case study is performed by combining physical layer spectrum sensing with multipath routing at the network layer. The results demonstrate the effectiveness of the proposed approach.