Cognitive radio (CR) has emerged as a solution to the problem of spectrum scarcity as it exploits transmission opportunities in the under-utilized spectrum bands of primary users. Collaborative (or distributed) spectrum sensing has been shown to have various advantages in terms of spectrum utilization and robustness. The data fusion scheme is a key component of collaborative spectrum sensing. In this paper, we analyze the performance limits of collaborative spectrum sensing under Byzantine Attacks where malicious users send false sensing data to the fusion center leading to increased probability of incorrect sensing results. We show that above a certain fraction of Byzantine attackers in the CR network, data fusion scheme becomes completely incapable and no reputation based fusion scheme can achieve any performance gain. We present optimal attacking strategies for given attacking resources and also analyze the possible counter measures at the fusion center (FC). Based on these analyses, we also propose a novel and easy to implement technique to counter Byzantine attacks in CRNs. In this approach, the FC identifies the attackers and removes them from the data fusion process. Our analysis indicates that the proposed scheme is robust against Byzantine attacks and can successfully remove the Byzantines in a short time span.