Computer Aided Diagnosis (CAD) systems for breast cancer play a crucial role in minimizing diagnostic errors and time-consuming nature for manual analysis. In pursuit of higher accuracy, numerous studies have introduced novel algorithms to fine-tune various steps within the CAD detection process. The goal is to conduct a comprehensive review of research articles that have utilized techniques related to feature selection and hyperparameter optimization. Additionally, we will undertake a comparative analysis based on the existing literature, thus enhancing our insights into the evolving landscape of these advanced techniques, future directions and the challenges within the context of breast cancer detection using thermography.