Machine learning techniques are widely used for discovering meaningful patterns and classifying real-world data. These datasets may be large and complex, so feature selection is the primary strategy for reducing the dimension of the data, with the general goal of reducing the amount of redundant and disruptive features in a dataset for fast and efficient data analysis without sacrificing significant predictive model performance. Due to exponentially high search space, feature selection is a complex optimization problem. It is practically impossible to evaluate all of the feature subsets manually. Nature-inspired optimization is widely used for this due to its inherent capability, and it solves feature selection tasks as a single objective optimization problem. However, the main issue is their frequent premature convergence, which results in an inadequate contribution to data mining. Even the majority of existing optimizers are not adaptive in nature. As a result, in this paper, we proposed QL-SSA, which combines Reinforcement Learning and the Squirrel Search Algorithm, making it more adaptive and robust for feature selection by maintaining a good balance between exploration and exploitation steps. It is tested on 20 real-world benchmark datasets using two classifiers, and the results show that it outperforms the baseline optimizer in most cases.