Feature selection is a technique of selecting a subset of relevant features for building robust learning models. In this paper, we developed a data dimensionality reduction approach using sparsified singular value decomposition (SSVD) technique to identify and remove trivial features before applying any advanced feature selection algorithm. First, we investigated how SSVD can be used to identify and remove nonessential features in order to facilitate feature selection performance. Second, we analyzed the application limitations and computing complexity. Next, a set of experiments were conducted and the empirical results show that applying feature selection techniques on the data of which the nonessential features are removed by the data dimensionality reduction approach generally results in better performance with significantly reduced computing time.