This paper presents a supervised machine learning (ML)-based approach to detect power system transmission lines faults. The measured signals at one end of the protected line were transformed into a time-frequency domain using spectrograms. Statistical features were extracted from spectrograms. Three datasets were considered to find the best classification models: the original dataset that was unbalanced with a complete set of features, a balanced dataset with a complete set of features, and a balanced dataset with a reduced number of features. Synthetic minority class oversampling technique (SMOTE) was used to balance the data to prevent biases toward the majority class. Four feature selection algorithms were applied to reduce the number of features, namely, neighborhood component analysis (NCA), minimum redundancy maximum relevance (mRMR), fit ensemble of learners, and forward sequential feature selection. Decision trees (DT), support vector machines (SVM), k-nearest neighbors (k-NN), ensemble trees classifiers were used to classify faults and non-fault events. The results showed that ensemble bagged trees of the balanced dataset and complete set of features achieved the maximum accuracy and the minimum false-negative rate. The proposed feature selection algorithms showed no improvement in the performance of the classifiers. The best classifier was then tested using an unseen scenario and showed accurate detection of the fault events for the system without inverter-based generators (IBGs). The classifier, however, was not able to detect faults during power swing after integrating IBGs such as PV plant and type-3 Wind farm behind the relay point in the protected line.