For the definition of physical activity monitoring with long activity times can be costly and there is a need for efficient computer based algorithms. Smartphone sensors such as accelerometer, magnetometer, and gyroscope for physical activity recognition are used in many researches. In this study, we propose a multi-modal approach to classify the different physical activities at the feature level by fusing electrocardiography (ECG), accelerometer, magnetometer, and gyroscope signals. We use Support Vector Machine (SVM), nearest neighbors, Naive Bayes, Random Tree and Bagging RepTree classifiers as learning algorithms and provide comprehensive empirical results on fusion strategy. Our experimental results on real clinical examples from the MHealth dataset show that the proposed feature-level fusion approach gives an average accuracy of 98.40% using SVM with the highest value in all scenarios. We also observe that when we use the SVM classifier with the gyroscope signal, which we take the highest value as a single modal, it gives an average accuracy of 96.27%. We achieve a significant improvement in comparision with existing studies.