The Electrocardiogram(ECG) is a wide spread used tool to monitor the health of a human heart. Detecting any abnormalities of heart signal is the primary objective. Researchers have given a great attention to make this detection error- less and to detect the heart beats abnormality as quick as possible. In this paper, we proposed a method to detect heart beats abnormality efficiently. Our proposed structure is quite lightweight requiring less computational power and memory. Furthermore, to reduce class imbalance while increasing accuracy, we preprocessed our data and augmented the lower numbered classes with 6 different operations. For arrhythmia classification, we achieved average accuracy of 97.3%, 98.9% with F1 score of 97.21%, 99.2% & specificity of 99.3%, 98.95% for MIT BIH Arrhythmia database and PTB Diagnostic ECG database respectively, which is higher enough for a lightweight architecture like proposed one.