Arrhythmia is a type of heart condition in which the rate or rhythm of the heartbeat is abnormal. Machine learning is increasingly being researched for automated computer-aided ECG diagnosis of arrhythmia detection. Previous works have shown that using Deep CNNs for time series classification has several significant advantages over other methods, since they are highly noise-resistant models, and they can extract very informative, deep features, which are independent of time. However, in using deep learning for arrhythmia detection, the interpretation of how the model learns from the ECG data is limited. In this paper, we propose an extension of CNN-based learning in detecting arrhythmia using recurrence plots from ECG signal data with accuracy within 95.8%, then we conduct the visualization using the Grad-CAM approach on the recurrence plot data to have a better interpretation of the classification process. We summarize our results by drawing comparisons between traditional diagnosis by clinicians and AI-based diagnosis using our classification model.