Daily cardiac health monitoring is of high importance for effective heart disease prediction/management. In this study, we propose a novel ear-worn system for long-term continuous ECG QRS duration tracking, to overcome challenges of current wearable ECG systems such as the uncomfortableness and inconvenience. Specifically, we place all the ECG electrodes behind the ear to enhance the wearability, and weak/noisy ear-ECG is obtained. Then, we use a support vector machine classifier for heartbeat identification, apply an unsupervised learning approach for heartbeat purification, and a regression model to derive the standard chest-ECG QRS durations from the ear-ECG QRS durations. We have evaluated the proof-of-concept system using an ear-ECG dataset acquired by a semi-customized wearable prototype, and demonstrated the effectiveness of the proposed system. To the best of our knowledge, it is the first study on an ear-worn system for ECG QRS duration estimation, which can be used in daily cardiac health monitoring applications.