Recently, backdoor attacks have become a serious security threat to Deep Neural Networks (DNNs). Backdoor attacks involve embedding a hidden backdoor into a DNN model, compelling it to correctly classify benign images while erroneously classifying images with backdoor triggers as the target label. However, both current backdoor attacks and defenses have their limitations. In backdoor attacks, they are either non-stealthy or vulnerable to well-designed backdoor defense strategies. As for backdoor defenses, they often rely heavily on additional assumptions (such as determined extra clean images) and are not universally applicable, which may become impractical in the face of the latest backdoor attacks. To address the above problems, in this paper, we investigate the backdoor attack and defense strategies from a multi-channel perspective. Specifically, in terms of attacks, we propose a recolorization based attack method (RC-Attack) to generate triggers in color ab channels, which is more stealthy and effective. In terms of defenses, we propose a reconstruction-based defense method (RC-Defense) to reconstruct the color AB channels and lightness channel respectively, thus making the triggers in the reconstructed images ineffective, which is a more practical solution. Extensive experiments are conducted to demonstrate the superior performance of the proposed RC-Attack in terms of effectiveness, stealthiness and defense-resistance, and also to validate the effectiveness of the proposed RC-Defense.