A new model for skin lesion segmentation is proposed in this paper, named multi-scale contextnal attention feature enhancement network (MsCAFE-Net) to segment skin lesions to overcome the challenges of confusing factors, contrast changes, irregular shapes, and other factors. By taking full advantage of the multi-scale context, semantic, and spatial information in an image, MsCAFE-Net can achieve better results. Firstly, a multi-scale attention feature selection module (MAFS) is proposed, which can properly gather the spatial information of different scale features by fusing the multi-scale attention mechanism, and reduce the interference of hybrid factors on model judgment. Secondly, a fine-grained deep noise filtering module (FDNF) is designed to effectively filter the noise in the jump connection by fusing the global contextual semanteme info,to enhance the model's sensitivity to targets with irregular sizes and shapes. Finally, a hierarchical feature enhancement module (HFE) is proposed to achieve finer grained feature enhancement by enhancing the feature expression of low contrast samples. The extensive trial results demonstrated that our approach outperforms other leading methods, with our model attaining an accuracy of 96.95%, a Dice score of 92.28%, and a Jaccard index of 85.68%.