In this paper, we have proposed a method to measure the weight efficiency of pointwise convolution (weight sharing index), which finds that the weight efficiency of pointwise convolution decreases rapidly as the network becomes deeper. The redundancy of the feature maps is mainly reflected in the similarity of the feature maps. We achieve the compression of the pointwise convolution layers by quantifying the similar feature maps with the help of structural similarity (SSIM), reducing the input channels of the pointwise convolution by fusing the similar feature maps. The results on the datasets CIFAR-10 and CIFAR-100 demonstrate that the compression method based on fusing similar feature maps can significantly compress the MobileNet-V1(48.74% FLOPs and 13.64% parameters are reduced) with an increase of 0.54% in the top-1 accuracy, improving the top-1 accuracy of ResNet-50 by 0.09% (9.2% of FLOPs and 6.03% of parameters are reduced), and improves the efficiency of the weight utilization of the pointwise convolution layer.