Residual network is a commonly used convolutional neural network, but as the network becomes wider and deeper, it is becoming more difficult for IoT devices or embedded systems to adopt it due to their hardware constraints. Network compression using knowledge distillation is helpful to overcome this problem. In this paper, we propose a tapered compression ratio for residual network compression, which achieves higher accuracy with fewer multiplications on CIFAR datasets.