Dynamic Neural Networks for Adaptive Implicit Image Compression

Abstract

Compression with Implicit Neural Presentations (COIN) is a neural network image compression method based on multilayer perceptron (MLP). COIN encodes an image with an MLP that maps pixel positions to RGB values matching, the weights of the MLP are quantized to obtain a code stored as an image. However, this single implicit network structure performs generally when dealing with images of multiple complexities. In this paper, we propose a novel implicit dynamic neural network to process images in a dynamic and adaptive manner. Specifically, this paper uses the Sobel operator to divide the complexity of the images and use it as a criterion to select the network width and depth adaptively. To better fit the image features, this paper concludes with further quantification of the dynamic network parameters and storage matrices. Therefore, only some of the relevant network parameters with their storage matrices are required when storing the images. In training this dynamic network, this paper uses a meta-learning approach for the multi-image compression task. Experimental results show that our method outperforms COIN and JPEG in terms of image reconstruction results for the CIFAR-10 dataset.

Appendices

A Overall Process Pseudocode

As follows, is the pseudo code of the method in this paper.

Algorithm 1

Employ dynamic neural networks to compress images of different complexities with implicit neural representations

B Class Meta-Learning $M_{\psi }$

Store each image in the dataset explicitly for its $M_{\psi }$ and evaluate each coordinate point when reconstructing the image. Denote the output y as

$$\begin{aligned} y = f_{\theta }\left( x,M_{\psi }, t\right) \end{aligned}$$

(8)

Only $M_{\psi }$ of each image alone needs to be overfitted to each image by the base network. Also, minimize the difference between the reconstruction result of the base dictionary model on the whole dataset and the dataset data.

$$\begin{aligned} L(x,M_\psi ,d)=\sum \limits _{j=1}^{n}||f_\theta (x_i,M_{\psi }, t), y_i||_{2} \end{aligned}$$

(9)

$$\begin{aligned} \min _{\theta ,M_{\psi }}L(x,M_\psi ,d) \end{aligned}$$

(10)

COIN++ [7], a special MAML is applied to generate well-initialized network parameters. While COIN++ emphasizes the generalization of the model to obtain updated network parameters by several gradient descent, in our experiments, multi-task learning is performed to overfit $M_\psi$. It is required to meta-learning a $\theta$ that over-fits the storage matrix $M_{\psi }$ at each new data point. Therefore, in the inner loop, $M_\psi$ is learned in the following way:

$$\begin{aligned} M_{\psi }^{(j)} = M_{\psi }^{(j)} - \alpha \nabla _{M_{\psi }}L\left( \theta , M_{\psi }, d^{(j)}\right) \end{aligned}$$

(11)

In the outer loop, the network parameters $\theta$ are updated using the errors generated for each data point:

$$\begin{aligned} \theta \leftarrow \theta -\beta \nabla _{\theta }\sum _{j=1}^{N}L\left( \theta , M_\psi ^{(j)}, d^{(j)}\right) \end{aligned}$$

(12)

The outer loop base model learns as many dictionary frequencies as possible, while the inner loop performs gradient updates for each image’s storage matrix, guiding the generation mask to correctly select the appropriate frequency in the base dictionary to achieve multi-image compression.

C Experimental Details

1.1 C.1 Verify the Need for Dynamic Networks

Fig. 4.

Verify the dynamic necessity of neural network. Take twenty pictures of each complexity and change the depth or width to get the image reconstruction effect of each complexity picture in a specific network, which is measured by PSNR.

1.2 C.2 Analysis of Experimental Results of Ablation Dynamic Depth

We selected 20 images from each complexity category, maintaining their original width (r) within the category. We then calculated the average image reconstruction effects while increasing the number of network layers.

According to the Figure 5(a), as the complexity of the images gradually increased, we observed that a greater number of network layers were required to achieve a better image reconstruction effect while consuming relatively more resources. For instance, easy images yielded the best effect at 7 layers, medium images at 8 layers, and difficult images required a deeper network with the best effect achieved at 9 layers. This demonstrates that the appropriate network depth varies depending on the complexity of the images. Deeper networks are necessary for more complex images to achieve superior image reconstruction. This further affirms the importance of dynamically adapting the network depth in our approach, allowing for optimal image reconstruction results across images of different complexities.

Table 2. Fixed width and dynamic depth image reconstruction effect table with different complexity

Our analysis confirms the significance of dynamically adjusting the network depth in our approach, achieving optimal image reconstruction results for images of different complexities.

1.3 C.3 Analysis of Experimental Results of Ablation Dynamic Width

Fig. 5.

The results of ablation experiments on dynamic neural networks. During the ablation of dynamic depth, the fixed network width, that is, the fixed R-value, gradually deepens the network depth. Compare the optimal network depth for images of different complexity. When the dynamic width is ablated, the network depth is fixed and the network width is gradually widened, that is, the R-value is increased. Compare the optimal network width for images of different complexity.

For each category, we selected 20 images of each and fixed their optimal depths (verified in previous experiments). Their average image reconstruction effect values were calculated with increasing network width (by adjusting the r value).

According to the Figure 5(b), we can intuitively observe that the best results are achieved with an r value of 2 in medium complexity images and with an r value of 3 in difficult complexity images, both with the same width setting. In the case of simple complexity images, as the r value increases, i.e., the network width becomes wider, the effect of image reconstruction increases slightly and then decreases to the same as the initial one. This is because the simple image requires less frequency composition, while the increase of network width brings more computational consumption, and the optimization effect is not obvious. Therefore the optimal r value for the simple image should be 1, again with the same setting as its width.

Specifically, our set the following r values: r=1 (easy image: 37.8, medium image: 35.2, difficult image: 37.8), r=2 (easy image: 38.03, medium image: 35.88, difficult image: 35.23), r=3 (easy image: 37.8, medium image: 35.3, difficult image: 35.9). From the image reconstruction results with different r values, it can be intuitively seen that the best results are achieved with a r value of 2 in the medium images and with a r value of 3 in the difficult images, which coincides with their width settings. In contrast, in the easy images, as the r value increases (i.e., the network width increases), the image reconstruction effect first slightly improves and then decreases to the same as the initial one. This is because the easy image requires less frequency composition [33], while increasing the network width at the same time adds more computational overhead and is less effective in optimization. Therefore, the optimal r value for simple images should be 1, which is the same as its initial width. And the optimal r values for medium and difficult images are 2 and 3, respectively, and their image reconstruction effects under other r values follow the variation trend.

These results analyzed further validate the necessity of dynamically adapting the network width in our method and being able to adapt to images of different complexity to achieve the best image reconstruction results.