ArchRepair: Block-Level Architecture-Oriented Repairing for Deep Neural Networks

In the standard training process, given a training dataset, we can train a DNN denoted as \(\phi _{(\mathcal {W,}\mathcal {A})}\) where \(\mathcal {A}\) represents the network architecture related parameters determining what operations (e.g., convolution layer, pooling layer) are used in the architecture, and \(\mathcal {W}\) is the respective weights (i.e., parameters of different operations). Generally, the architecture \(\mathcal {A}\) is pre-defined and fixed during the training and testing processes. The variable \(\mathcal {W}\) consists of weights for different layers.

Although existing DNNs (e.g., ResNet [25]) have achieved significantly high accuracy on popular datasets, incorrect behaviors are always found in these models when we deploy them in the real world or test them on challenging datasets. There are a series of works that study how to repair these DNNs to be generalizable to misclassified examples, challenging corruptions, or bias errors [54, 59, 63, 72]. In general, we can formulate the existing repairing methods aswhere \(\mathcal {W}^*\) is a subset of \(\mathcal {W}\) and \(\hat{\mathcal {W}}^*\) is the fixed counterpart of \(\mathcal {W}^*\). The dataset \(\mathcal {D}^\text{repair}\) contains the examples for repairing guidance. Different works may set different \(\mathcal {D}^\text{repair}\) according to the repairing scenarios. For example, Yu et al. [72] sets \(\mathcal {D}^\text{repair}\) as the combination of the augmented training dataset. We will show that our method can address different repairing scenarios. Intuitively, Equation (1) is to find the weights we need to fix in the DNN, and Equation (2) with a task-related objective function \(\text{J}(\cdot)\) is to fix the selected weights \({\mathcal {W}}^*\) and produce a new one \(\hat{\mathcal {W}}^*\).

The above formulation can represent a series of existing repairing methods. For example, when we try to fix all weights of a DNN (i.e., \(\mathcal {W}^*=\mathcal {W}\)) and set the objective function \(\text{J}(\cdot)\) as the task-related loss function (e.g., cross-entropy function for image classification) with different data augmentation techniques on collected failure cases as \(\mathcal {D}^\text{repair}\) to retrain the weights, we actually get the methods proposed by [54] and [72]. In addition, when we employ the gradient loss of weights and forward impact to localize the targeted weights and use a fitness function to fix localized weights, the formulation becomes the method [59].

Nevertheless, with the general formulation in Equations (1) and (2), we can see that existing repairing methods have the following limitations:

Note that, the state-of-the-art DNNs (e.g., ResNet [25]) are often made up of several blocks where each block is built with stacked convolutional and activation layers. Such block-like architecture is mainly inspired by the philosophy of VGG nets [57] and its effectiveness has been demonstrated in wide applications. Therefore in this work, we focus on DNN repairing at the block level. In particular, we consider both the architecture and weights repairing of a specific block.

First, we perform a preliminary experiment to discuss the effectiveness of the repairing methods at different levels. In this experiment, we choose 3 variants of ResNet [25] (specifically, ResNet-18, ResNet-50, and ResNet-101) as the targeted DNNs \(\phi\), and we select CIFAR-10 and Tiny-ImageNet dataset as the experimental environment. We repair the DNN at four levels, i.e., Neuron-level (i.e., only fixing weights of one neuron), Layer-level (i.e., only fixing the weights of one layer), Block-level (i.e., fixing the weights of a block) and the Network-level (i.e., fixing all weights of the DNN). Inspired by recent work [59], we choose the neuron (or layer/block) with the greatest gradient (mean gradient for layer and block) as our target to fix. Note that as the previous works have shown that repairing DNN with only a few failure cases is meaningful and important [54, 72], we only randomly select 100 failure cases from the testing dataset to calculate the gradients and choose such neuron (or layer/block). Then, we adjust the weights of the chosen neuron/layer/block by gradient descent w.r.t. the loss function (e.g., cross-entropy loss for image classification). To compare their effectiveness, we apply all methods on the same training dataset of CIFAR-10 and Tiny-ImageNet, then measure the accuracy on the respective testing dataset. We also record the execution time of the total repairing phase (100 epochs) as the indicator of time cost. We show the repairing result in Table 1. Note that, we repeat each experiment five times and take the average of each result.

According to Table 1, the network-level repairing achieves the highest accuracy on ResNet-18 and ResNet-101 when repairing on CIFAR-10 dataset, and all 3 variants of ResNet when repairing on Tiny-ImageNet dataset, but also leads to the highest time cost under every configuration. Among 3 other levels of repairing methods, the block-level repairing achieves the highest accuracy improvement without having a drastic increment on time cost (i.e., the run-time increment comparing with neuron-level and layer-level is less than 500 seconds on 100 epochs across all 3 ResNets) when repairing on both CIFAR-10 and Tiny-ImageNet.

Overall, the network-level repairing is significantly effective in accuracy improvement but leads to a high time cost. Nevertheless, the block-level repairing achieves impressive accuracy enhancement with much less execution time compared to network-level method (e.g., about \(2\times\) less on ResNet-18), making it a good tradeoff between effectiveness and efficiency. This fact inspires and motivates us to further investigate the block-level repairing method.

Given a deployed DNN \(\phi _{(\mathcal {W},\mathcal {A})}\), the weights and architecture usually consists of several blocks, each of which is built by stacking basic operations, e.g., convolutional layer. Then, we represent the weights and architecture with B blocks, i.e., \(\mathcal {W} = \lbrace \mathcal {W}_{\text{b}}^i\rbrace _{i=1}^{B}\) and \(\mathcal {A} = \lbrace \mathcal {A}_{\text{b}}^i\rbrace _{i=1}^{B}\), where the weights or architecture of each block are made up of one or multiple layers. For example, when we consider the ResNet18 [25], we can say that it has six blocks (See Table 2). The first block contains only one convolution layer with the kernel size of \(7\times 7 \times 64\) and the stride of 2. The second to the fifth blocks have two convolutional layers and the last block contains a fully connected layer and a softmax layer. Then, we can reformulate Equations (1) and (2) for the proposed block-level repairing bywhere Equation (3) is to locate the block (i.e., \((\mathcal {W}_\text{b}^*,\mathcal {A}_\text{b}^*)\)) that should be fixed through the proposed adversarial-aware block localization, and Equation (4) is to repair the localized block by formulating it as a network architecture searching problem. Clearly, compared with the general repairing method (i.e., Equations (1) and (2)), the proposed method focuses on fixing the weights and architecture at the block level. We detail the vulnerable block localization in Section 3.2 and architecture search-based repairing in Section 3.3.

There are two main solutions for vulnerable neuron localization [14, 59]. The first one employs the neuron spectrum analysis during the forward process of DNN on a testing dataset. It calculates the spectrum of all neurons (e.g., activated/non-activated times of neurons for correctly classified examples and activated/non-activated times of neurons for misclassified examples). These attributes are used to measure the suspiciousness of all neurons. The general principle is that a neuron is more suspicious when the neuron is more often activated under the misclassified examples than that under the correctly classified examples [14]. This solution is able to localize the vulnerable neurons accurately but requires a large testing dataset, which is not suitable for the scenario where a few examples are available for repairing. The second solution is to actively localize the vulnerable neurons by performing backpropagation on the misclassified examples and calculating the gradients of neurons w.r.t. the loss function. The neurons with large gradients are responsible for the misclassification [59]. This solution is able to localize the vulnerable neuron with fewer examples but ignores the effects of correctly classified examples. As shown in Figure 1, with different failure examples, the gradients of different convolutional blocks in ResNet18 may have similar values, which demonstrates that the gradient-based localization is not sensitive to the variance of the number of failure examples.

Overall, existing methods mainly focus on localizing vulnerable neurons while ignoring the blocks in DNNs. In addition, they have their respective defects. In this work, we propose a novel localization method that aims at finding the most vulnerable block in the DNN, which can lead to the buggy behavior of a deployed DNN. To take the respective advantages of existing works and avoid their defects, we propose adversarial-aware spectrum analysis to localize the vulnerable block.

After localizing the targeted block, how to break the old architecture’s bottleneck and fix it to become competent in the tasks is another challenge. To this end, we formulate the very first block-level architecture and weights repairing as the network architecture search task. Given a deployed DNN with pre-trained weights and fixed architecture (i.e., \(\phi _{(\mathcal {W},\mathcal {A})}\)), we first relax the targeted block (i.e., \(\phi _{(\mathcal {W}^*_\text{b},\mathcal {A}^*_\text{b})}\)) to a directed acyclic graph like the cell structure in the differentiable architecture search (DARTS) [39], which is composed of an ordered sequence of nodes that are connected by edges. Intuitively, the node corresponds to the deep feature while the edge denotes the operation layer like the convolutional layer. Our goal is to optimize the edges, i.e., to determine which two nodes should be connected and which operation should be selected for that connection. To this end, the key issues are to define the architecture search space and optimization strategy.

Figure 3 summarizes the whole workflow of ArchRepair. Given a deployed DNN, we first employ the proposed vulnerable block localization to determine the block we aim at repairing. Specifically, we use the repair dataset \(\mathcal {D}^\text{repair}\) and the neuron spectrum analysis to calculate the suspiciousness of all neurons, i.e., \(\mathcal {S}=\lbrace s_j\rbrace\). Meanwhile, we use the failure examples in \(\mathcal {D}^\text{repair}\) (i.e., \(\mathcal {D}^\text{repair}_\text{fail}\)) to obtain the gradients of all neurons w.r.t. the loss function (i.e., \(\mathcal {G}=\lbrace g_j\rbrace\)). Then, we use Equation (6) and the gradients \(\mathcal {G}=\lbrace g_j\rbrace\) to reweight \(\mathcal {S}=\lbrace s_j\rbrace\), thus get the suspiciousness scores \(\hat{\mathcal {S}}=\lbrace \hat{s}_j\rbrace\). After that, we can calculate the number of vulnerable neurons through a threshold \(\epsilon\), that is, when the suspiciousness score of a neuron \(\hat{\mathcal {S}}=\lbrace \hat{s}_j\rbrace\) is larger than \(\epsilon\), the neuron is identified as a vulnerable case. Finally, the block with the largest number of vulnerable cases is selected as the targeted block we want to repair.

During the architecture search-based repairing, we reformulate the targeted block as a directed acyclic graph, where the deep features are nodes and operations are edges. Then, we relax each edge as a combination of six operations (i.e., Equation (8)), where the combination weights correspond to the architecture parameters \(\mathcal {A}_\text{b}^*=\lbrace \mathbf {a}_{(i,j)}\rbrace\). We use the dataset \(\mathcal {D}^\text{repair}\) to conduct the architecture and weights optimization via Equations (9) and (10), where the original architecture and weights are inherited and serve as the optimization initialization. Therefore, given the optimized block architecture in the continuous space (i.e., \(\hat{\mathcal {A}}^*_\text{b}\)), we discretize it to the final architecture by preserving the operation with the maximum combination weight and removing other operations. Finally, we use the \(\mathcal {D}^\text{repair}\) to fine-tune the weights by fixing the optimized architecture for the repaired DNN.

To answer the research questions above, we design our evaluation from multiple perspectives listed in the following.

Subject Datasets and Repairing Scenarios. Given a deployed DNN trained on a training dataset \(\mathcal {D}^\text{t}\), we can evaluate it on a testing dataset \(\mathcal {D}^\text{v}\). In the real world, there are a lot of scenes that cannot be covered by \(\mathcal {D}^\text{v}\) and the DNN’s performance may decrease significantly after the DNN is deployed in its operational environment. For example, there are common corruptions (i.e., noise patterns) in the real world that can affect the DNN significantly [26]: Gaussian noise (GN), shot noise (SN), impulse noise (IN), defocus blur (DB), Gaussian blur (GB), motion blur (MB), zoom blur (ZB), snow (SNW), frost (FRO), fog (FOG), brightness (BR), contrast (CTR), elastic transform (ET), pixelate (PIX), and JPEG compression (JPEG).

According to the aftermentioned situations, we consider two repairing scenarios that commonly occur in practice:

We choose CIFAR-10 [33], CIFAR-100 [33], Tiny-ImageNet [36], and ImageNet [11] as the evaluation datasets. They are commonly used datasets in recent DNN repair studies, enabling us to perform comparative studies in a relatively fair way. Each dataset contains its respective training dataset \(\mathcal {D}^\text{t}\) and testing dataset \(\mathcal {D}^\text{v}\). CIFAR-10 contains a total of 60,000 images in 10 categories, in which 50,000 images are for \(\mathcal {D}^\text{t}\) and the other 10,000 are for \(\mathcal {D}^\text{v}\). CIFAR-100 has 100 classes containing 600 images each. There are 500 images in the training dataset \(\mathcal {D}^{\bf t}\) and 100 images in the testing dataset \(\mathcal {D}^{\bf v}\) for each class. Tiny-ImageNet has a training dataset \(\mathcal {D}^\text{t}\) with the size of 100,000 images, and a testing dataset \(\mathcal {D}^\text{v}\) with the size of 10,000 images. ImageNet contains over 14 million images. In our experiment, the training dataset \(\mathcal {D}^\text{t}\) uses 1.3 million images, and the testing dataset \(\mathcal {D}^\text{v}\) uses over 50,000 images. Therefore, we have corrupted testing datasets \(\lbrace \mathcal {D}^\text{c}_i\rbrace\) where \(i=1,2,\dots , 15\) corresponding to the above fifteen corruptions [26].

DNN architectures. We select six different architectures of DNN, i.e., VGGNet-16 [58], ResNet-18, ResNet-50, ResNet-101 [25], DenseNet-121 [27], and EfficientNet-B0 [60]. Given that ArchRepair is a block-based repairing method, the block-like architecture, ResNet, turns out to be a perfect research subject. For a broad comparison, we also choose a non-block-like architecture, DenseNet-121, to examine the repairing capability of ArchRepair.¹ For each architecture, we first pre-train them with the original training dataset \(\mathcal {D}^\text{t}\) (from CIFAR-10, CIFAR-100, Tiny-ImageNet or ImageNet), the model with the highest accuracy in testing dataset \(\mathcal {D}^\text{v}\) (from CIFAR-10, CIFAR-100, Tiny-ImageNet or ImageNet) will be saved as pre-trained model \(\phi _\theta\). As the original ResNet and DenseNet are not designed for CIFAR-10 and Tiny-ImageNet datasets, we use the unofficial architecture code offered by a popular GitHub project,² which has more than 4.1 K stars.

Block definition. We divide each of the six selected DNN architectures into several blocks. For each of ResNet-18, ResNet-50, and ResNet-101, we follow its block structures and divide it into four blocks, as shown in Table 2. For DenseNet-121, we divide it by every two convolutional layers as one block. For VGGNet-16, we manually divide it into six blocks by maxpool layer as Table 2 shows and select five of them as repairing targets (i.e., Block 1~5, Block 6 is used for getting output, so we left it out of repairing). For EfficientNet-B0, we follow its block structures and divide them into seven blocks (see Table 2).

NAS method. We select PC-DARTS [70] as our NAS solution for ArchRepair . While all popular NAS techniques should fit into ArchRepair (e.g., DARTS, SNAS, and BayesNAS), these techniques use more time than PC-DARTS in searching for a better network architecture. Given that DNN repair is a time-sensitive task, we choose PC-DARTS [70] as it is among the fastest NAS methods. Though ArchRepair remains the interface for switching to other NAS techniques when the task cares more about the performance of repaired models than the time cost of repairing.

Hyper-parameters. Regarding the training setup, we employ stochastic gradient descent (SGD) as the optimizer, setting batch size as 128, the initial learning rate as 0.1, and the weight decay as 0.0005. We use the cross-entropy loss as the loss function. The maximum number of epochs is 500, and an early-stop function will terminate the training phase when the validation loss no longer decreases in 10 epochs.

Baselines. To demonstrate the repairing capability of the proposed ArchRepair, we select six SOTA DNN repair methods from two different categories as baselines: neuron-level repairing methods and network-level repairing methods. The neuron-level repairing methods focus on fixing certain neurons’ weights in order to repair the DNNs. Representative methods from this category are MODE [45], Apricot [74], and Arachne [59]. While network-level repairing methods mainly repair DNNs by using augmented datasets to fine-tune the whole network, where SENSEI [19], Few-Shot [54], and DeepRepair [72] are the most popular ones. For a fair comparison, we employ the same settings on all six repairing methods and ArchRepair. In order to fully evaluate the effectiveness of the proposed method, we apply all methods (six baselines and ArchRepair) to fix four different DNN architectures on large-scale datasets, including the clean version and 15 corrupted versions from CIFAR-10 and Tiny-ImageNet, to assess the repairing capability.

Other configurations. We implement ArchRepair in Python 3.9 based on PyTorch framework. All the experiments were performed on a server with a 12-core 3.60 GHz Xeon CPU E5-1650, 128 GB RAM, and four NVIDIA GeForce RTX 3,090 GPUs (each has 4 GB memory), which runs Ubuntu 18.04.

In summary, for each baseline method and ArchRepair, our evaluation consists of 96 configurations (6 DNN architectures \(\times\) 16 versions of a dataset³) on four datasets (i.e., CIFAR-10, CIFAR-100, Tiny-InageNet, and ImageNet. For CIFAR-10 dataset, an execution of training and repairing a model under one specific configuration costs about 12 hours on average (the maximum one is about 50 hours); while for Tiny-ImageNet dataset, an execution of training and repairing a model takes about 18 hours on average (the maximum one is about 64 hours). We measured the execution time of repairing six different DNN architectures (i.e., VGGNet-16, ResNet-18, ResNet-50, ResNet-101, DenseNet-121, and EfficientNet-B0) repaired on CIFAR-10 within 100 epochs by different repairing methods. The results are reported in Table 4. According to Table 4, the Neuron-lv’s methods use less execution time than other repairing methods (The cell with green background), and the Network-lv’s methods use more execution time than others. Our method, ArchRepair, uses more execution time than Neuron-lv’s methods but less than Network-lv’s methods. This is because ArchRepair repairs DNN models on the block level, which works at a larger size than the neuron level but at a smaller size than the network level. This is also consistent with our expectation in Section 2.2, i.e., block-level repairing makes a good tradeoff between effectiveness and efficiency. Overall, the total execution time of our experiments takes more than two months.

To answer RQ1, we train 6 DNNs (i.e., VGGNet-16, ResNet-18, ResNet-50, ResNet-101, DenseNet-121, and EfficientNet-B0) on 4 datasets’ (i.e., CIFAR-10, CIFAR-100, Tiny-ImageNet, and ImageNet) training datasets (i.e., \(\mathcal {D}^\text{t}\)) and evaluate them on testing datasets (i.e., \(\mathcal {D}^\text{v}\)), respectively. To evaluate the performance of our method (i.e., ArchRepair), we apply six different SOTA methods as well as ArchRepair to repair these 4 DNNs. The evaluation results of repairing are summarized in Table 5. In general, ArchRepair exhibits significant advantages over all baseline methods on the 6 DNNs, demonstrating the effectiveness and generalization ability of the proposed method. In particular, comparing with the state-of-the-art DNN repair methods (i.e., neuron-level repairing method Arachne [59], and network-level repairing method DeepRepair [72]), ArchRepair achieves much higher accuracy on 5 out of 6 DNNs on CIFAR-10 dataset. On the more challenging dataset, Tiny-ImageNet, ArchRepair still achieves much higher accuracy on 3 out of 6 DNNs. Note that on DenseNet-121, all the repairing methods failed to repair, i.e., failing to improve the performance compared to the original network. One possible explanation is that the original DenseNet-121’s performance has almost reached the upper bound of the classification accuracy on Tiny-ImageNet, hence there might not be much room for improvement in terms of accuracy. To better illustrate the performance of ArchRepair compared with other baselines, we also conduct the statistical test (i.e., Wilcoxon Signed-rank Test) on the results obtained by our method, compared with each of the 6 corresponding repairing methods across all 6 different models (i.e., VGGNet-16, ResNet-18/50/101, DenseNet-101, and EfficientNet-B0), all the 4 evaluated datasets (i.e., CIFAR-10/100, Tiny-ImageNet, and ImageNet). Table 6 summarizes the obtained statistical test results, which demonstrate the advantage of our method to be statistically significant at the 0.01 confidence level (i.e., \(p\lt 0.01\)), compared with the SOTA. We report the obtained significant test results in Table 6 and add a paragraph of discussion in the original article, the results confirm the advantage of our method to be statistically significant at the 0.01 confidence level (i.e., \(p\lt 0.01\)).

Furthermore, to understand the influence of repairing on DNN’s robustness, we evaluate the repaired DNNs’ performance on corruption datasets (i.e., CIFAR-10-C [26] and Tiny-ImageNet-C [26]). The CIFAR-10-C and Tiny-ImageNet-C contain over 15 types of natural corruption datasets, and we show the results on CIFAR-10-C in Figure 4 and Tiny-ImageNet-C in Figure 5. Obviously in Figure 4, ArchRepair achieves the highest accuracy on a majority of corruption datasets across three variants of ResNet (8/15, 9/15, and 7/15 on ResNet-18, ResNet-50, and ResNet-101, respectively) besides the best performance on the clean dataset. Even on DenseNet-121, which is not a block-like DNN, ArchRepair also achieves promising performance compared with SOTA method Apricot [74]. The performance of ArchRepair is also significant on Tiny-ImageNet-C. As we’ve mentioned before, Tiny-ImageNet is way more challenging. Nevertheless, ArchRepair still outperforms baselines in terms of the robustness on a majority of corruption datasets across three variants of ResNet (9/15, 9/15, and 7/15 on ResNet-18, ResNet-50, and ResNet-101, respectively) as well as the non-block-like DNN DenseNet-121 (8/15). The results confirm that ArchRepair doesn’t harm the DNN’s robustness, and on the contrary, it can even sometimes improve DNN’s generalization ability towards classifying corrupted data.

In Section 5.1, our investigation results demonstrated that ArchRepair will not affect DNN’s robustness when repairing on the clean dataset. Hence in this section, we continue to validate whether our method harms DNN’s robustness when repairing a specific failure pattern.

We first verify the repairing capability of ArchRepair. We repair a deployed DNN (i.e., ResNet-18⁴) on each of the corruption datasets from CIFAR-10-C and Tiny-ImageNet-C, and compare the performance with the other repairing methods, where the results are summarized in Table 6. Comparing the experimental results on the corruption dataset, we see that all repairing methods have the capability to repair the failure patterns, except shot noise (SN) on Tiny-ImageNet-C (all repairing methods fail to repair this corruption pattern). Among these repairing techniques, our method ArchRepair has the highest accuracy on 8 out of 15 the corruption datasets on CIFAR-10-C dataset, and 9 out of 15 the corruption datasets on Tiny-ImageNet-C, respectively, demonstrating that ArchRepair exhibits the advantages in repairing failure patterns.

To validate whether our method has harmed DNN’s robustness, we also evaluate the performance of repaired DNNs on the other corruption datasets. The evaluation results on CIFAR-10 and Tiny-ImageNet are shown in Figures 6 and 7, respectively. Besides, we calculate the robustness of repaired models with the formula used in SENSEI. The results of robustness are recorded in Table 8. Comparing the accuracy difference on CIFAR-10-C (see Figure 6), we observe that the DNNs repaired by ArchRepair (i.e., the red bar) have higher accuracies on both clean and corruption datasets than the original DNN (i.e., the gray bar, which is lower than others in most of the cases), indicating that repairing method will not harm the DNN’s robustness when having fixed certain corruption patterns. Also, this fact proves that the repairing procedure will not cause over-fit. This is also verified by the results on Tiny-ImageNet-C (see Figure 7), where repairing on a certain corruption pattern does not affect the DNN’s robustness on clean dataset and other corruption patterns. Instead, it can even enhance the robustness in some cases (e.g., when repairing on Fog corruption is performed, the performance on other corruptions is also improved).

To verify the effectiveness of our localization method, we conduct an experiment by applying the repairing method on all 4 blocks of ResNet-18 and ResNet-50, and comparing the accuracy on the clean datasets \(\mathcal {D}^\text{v}\) of both CIFAR-10 and Tiny-ImageNet with their block suspiciousness \(\mathcal {S}_\text{B}\) (i.e., the number of suspicious neurons in corresponding block). We calculate the block suspiciousness under 8 different thresholds \(\epsilon _i\)⁵ (\(i\in \lbrace 10, 20, 30, 40, 50, 75, 100, 150\rbrace\)) to evaluate how the threshold \(\epsilon _i\) affects the block suspiciousness. The experimental results are summarized in Table 9.

As shown in Table 9, the block suspiciousness \(\mathcal {S}_\text{B}\) of Block 4 in ResNet-18 and Block 3 in ResNet-50 are always the highest on both CIFAR-10 and Tiny-ImageNet datasets, no matter what value the threshold \(\epsilon _i\) is. It matches the performance of repaired DNNs, where the DNN repaired on Block 4 in ResNet-18 and Block 3 in ResNet-50 has the highest accuracy, respectively. This demonstrates that our localization method can correctly locate the most vulnerable block.

It’s worth mentioning that for a simpler DNN architecture, i.e., ResNet-18, the vulnerable candidate block can be located more accurately when the threshold \(\epsilon _i\) is small. As the threshold \(\epsilon _i\) increases, the block suspiciousness \(\mathcal {S}_\text{B}\) on other blocks becomes larger, making the localization method difficult to identify the vulnerable block. While for ResNet-50 (a relatively complex DNN), no matter what value the threshold \(\epsilon _i\) is, the localization result is always significantly accurate (with a much higher suspiciousness \(\mathcal {S}_\text{B}\) compared with other blocks).

To demonstrate the effectiveness of our ArchRepair and investigate how each component contributes to its overall performance, we conduct an ablation study by repairing 4 pre-trained models (i.e., ResNet-18, ResNet-50, ResNet-101, and DenseNet-121) with two variants of our method on both CIFAR-10 and Tiny-ImageNet datasets. Table 10 summarizes the evaluation results. The first one performs ArchRepair on one single layer of the DNN, and we denote these variants as “Layer-lv” in Table 10. The second one is our full (complete) version that applies ArchRepair at the block level, we denote this variant as “Block-lv” in Table 10.

Compared with the original DNNs, the performance of “Layer-lv” is acceptable on CIFAR-10 dataset, as it slightly improves the behaviors on three DNNs (i.e., ResNet-18, ResNet-50, and DenseNet-121) and only decreases slightly on ResNet-101. The “Block-lv” achieves better performance on all of the four DNNs on CIFAR-10, and these results indicate that ArchRepair’s repairing capability is effective at both levels. The performance on “Block-lv” is better than the “Layer-lv” on all the four DNNs on two different datasets, especially on the more challenging dataset Tiny-ImageNet, where “Layer-lv” only shows small improvement on ResNet-18 while “Block-lv” has significant improvement on all three variants of ResNet. This demonstrates that repairing on one specific layer cannot fully unleash ArchRepair’s potential while repairing on a block enables to take the advantage of all components of ArchRepair. Note that even though both “Block-lv” and “Layer-lv” fail to repair DenseNet-121 on Tiny-ImageNet (as well as all the SOTA baseline methods, see evaluation results in Table 5), “Block-lv” still performs better than “Layer-lv”.

The threats to the validity of this article could come from the following aspects: (1) The selected dataset and the used model architectures could be a threat. To mitigate this, we selected popular datasets as well as diverse architectures to evaluate our method. (2) The selection of the corruption dataset could be biased, i.e., our method and results may not generalize well on other corruptions. To counteract this, we tried our best and selected as many as 15 diverse and commonly used natural corruptions in the standard benchmarks of previous work [26]. (3) Another threat is from the implementation of our method as well as the usage of the existing baselines. To mitigate the threat, we carefully follow the configuration as stated in the original articles or implementation, respectively. Moreover, our co-authors carefully test and review our code and the configuration of other tools. Furthermore, to be comprehensive for better understanding the position of ArchRepair, we perform a large-scale comparative study against 6 SOTA DNN repair techniques. The results confirm DNN repair could be even more promising and there are still opportunities ahead when going beyond focusing on repairing DNN weights only.

DNN testing is an important and relevant technique to DNN repair, aiming to detect potential buggy issues of a DNN. Some recent work focuses on testing criteria design. For example, DeepXplore [49] proposes the neuron coverage based on the number of activated neurons on given testing data, where the neuron coverage represents the adequacy of the testing data. Similarly, DeepGauge [43] proposes multi-granularity testing criteria, which are based on the analysis of neural behaviors. DeepCT [42] considers the interactions between the different neurons, and further Kim et al. [31] propose the coverage criteria to measure the surprise of the inputs based on the neuron features at the layer level. Some researchers [24, 56] recently also point out that the neuron coverage might fail if most of the neurons are activated by a few test cases, and further in-depth research is still needed along this line.

Overall, these testing criteria lay the early foundation for testing generation techniques to detect defects in DNNs. DeepTest [64] generates test cases based on the guidance of neuron coverage. TensorFuzz [48] proposes a distance-based coverage-guided fuzzing technique to test DNNs. Similarly, DeepHunter [69] proposes another coverage-guided testing technique by integrating the coverage criteria from DeepGauge. Readers can also see [44]. DeepStellar [13] employs the coverage criteria and fuzzing technique, to test and analyze the recurrent neural network. More discussions on the progress of machine learning testing can be referred to the recent survey [41, 75]. Different from these testing techniques, our work mainly focuses on repairing DNNs and enhancing their robustness and generalization capability, which can be considered as the downstream tasks of DNN testing.

Fault localization aims to locate the root causes of software failures. Similar approaches have been widely studied for traditional software, which focuses on developing faults identification methods such as spectral-based [1, 29, 34, 35, 46, 50, 76], model-based [4, 55], slice-based [2], and semantic fault localization [10]. Several works recently introduce fault localization on DNNs to find vulnerable neurons and repair their weights. Representative techniques include sensitivity-based fault localization [59] and spectrum-based fault localization [14]. Eniser et al. [14] try to identify suspicious neurons responsible for unsatisfactory DNN performance, which is an early attempt to introduce fault localization techniques on DNNs with promising results. However, these methods only consider a fixed DNN architecture and neuron-aware buggy behaviors, which is less flexible for real-world applications. Our work repairs DNN at a higher level (i.e., block level) by localizing the vulnerable block and jointly repairing the block architecture and weights, which is novel and has not been investigated in previous work.

So far, there are several attempts for repairing DNN models. Inspired by software debugging, Ma et al. [45] propose a novel model debugging technique for neural network models, which is denoted as MODE. MODE first performs state differential analysis on hidden layers to identify the faulty neurons that are responsible for the misclassification. Then, an input selection algorithm is used to select new input samples to retrain the faulty neurons.

Zhang et al. [74] propose a weight-adjustment approach named Apricot to fix the DNN. Apricot first generates a set of reduced DNNs from the original model and trains them with a random subset of the original training dataset, respectively. For each failure example, Apricot separates reduced DNN models into two partitions, one successfully predicts the label and the other does not, and takes the mean of the corresponding weight assignments of two partitions. After that, Apricot automatically adjusts the weight with these mean values. Further, Sohn et al. [59] propose a search-based repair technique for DNNs, called Arachne. Unlike other techniques, Arachne directly manipulates the neuron weights without retraining. Arachne first uses positive and negative input data to retain correct behavior and generate a patch, respectively. Then uses Particle Swarm Optimization (PSO) to search and locate faulty neurons, and uses the result of PSO candidate to update neurons’ weights, and further calculates fitness value based on the outcomes.

Recently, Gao et al. [19] have proposed a new algorithm called SENSEI, which uses guided test generation techniques to address the data augmentation problem for robust generalization of DNNs under natural environmental variations. Firstly, SENSEI uses a genetic search on a space of the natural environmental variants of each training input data to identify the worst variant for augmentation on each epoch. Besides, SENSEI uses a heuristic technique named selective augmentation, which allows skipping augmentation in certain epochs based on an analysis of the DNN’s current robustness. Ren et al. [54] uses Gaussian mixture model (GMM) to estimate the noise distribution and guide the data augmentation process for DNN repairing. It first applies GMM on collected failure data. Then the estimated GMM samples augment weights to mix the augmented data and generate a set for repairing. Another recent attempt for DNN repair is DeepRepair [72], a method that repairs the DNN on the image classification task. DeepRepair uses a style-guided data augmentation for DNN repairing to introduce the unknown failure patterns into the training data to retrain the model and applies clustering-based failure data generation to improve the effectiveness of data augmentation. NeuRecover [65] uses the history of training process to determine which DNN parameters should be corrected and corrects it through the use of particle swarm optimization. Nakagawa et al. [47] address the difficulty of repairing DNN by first searching for correction proposals for each different type of recognition error and performing the correction of “suspicious parameters”. DistrRep [5] first searches for the best fix for each different misclassification, then combines these fixes into a single repaired DNN model according to their risk levels. Kim et al. [32] make one of the earliest attempts on repairing neural network architectures through proposing a benchmark on both real and artificial DNN architecture faults with different hyperparameter optimization methods. Our work differs from them in two ways: (1) our repairing method replaces the faulty neural network blocks instead of only tuning hyperparameters, (2) our work could repair DNNs’ performance against corrupted data in the operational environment, with potentially both the weights and architecture of DNN enhanced.

Our repairing method is orthogonal to existing data augmentation-based methods such as SENSEI [19] and DeepRepair [72], where we focus on repairing DNN from the architecture and weight perspective. Our method also goes one step further beyond the weight level (e.g., MODE [45], Apricot [74], and Arachne [59]), and considers at a higher granularity by jointly repairing architecture and weights at the block level, which is demonstrated to be a promising direction for DNN repairing.

Note that, the field of DNN repairing has been progressing very fast, with some concurrent work proposed during the enhancement of our work. We would continuously update our supplementary website [52] to keep the relevant techniques of DNN repairing updated, and hopefully provide a basis to ease further research in this direction.

Neural architecture search (NAS) could be another relevant line of our work, aiming to automatically design an architecture instead of handcrafting one. Typical NAS includes evolution-based [53, 68], and reinforcement-learning-based [3] methods. However, the resources RL or evolution-based methods leveraged are often very expensive and still unaffordable in practice. More recently, DARTS [39] relaxes the search space to make it continuous so that the search processes can be performed based on the gradient. Differentiable NAS approaches can significantly reduce computational costs. Our search method is based on PC-DARTS [70], a stability-improved variant of DARTS by introducing a partially connected mechanism.

The purpose of repairing and NAS is very different. The former intends to fix the buggy behaviors that follow some patterns with generalization capability, while NAS is to design general architecture automatically for better performance (e.g., energy efficiency). In this article, we formulate the block-level joint architecture and weight repairing as a NAS problem, which demonstrates the possibilities and chances for DNN repair along this direction.