AdaFlow: Aggressive Convolutional Neural Networks Approximation by Leveraging the Input Variability

The intrinsic error tolerance of Convolutional Neural Networks (CNNs) presents opportunities for approximate computing techniques to tradeoff classification accuracy for computational effort reduction. The reduction can directly translate into energy savings. However, most of conventional approximating schemes gain very limited Return-of-Approximation (ROA)—computational effort reduction at certain accuracy degradation level—because they rely on the "one-fits-all" approach that exploits single "approximate (inexact) CNN model" to equally serve for all inputs. We find that the classification difficulty varies widely across inputs for a given application: a large majority of inputs can be easily classified by a simple model with low computational effort (i.e., high ROA), while only a small fraction of inputs have to be classified by a very complex model with high computational effort (i.e., low ROA). The "worst cases," hard classified inputs, forced conventional approximating schemes to choose more complex approximate CNN model to meet application-specified Quality of Service (QoS) requirement thereby sacrificing the high ROA on easy classified inputs. Consequently, finegrained approximating technology, separately treating hard and easy inputs, is critical to achieve more aggressive approximation. In this paper, we propose an adaptive approximation flow, AdaFlow, to exploit the variability of input classification difficulty. AdaFlow exploits multiple basic learners with different computational efforts, and adaptively selects suitable learner for each input according to its classification difficulty. Furthermore, we design a hardware accelerator to efficiently support the proposed adaptive approximation flow. Compared with state-of-the-art approximating schemes, AdaFlow gains more than 2× performance speedup and 1.5× energy efficiency improvement on average.