DeepMaker: A multi-objective optimization framework for deep neural networks in embedded systems

Abstract

Deep Neural Networks (DNNs) are compute-intensive learning models with growing applicability in a wide range of domains. Due to their computational complexity, DNNs benefit from implementations that utilize custom hardware accelerators to meet performance and response time as well as classification accuracy constraints. In this paper, we propose DeepMaker framework that aims to automatically design a set of highly robust DNN architectures for embedded devices as the closest processing unit to the sensors. DeepMaker explores and prunes the design space to find improved neural architectures. Our proposed framework takes advantage of a multi-objective evolutionary approach that exploits a pruned design space inspired by a dense architecture. DeepMaker considers the accuracy along with the network size factor as two objectives to build a highly optimized network fitting with limited computational resource budgets while delivers an acceptable accuracy level. In comparison with the best result on the CIFAR-10 dataset, a generated network by DeepMaker presents up to a 26.4x compression rate while loses only 4% accuracy. Besides, DeepMaker maps the generated CNN on the programmable commodity devices, including ARM Processor, High-Performance CPU, GPU, and FPGA.

Introduction

In recent years, deep learning, which uses deep neural networks as the learning model, has shown excellent performance on many challenging artificial intelligence and machine learning tasks, such as image classification [1], speech recognition [2], and unsupervised learning tasks [3]. In particular, Convolutional Neural Networks (CNNs) propose massive success in visual recognition tasks in the past few years and are applied to various computer vision applications [4]. CNNs have penetrated in a broad spectrum of platforms from workstations to embedded devices due to influential learning capabilities.

However, as CNN architectures become increasingly complex in order to improve accuracy, also the energy consumption for inference is becoming a bottleneck. Dealing with enormous computing throughput demand of up-coming complex learning models in the context of big data will be more acute where the failure of traditional energy and performance scaling paradigm in affording of modern applications requirements leads computing landscape towards inefficiency [42]. On the other hand, leveraging high-performance cloud infrastructures for providing required computational capacity is not always feasible, especially for mission-critical applications due to limited network bandwidth, privacy constraints, low-power efficiency, and not guaranteeing worst-case response-time.

Generally, two approaches are presented to tackle these challenges: 1) diminishing the network size by leveraging network pruning techniques during the training phase [1] and 2) employing customized hardware accelerators [13,9,35]. However, optimizing the network architecture at design time should be taken into account as the third approach since the choice of the architecture strongly impacts on both the performance and the output quality of DNNs. To benefit from this opportunity, we propose a neural acceleration framework, named DeepMaker, which automatically generates a robust DNN in terms of network accuracy and network size, then maps the generated network to an embedded device. Unlike previous neural architectural solutions that their focus is only on improving the accuracy level, DeepMaker also considers network size as the second objective of the search space in order to adaptively find a fit DNN for limited resource embedded devices. For this, DeepMaker is equipped with a Multi-Objective Optimization (MOO) method to solve the neural architectural search problem by finding a set of Pareto-optimal surfaces. The design space has been pruned by taking inspirations from a cutting-edge architecture, DenseNet [6], to boost the convergence speed to an optimal result.

The proposed DeepMaker framework uses a multi-objective neuro-evolutionary approach for the space exploration of finding optimal deep neural architectures while mapping the generated network to the given hardware. An overview of the proposed framework is illustrated in Fig. 1. The configuration file of DeepMaker comprises predefined parameters for the MOO algorithm and network training parameters. As shown in Fig. 1, the input of the framework is a dataset for generating a neural network.

To approximate an application, developers first need to identify the approximation region of the code, then provide a training dataset for the specified code block in order to be mimicked by a DNN generated by DeepMaker. The approximation region of the code should be both hotspot and less sensitive to a quality loss in both data and operations. We can define a hotspot as a code region that consumes considerable energy or occupies a significant part of execution time [7].

The output of the DeepMaker framework is a set of optimized architectures. Network pruning is a popular solution for diminishing the amount of network computation. In addition to the design space exploration, DeepMaker can apply a network pruning method on a dense architecture to accelerate finding the optimal neural networks. In a nutshell, our main contributions in DeepMaker are as follows:

• Developed a multi-objective neuro-evolutionary method to discover near-optimal DNN architectures in terms of the accuracy and the network size.

• We applied a network pruning method [39] on the optimized neural network architectures designed by DeepMaker to obtain a higher level of network compression rate.

• Supporting both Multi-Layer Perceptron (MLP) and Convolutional Neural Network (CNN) models fitting with the required accuracy of diverse applications from mathematical function to image classification.

• Adaptive finding the best architecture regarding resource budget and execution time constraints. Then, mapping the generated network on different platforms to evaluate the applicability of DeepMaker is our last contribution.

The remainder of this paper is organized as follows: Section 2 gives preliminaries on CNN and the MOO algorithm. Details of the proposed framework are presented in Section 3, which consists of two solutions for network optimization: Design Space Exploration and Neural Network Pruning. The experimental results are presented in Section 4. Section 5 reviews related work in this scope, after which Section 6 concludes the paper.