Graph-Cut Based DNN Inference Task Partitioning and Deployment Method

Abstract

As users demand higher inference accuracy, the number of network layers and neurons in Deep Neural Network (DNN) models continues to grow, resulting in increasingly demanding requirements for computational power, storage, and other resources for DNN inference tasks. On the edge side, partitioning resource-intensive DNN inference tasks into multiple dependent subtasks and deploying them to different nodes has become a crucial approach to ensuring task computation efficiency. To address the problem of fine-grained partitioning of DNN inference tasks with directed acyclic graph (DAG) topology, a graph-cut-based method for DNN inference task partitioning and deployment is proposed. Firstly, a distributed edge-terminal collaborative architecture is constructed to model the partitioning and deployment of DNN inference tasks with DAG topology. Then, the problem of optimal partitioning and deployment of DNN inference tasks with minimal latency and energy consumption is formulated. Finally, graph-cut-based algorithms for DNN inference task partitioning and computation resource allocation are designed. Experimental results demonstrate that the proposed method optimally utilizes the limited and distributed resources at the edge, effectively ensuring service timeliness.