In the edge computing Internet of Things network, we minimize the offloading overhead (caused by the bandwidth cost for data transmission and computation resource consumption for task remote processing) while providing the end-to-end (E2E) delay provisioning. Under the scenario, a tandem queue consisting of a transmission queue and a computing process queue is formed by the tasks offloaded to the edge server via wireless link and then processed through the computing resource. Due to the tandem queue, the offloading decision and computing resource allocation are coupled over the tandem queue. To make the problem tractable, we decouple the above two operations and propose a two-stage offloading filtering and computing resource allocation policy. After decouple, we then investigate the delay bound violation probability of the tandem queue by leveraging large deviation analysis. Further, we reveal that under the same E2E delay provisioning, the offloading overhead under the proposed decoupled policy can approach to the non-decoupled optimum by selecting an appropriate value of control parameter. Simulation results verify the theoretical analysis and show the efficiency of the proposed policy.