The sixth generation mobile networks (6G) will undergo an unprecedented transformation to revolutionize the wireless system evolution from connected things to connected intelligence, where future 6G Industrial Internet of Things (IIoT) covers a range of industrial nodes such as sensors, controllers, and actuators. Additionally, data scattered around the industrial environments can be collected for the sake of enabling intelligent operations. In our work, the promising multi-access edge computing (MEC) service is introduced into the IIoT system to execute the task scheduling and resource allocation for the sake of various compelling applications. Moreover, we define the objective function as the weighted sum of delay and energy consumption. Next, a novel deep reinforcement learning (DRL)-based network structure is proposed to jointly optimize task offloading and resource allocation. More specifically, the task offloading is decomposed via the new isotone action generation technique (2AGT) and adaptive action aggregation update strategy (3AUS) based on the proposed DRL framework, and the initial problem can be transformed into a convex optimization problem to solve the resource allocation for each IIoT device. Additionally, we periodically renovate the offloading policy in the DRL framework so that our proposed DRL-based decision-making algorithm can better accommodate time-varying environments. Numerous experimental results demonstrate our proposed DRL-based network structure for each IIoT device can obtain quasi-optimal system performance compared with some conventional baseline algorithms.