Advanced sensing, data analysis, and communication techniques have promoted the emergence and tremendous development of the Intelligent Industrial Internet of Things (IIoT). Intelligent IIoT-enabled 5G communication networks improve overall efficiency and open up a new market opportunity and economic growth era. In particular, for the ultra-reliable and low-latency communication (URLLC) scenario, the system requires a lightweight algorithm to ensure transmission reliability while providing rapid radio resource allocation. A proactive downlink system framework, supported by the reinforcement learning-based online model-free algorithm, is proposed to meet the upcoming challenge. The proactive task data transmission problem is decomposed into three sub-problems. With the help of anticipatory mobility management and virtual cell, the system gains high reliability through multipath diversity. The anchor node forwards the task data to multiple access points and manages radio resource allocation among and under the access points. Simulations demonstrate that the proposed framework handles the URLLC scenario well.