Designing an effective and efficient end-to-end optimization framework with good generalization for shop scheduling is an emerging topic in the informational manufacturing system. Existing end-to-end frameworks have achieved satisfactory results for combinatorial optimization problems (COPs), such as traveling salesman problem and vehicle routing problem. However, the performances of these methods in solving complex COPs, such as shop scheduling, need to be improved. In this article, a knowledge-guided end-to-end optimization framework based on reinforcement learning (RL) is proposed to solve the permutation flow shop scheduling problem (PFSP). First, a new policy network is designed based on the problem characteristics to deal with different scales of PFSPs and achieve iterative end-to-end generation. Second, an improved policy-based RL algorithm by using the knowledge accumulated during the training process is designed to enhance the training quality. Third, a knowledge-guided improvement strategy is introduced through the cooperation of local search and supervised learning to improve the learning of the policy. Simulation results and comparisons show that the knowledge-guided end-to-end optimization framework can obtain better results than different kinds of commonly used optimization methods in limited computation time for solving the PFSP.