Source mask optimization (SMO) is vital for mitigating lithography imaging distortions caused by shrinking critical dimensions in integrated circuit fabrication. However, the computational intensity of SMO, involving multiple integrals in Abbe’s theory, hinders its widespread adoption and advancement. In this article, we present Diff-SMO, a highly efficient and accurate SMO framework with a primary emphasis on enhancing source optimization techniques. Previous research was confined to mask optimization acceleration due to the constraints of the academia lithography model. Diff-SMO extends the scope of optimization by concurrently refining the intricate interplay between the source and mask. We first develop a GPU-accelerated lithography simulator grounded in Abbe’s theory, enabling full GPU acceleration throughout the SMO process. Furthermore, we propose a discrete diffusion model for generating quasi-optimal sources, significantly improving computational efficiency. Our experimental results demonstrate exceptional imaging fidelity, surpassing the state-of-the-art, with over 200 times higher throughput compared to traditional SMO methods.