Underexposed images are usually low in brightness and contrast, which degrade the performance of many computer vision algorithms. To solve the problem of overexposing areas that tend to be normal while recovering dark areas in low-light image enhancement tasks, we propose an image-to-patch enhancement model and design a lightweight convolutional neural network called PatchNet. Specifically, the new enhancement model indirectly enhances the network by introducing a patch image, which preserves the incremental information from the low-light image to the normal image. The incremental information is fused with the input image to recover the dark areas while protecting the normal areas of the image. Extensive experiments on real datasets demonstrate the advantages of our method over state-of-the-art methods in subjective feeling and objective evaluation. Our method has achieved better results in restoring details and the adjustment of brightness. By comparing to other methods, our method is more efficient.