Neural network-based filters have shown their potential in removing video compression artifacts. However, previously studied neural networks have achieved boosted filtering performance by continuously increasing network complexity, causing heavy burden on memory cost and computation speed. In this paper, we firstly analyze properties of original residuals which are the difference between original and predicted pixel values. Then an in-loop filter based on low-complexity CNN using residuals(CNNF-R), which are generated after compression and reconstruction from original residuals, is proposed for intra video coding. Insights of designing the network are also demonstrated. Compared with the state-of-the-art video coding standard HEVC, CNNF-R achieves up to 6.8% BD-rate reduction and 4.8% on average under all intra configuration, and 2.3% on average under random access configuration. Meanwhile, CNNF-R outperforms the previous network VRCNN in terms of nearly 70% decrease in computation complexity, considerable decrease in memory consumption and 1.2% increase in BD-rate reduction.