Image deraining is an effective solution to avoid performance drop of vision-oriented tasks in rainy weather. Most existing image deraining approaches either fail to produce satisfactory restoration results or cost too much computation. In this paper, we propose a low-complexity and high-performance coupled representation module (CRM), designed to learn the joint features of rain-free contents and rain information as well as their blending correlations. To promote the computation efficiency, we employ depth-wise separable convolutions, and construct CRM in an asymmetric U-shaped architecture to reduce model parameters and memory footprint. Our final model–PCNet achieves the progressive separation of rain-free contents and rain streaks using cascaded residual learning. Extensive experiments are conducted to evaluate the efficacy of the proposed PCNet on several synthetic and real-world rain datasets.