Recently, semantic segmentation has made promising progress, but the high cost of processing still limits its application. With focusing on removing the parameters of the networks, filter pruning using the importance criterion is a straightforward and effective technique to obtain the lightweight sub-network. However, we argue that the long-tail distribution in segmentation datasets poses two significant problems which are ignored in existing pruning algorithms: 1) The importance criterion is dominated by head classes which contain numerous positive samples, where the knowledge of tail classes is easily degenerated. 2) The degenerated knowledge of tail classes is hard to recover as their samples are also insufficient during fine-tuning. To address these issues, we propose a Distribution Calibrated Filter Pruning (DCFP) framework for segmentation. Firstly, a gradient-based Equalization Importance Criterion (EIC) is designed to generate a class-balanced pruning procedure. It avoids the bias on head classes by discarding the imbalanced positive gradients. Secondly, we introduce a Geometric-Semantic Re-balanced Loss (GSRL) to emphasize the learning on tail classes during fine-tuning. The GSRL consists of two cooperative components to calibrate the imbalanced optimization on geometric and semantic domains dynamically. Compared with previous methods, DCFP explores a novel distribution-aware pruning framework to obtain lightweight architectures with accurate results. Extensive experiments proved that DCFP achieves impressive performance on four popular segmentation benchmarks.