In this paper, we address the problem of detecting and segmenting partial image blur from a single input image. Instead of assuming particular image priors or requiring additional user annotation, we propose a novel learning framework which jointly solves the tasks of blur kernel estimation and image blur segmentation, so that partial image blur can be automatically separated from the remaining parts of the input image. By alternating between the two learning tasks, we show that our proposed method would achieve promising detection and segmentation performance, which would benefit further processing or analysis tasks of interest. We also verify that, via both qualitative and quantitative evaluation, our approach would perform favorably against state-of-the-art blur detection or segmentation works.