Few-shot segmentation aims to learn a segmentation model that can be generalized to novel classes with a few annotations. Previous methods mainly establish the correspondence between support images and query images with global information. However, human perception does not tend to learn a whole representation in its entirety at once. In this paper, we propose a novel network to build the correspondence from subparts, parts and whole. Our network mainly contain two novel designs: we firstly adopt graph convolutional network to make pixels not only contain the information of each pixel itself but also include its contextual pixels, and then a learnable Graph Affinity Module(GAM) is proposed to mine more accurate relationships as well as common object location inference between the support images and the query images. Experiments on the PASCAL-5ⁱ dataset show that our method achieves state-of-the-art performance.