Geospatial object detection has made considerable progress with the use of anchor-based object detectors. In such a situation, the detection performance relies heavily on the parameter settings of anchor boxes. We present a Single Shot Anchor-Free Network (SSAFNet) to tackle with this problem. By eliminating the anchor boxes, the SSAFNet completely avoids the carefully predefined anchor boxes parameters and the computation for adapting the huge scale variation of geospatial objects. A compositional attention network is further introduced to enhance the saliency of foreground objects. We evaluate the SSAFNet on the representative NWPU VHR-10 and RSOD datasets, achieving competitive performance with state-of-the-art anchor-based detection methods.