For automatic building detection in aerial images, small and dense buildings make it a very challenging task. It is because small objects lack sufficient information, and dense building distribution makes the localization of the objects confusing. High-quality building proposals can certainly promote the detection performance. The key to the problem is adopting sufficiently proper size and location of bounding boxes to use the image information for the proposal generation. Based on machine learning with a deep convolutional neural network, this letter proposes a new pipeline of building proposal generation, which is an end-to-end process during training and testing. First, the proposed pipeline attempts to find possible object center points called point proposals. Subsequently, a location refinement module and an object scoring module are applied to the boxes generated from the point proposals with a series of sizes and aspect ratios to obtain the final object proposals. This center-point-guided location refinement and multibox scoring method effectively alleviates the small and dense object problems. Experiments in INRIA Aerial Image Labeling data set demonstrate the better performance of our approach than other state-of-the-art proposal methods. In addition, we add a normal classification branch based on our generated proposals to conduct experiments on detection task. Detection result outperforms the latest detection framework R-FCN equipped with ResNet-101 7% mean average precision at 0.7.