In this paper, we explore the task to estimate the density map of objects from single image with unknown perspective map. We follow the recent progress in object counting through density map estimation. Object density map estimation is usually suffered from scale variance of objects caused by unknown perspective. In addition, the background and irrelevant objects in the image lead to artifacts in the resulting density maps, which build up the error when heat maps are needed by aggregating density maps. We propose a multi-CNN network with gradient boosting (MCNN-boost) to address these two problems at once. The receptive fields with different sizes in each column of CNNs make the model adaptive to variance in sizes caused by perspective, and the gradient boosting strategy will effectively remove the artifacts caused by the background and irrelevant objects. Thanks to the simple architecture of the network, MCNN-boost is efficient to train and highly adaptive to different data sets with different scenes. We run experiments on one shopping mall data sets with heavy perspective distortion and complex background to show the effectiveness of the proposed algorithm.