This paper presents a novel deep hierarchical representation and segmentation approach for high resolution remote sensing image understanding. An information extraction approach using deep hierarchical exploitation for remote sensing image is presented. The key idea is that we adopt a fast scanning image segmentation within a deep hierarchical feature representation framework, using a deep learning technique to split and merge over-segmented regions until they form meaningful objects. The contribution is to develop an effective procedure for multi-scale image representation to address the issue of information uncertainty in practical applications. We test our method on two optical high resolution remote sensing image datasets and produce promising experimental results in the form of multiple layer outputs, which confirm the effectiveness and robustness of the proposed procedure.