Objectness has recently become a standard step in many computer vision tasks. Among various techniques, those based on hierarchical image segmentation play a fundamental role for developments in new data modalities. In this paper, we address the problem of objectness in RGB-D images and propose a novel and effective approach, namely, hierarchical image segmentation ensemble (HISE). Different from existing image segmentation based methods that generate object segments or proposals largely by heuristics or empirical rules, HISE learns superpixel mergings with a hierarchical tree-structured ensemble, where individual merging models of the ensemble are formed by traversing different paths of the tree, and where both the merging accuracy and proposal diversity are emphasized. Furthermore, we use efficient feature measurements that support easy integration of additional clues. Extensive experiments conducted on the benchmark NYU-v2 RGB-D and SUN RGB-D data sets show the competency of our proposed method.