In this paper, we present an unsupervised hierarchical image segmentation algorithm based on a split-and-merge scheme. In the split phase, we propose an efficient partition algorithm, called Just-Noticeable-Difference Bayesian Sequential Partitioning (JND-BSP), to partition image pixels into a few regions, within which the color variations are perceived to be smoothly changing without apparent color differences. In the merge phase, we propose a simple but effective merging criterion to sequentially construct a hierarchical structure that represents the relative similarity among these partitioned regions. Instead of generating a segmentation result with a fixed number of segments, the new algorithm produces an entire hierarchical representation of the given image in a single run. This hierarchical representation is informative and can be very useful for subsequent processing, like object recognition and scene analysis. To demonstrate the effectiveness and efficiency of our method, we compare our new segmentation algorithm with several existing algorithms. Experiment results show that our new algorithm can not only offers a more flexible way to segment images but also provides segmented results close to human's visual perception.