Hidden Markov Models (HMMs) are one of the most popular and successful models in statistics and machine learning for modeling sequential data. However, one main issue in HMMs is the one of choosing the number of hidden states. The Hierarchical Dirichlet Process (HDP)-HMM is a Bayesian non-parametric alternative for standard HMMs that offers a principled way to tackle this challenging problem by relying on a Hierarchical Dirichlet Process (HDP) prior. We investigate the HDP-HMM in a challenging problem of unsupervised learning from bioacoustic data by using Markov-Chain Monte Carlo (MCMC) sampling techniques, namely the Gibbs sampler. We consider a real problem of fully unsupervised humpback whale song decomposition. It consists in simultaneously finding the structure of hidden whale song units, and automatically inferring the unknown number of the hidden units from the Mel Frequency Cepstral Coefficients (MFCC) of bioacoustic signals. The experimental results show the very good performance of the proposed Bayesian non-parametric approach and open new insights for unsupervised analysis of such bioacoustic signals.