Activity recognition in videos is a challenging task, mainly if a scarce number of samples is available for modelling the problem. The task becomes even harder when using generative models such as mixture models or Hidden Markov Models (HMMs), as they demand a lot of samples to determinate their parameters. Additionally, these models rely on the appropriate selection of some parameters, for instance the number of hidden states. Therefore, we propose in this paper the creation of a Universal Background Model (UBM) of features, using videos from public datasets, applied to the activity encoding and an unsupervised modelling of the activities with a distance dependent Chinese Restaurant Process (ddCRP), where the number of states is automatically determined by the process. In order to classify an incoming video-sequence we propose to model it as a ddCRP distribution and to apply a nearest neighbour algorithm based on a kernel between distributions. To carry out this process we use a Probability Product Kernel (PPK) algorithm by previously mapping the ddCRP into a HMM with discrete observations. Preliminary experiments in two public data sets, as Weizmann and KTH, show that this proposal achieves state-of-the-art results.