Person re-identification involves retrieving correct matches for a target image (query) from a set of gallery images, while video based re-identification extends this to the case of query and gallery videos. Typical video-based re-id methods ignore the temporal evolution of the intermediate representations of the video sequences. We propose a novel loss function, termed rank loss, to explicitly ensure that the learnt representations achieve enhanced performance and robustness as the sequence progresses and that better intermediate representations result in an improved final representation. Experiments indicate that the addition of rank loss indeed helps in improving the re-id performance while achieving performance comparable to state-of-the-art approaches.