Recent studies have proposed various attacks against location privacy using a Markov Chain transition matrix trained for each user. However, when a user has disclosed only a small amount of location information in the past, the training data can be extremely sparse. In this paper, we show how the attacker can solve this sparse data problem, and how the defender can defend against this type of attack. Our proposal is twofold: 1) We propose a training method that regards a set of transition matrices as a “tensor”, and adopt tensor factorization to robustly estimate transition matrices from a small amount of training data. 2) We then focus on a location prediction attack, which predicts a location of a target user from a past location that he/she disclosed, and propose a region merging method to minimize the region size as an optimal defense. The experimental results using the dataset of taxi traces show the effectiveness of our proposals. We also point out that our region merging method is effective especially when the defender has Big Data to train transition matrices.