Hidden Markov Models (HMMs) and associated Markov modulated time series are widely used for estimation and control in e.g. robotics, econometrics and bioinformatics. In this paper, we modify and extend a recently proposed approach in the machine learning literature that uses the method of moments and a Non-Negative Matrix Factorization (NNMF) to estimate the parameters of an HMM. In general, the method aims to solve a constrained non-convex optimization problem. In this paper, it is shown that if the observation probabilities of the HMM are known, then estimating the transition probabilities reduces to a convex optimization problem. Three recursive algorithms are proposed for estimating the transition probabilities of the underlying Markov chain, one of which employs a generalization of the Pythagorean trigonometric identity to recast the problem into a non-constrained optimization problem. Numerical examples are presented to illustrate how these algorithms can track slowly time-varying transition probabilities.