We consider the problem of Nonnegative Matrix Factorization (NMF) which is a non-convex optimization problem with many applications in machine learning, computer vision, and topic modeling. General existing methods for finding a solution to the problem include additive or multiplicative update rules for doing alternate minimizations and ADMM, which find a locally optimal point for NMF. We propose a new method for finding a solution of NMF which considers transformations of initial factors derived from the singular value decomposition (SVD). This problem is shown to be equivalent to the NMF problem in a sense, and is then restricted to optimize over the set of orthonormal matrices known as the Stiefel manifold. We then utilize a method developed for optimization over this manifold to find solutions to the NMF problem. Application to synthetic data shows that the method exhibits promising characteristics, as it outperforms some traditional methods both in terms of reconstruction error and running time. Using the solution of this restriction as a starting point for the broader problem shows a further rapid decline in the error.