We address the problem of identifying a graph structure from the observation of signals defined on its nodes. Fundamentally, the unknown graph encodes direct relationships between signal elements, which we aim to recover from observable indirect relationships generated by a diffusion process on the graph. We put forth a novel network topology inference approach whereby we: i) identify the eigenvectors of a matrix representation of the graph from realizations of the diffused signal; and ii) rely on these (possibly imperfect) spectral templates to estimate the eigenvalues by imposing desirable properties on the graph to be recovered. Robust algorithms with quantifiable performance are developed for the pragmatic settings where the eigenvectors are estimated with errors, or, when the eigenbasis is only partially known. Numerical tests showcase the effectiveness of the proposed algorithm in recovering amino-acid networks.