In this work, we consider the joint sparse support recovery problem where the goal is to recover the common support of multiple joint sparse vectors from their compressive, linear measurements. We propose a Rényi Divergence based Covariance Matching Pursuit (RD-CMP) algorithm which recovers the common support of the joint sparse signals as the hyperparameters of a joint sparsity inducing Gaussian signal prior. The support hyperparameters are learned as a set variable by solving a reverse information projection problem based on the α-Rényi information divergence. We show that the α-Rényi divergence objective can be expressed as a difference of two submodular functions, and propose an iterative majorization-minimization procedure to minimize the objective, with each iteration involving a greedy optimization. Through simulations, we demonstrate that the proposed RD-CMP algorithm is capable of recovering k-sparse support from fewer than k measurements per signal. Compared with existing covariance matching based joint sparse support recovery methods, RD-CMP is empirically shown to be several times faster in execution.