The Guided Regularized Random Forests (GRRF) is an ensemble learning method based on random forests and has been shown to perform well in terms of both the gene selection and the prediction of accuracy for gene classification. However, the performance may be downgraded because the feature selection in the GRRF uses scores yielded by the original random forests. In this paper, we improve the GRRF's performance by proposing new importance scores. In our experiments, the improved random forests model based on the GRRF enhances the prediction accuracy and outperforms the GRRF model when applied to high dimensional gene data.