Rating Prediction is a key problem in recommendation system, especially in Bigdata environment with data sparsity. Recently, Factorization Machine (FM) has been proven to be effective in solving the recommendation problem. Whereas, valuable category information of users and items are neglected in basic FM model. In this paper, we fully explore the capabilities of category information to improve the accuracy of rating prediction, and proposed a Category Weight Factorization Machine (CW-FM) based on FM. CW-FM utilizes hierarchical category information to avoid the interaction between feature vectors which have the subordinate relations. Combined with user and item category information, CW-FM is proven to be an effective solutions to reducing the rating error in recommendation systems. The proposed CW-FM is evaluated by extensive experiments with real world datasets. Results show that CW-FM model achieves better iterative efficiency and higher rating accuracy compared to contemporary schemes.