Existing recommendation methods usually train several independent modules for each rating information instead of an end-to-end manner. Therefore, these methods may be incapable of collaborative learning leading to sub-optimal results in predicting users' overall interests. The main disadvantage of those two-stage methods is that the overall rating heavily relies on the predicted sub-ratings, and the predictive error of sub-ratings is accumulated during the regression step. Moreover, the regression model is trained with unbiased sub-ratings but used with biased predictive sub-ratings. Meanwhile, the separate training pattern induces more training overhead. To address these problems, we propose a collective model to predict a user's overall rating, which can learn each of the multi-criteria sub-scores simultaneously in an end-to-end manner. This enables the proposed method to improve its prediction quality by transferring the knowledge of criterion to the domain of overall ratings. It reduces the dependence of the predicted score of a specific criterion, making the overall system more robust. In addition, our end-to-end method avoids learning the regress part directly from the unbiased sub-ratings, improving the performance of the overall model. Experiments on three real-world datasets show that our proposed architecture achieves up to 13.14% lower prediction error over baseline approaches.