Caching popular contents at cell edge has been recognized as a promising way to facilitate rapid content delivery and alleviate backhaul burden. The content popularity is greatly influenced by recommendations by content providers. In this paper, we leverage this fact to jointly optimize caching and recommendation towards higher caching efficiency. We focus on both personalized and incumbent-aware recommendation. The incumbent content refers to the content that a user is currently browsing, resulted by the user's short-term interest. We model and formulate the resulting cache efficiency maximization problem subject to user satisfaction requirements. We prove the NP-hardness of the problem, and reformulate it using integer linear programming, enabling to solve optimally small-scale instances. Based on problem analysis with a graph representation, we derive three polynomial-time algorithms, where the recommendation sub-problem is solved to global optimum. Among these algorithms, the first two are based on sub-modularity, with $1-e^{-1}$ approximation guarantee under mild conditions, while the last one is an alternation-based algorithm with fast convergence. Numerical results show the close-to-optimal performance of the proposed algorithms.