In 5G small cell networks, edge caching is a key technique to alleviate the backhaul burden by caching user desired contents at network edges such as small base stations (SBSs). However, due to storage space limitation and diverse user preference patterns, a single SBS is unable to cache all the user desired contents and thus leading to low caching efficiency. In this paper, we propose a recommendation-driven multi-cell cooperative caching strategy to improve the caching efficiency. The idea is to aggregate the storage spaces of multiple SBSs into a large shared resource pool, and guide users to access cached contents by content recommendation. First, we formulate the joint cooperative caching and recommendation problem as a multi-agent multi-armed bandit (MAMAB) problem with the aim of minimizing the average download latency. Then, we propose a multi-agent reinforcement learning (MARL)-based algorithm, MARL-JCR, to solve the problem in a fully distributed manner with limited information exchange among the agents. We also develop a modified combinatorial upper confidence bound algorithm to reduce each agent's decision space to reduce computational complexity. The experiment results evaluated on the MovieLens dataset show MARL-JCR decreases the average download latency by up to 60% as compared with the state-of-the-art solutions.