Balancing traffic among network installed radio base stations is one of the main challenges facing mobile operators because of the unhomogeneous geographical distribution of mobile subscribers in addition to practical and environmental limitations preventing acquiring the best locations to build radio sites. This increases the challenge of satisfying the increasing data speed demand for smartphone users. In this paper, we present a reinforcement learning framework for optimizing neighbor cell relational parameters that can better balance the traffic between different cells within a defined geographical cluster. We present a comprehensive design of the learning framework that includes key system performance indicators and the design of a general reward function. System level simulations show that reinforcement learning based optimization for neighbor cell borders can significantly improve overall system performance; in particular, with a reward function defined as throughput, an improvement up to 50% is achieved.