Heterogeneous networks are considered a promising solution to address the explosive increase in wireless networks capacity demand. Due to their limited coverage, small cells offer better area spectral efficiency than macro-cells, and advanced features, such as cell range extension, aim at biasing the choice of users towards small cells. Whereas these features are designed to maximise the air interface capacity while respecting interference limitations and signal quality degradation, they do not address the backhaul constraints. The backhaul network is expected to match the number of small cells and their corresponding capacity while maintaining a minimum latency. Thus, with the advent of small cells and the help of enabling features to control inter-layer interference, the bottleneck of the wireless network is shifting from the traditional air interface towards the backhaul. In this paper, we propose an adaptive cell range extension approach that is optimised in view of the backhaul capacity and resilience as well as air interface constraints, thus, gears the traffic towards the cell that is capable of insuring an end-to-end service from the user to the core network. We use a reinforcement learning technique, whereby each small cell dynamically sets its bias value in view of the air interface and backhaul varying conditions.