The problem of dynamic load balancing in heterogeneous networks is investigated. The sub-systems considered are a UMTS network with WLAN hotspots. A load balancing algorithm is first proposed that can push traffic between the two sub-systems according to load conditions and predetermined load thresholds. To adapt the algorithm to traffic condition, the load threshold for the UMTS to WLAN vertical handover is auto-tuned. The auto-tuning process is orchestrated by a fuzzy logic controller that is optimized using the fuzzy Q-learning algorithm. Numerical simulations using a semi-dynamic network simulator illustrate the effectiveness of the proposed approach.