Since the bus holding problem is an operational control problem, bus holding decisions should be made in realtime. For this reason, common bus holding approaches, such as the one-headway-based holding, focus on computationally inexpensive, rule-based techniques that try to minimize the deviation of the actual headways from the planned ones. Nevertheless, rule-based methods optimize the system locally without considering the full effect of the bus holding decisions to future trips or other performance indicators. For this reason, this work introduces a Reinforcement Learning approach which is capable of making holistic bus holding decisions in realtime after the completion of a training period. The proposed approach is trained in a circular bus line in Singapore using 400 episodes (where an episode is one day of operations) and evaluated using 200 episodes demonstrating a significant improvement in scenarios with strong travel time disturbances and a slight improvement in scenarios with low travel time variations.