Numerous network management tasks in Software-defined networking (SDN) infrastructures, such as routing, resource allocation, and service placement, heavily depend on obtaining an accurate view of the network state. However, monitoring individual network elements incurs substantial overhead and often proves infeasible. To address this challenge, network tomography has emerged as a promising approach, capable of inferring the internal network state using end-to-end metrics observed by a limited set of nodes acting as monitors. Despite its potential, previous research in network tomography has not considered specific network management objectives and corresponding challenges, resulting in unsatisfactory performance. In this paper, we propose Subito (Shortest Path Routing with Multi-armed Bandits and Network Tomography), which integrates network tomography and reinforcement learning within software-defined networks to address the specific needs and challenges of delay-aware shortest path routing–a cornerstone of various network management tasks. By harnessing the capabilities of network tomography and reinforcement learning, Subito efficiently learns routing strategies with bounded regret, achieves minimal monitoring overhead, and maintains stable routing. Extensive experimental evaluations on synthetic networks and the GENI testbed show significant performance improvements of Subito versus two state-of-the-art approaches.