Using directional antennas in wireless mesh networks (termed DMesh in this paper) is attractive due to its longer coverage range and spatial separation between transmissions of the same channel. However, the connectivity in a DMesh is much lower than its omni-directional counterpart. This makes topology control (through beaming) a critical problem in DMesh. Because topology control coupled with routing decision, their joint optimization is critical to achieve the best performance. In this paper, we consider a multimedia DMesh with a certain traffic demand at each mesh router. We first formulate the joint topology control and routing assignment as an optimization problem and show that it is NP-hard. To address the problem, we propose a novel and efficient heuristic called TORA (Joint Topology Control and Routing Assignment), an ant-colony optimization algorithm which seeks to jointly optimize topology and routing for DMesh. Simulation results based on NS3 show that TORA performs substantially better in terms of loss rate, delay and throughput as compared with a recent scheme.