Multi-hop V2V communication is a fundamental way to realize data transmission in Vehicular Ad-hoc Networks (VANET). It has excellent potential in intelligent transportation systems and automatic vehicle driving, and positively affects the safety, reliability, and comfort of vehicles. With advantages in speed and trajectory, distribution along the route, size, etc., the urban buses have become prospective relay nodes for urban VANETs. However, it is a considerable challenge to construct stable and reliable (meeting the requirements of bandwidth, delay, and bit error rate) multi-hop routing because of the complexity of the urban road and bus line network in the communication area, as well as many unevenly distributed buses on the road, etc. Given this above, this paper proposes a new hierarchical routing algorithm based on V2V geographic topology segmentation. Urban hierarchical routing is divided into two layers. The first layer of routing is called coarse routing, which is composed of areas; the second layer of routing is called internal routing (bus routing within the area). Q-learning is used to formulate the sequence of buses that transmit information within each area. Details are as follows: Firstly, based on a city map containing road network information, the entire city is divided into small grids by physical streets. Secondly, based on an analysis of the characteristics of the adjacent grid bus lines, the grids with the same routing attributes are integrated into the same area, reducing the algorithm's computational complexity during route discovery. Then, for the calculated area set, a coarse route composed of the selected area is established by filtering out a group of areas satisfying from the source node to the destination node. Finally, the bus sequence between anchor intersections is selected within the chosen area, and a complete multi-hop route from the source node to the destination node is finally constructed. Sufficient simulations show that the proposed routing algorithm has more stable performance in terms of packet transmission rate, average end-to-end delay, routing duration, and other indicators than similar algorithms.