Topology information plays an important role in network management. The existing methods for topology inference based on end-to-end measurements need a threshold for general topologies, which is difficult to select to ensure the inference accuracy. In this paper, we propose a sort-based approach, named SBA, to infer the general topologies without using a threshold. First, a sort-based clustering algorithm, named SBC-AL, is proposed to cluster a group of nodes in which every node has at least one sibling. In the SBA, the nodes are classified into disjoint groups by a fan-out decrement mechanism. Then the SBA uses the SBC-AL to cluster the nodes group by group from the bottom up to infer the topology. We prove that the SBA is consistent and suitable for general topologies. The simulation results show that the SBA has a good performance in both accuracy and efficiency.