Decentralized learning has emerged as an influential paradigm for facilitating collaborative learning without a central server. With the Device-to-Device (D2D) communication mechanism, the collaborative topology in decentralized learning dictates the pathways of information flow, significantly affecting learning performance. Furthermore, latency becomes a critical factor in the architectural design of the decentralized learning system. Motivated by these considerations, this letter addresses the convergence prerequisites of decentralized learning by proposing two distinct topological configurations: ring topology and multiple ring topology. We transform these topology design challenges into optimization problems with the goal of minimizing global latency. Finally, we propose corresponding solution strategies and design time-efficient topologies for the optimization problems. Numerical results demonstrate that the proposed topological structures and solution methods can effectively balance latency and test accuracy performance in the D2D communication environment for decentralized learning.