In this paper, we endeavor to jointly learn multiple distinct but related graphs by exploiting the underlying topological relationships between them. The difficulty lies in how to design a regularizer that accurately describes the intricate topological relationships, especially without prior knowledge. This problem becomes more challenging for the scenarios where data for different graphs are stored separately and prohibited from being transmitted to an unreliable central server due to privacy concerns. To address these issues, we propose a novel regularizer termed pattern graph to flexibly describe our priors on topological patterns. Theoretically, we provide the estimation error upper bound of the proposed graph estimator, which characterizes the impact of some factors on estimation errors. Furthermore, an approach that can automatically discover relationships among graphs is proposed to handle awkward situations without priors. On the algorithmic aspect, we develop a decentralized algorithm that updates each graph locally without sending the private data to a central server. Finally, extensive experiments on both synthetic and real data are carried out to validate the proposed method, and the results demonstrate that our framework outperforms the state-of-the-art methods.