This article proposes a new distributed algorithm for solving linear systems associated with a sparse graph under a generalized diagonal dominance assumption. The algorithm runs iteratively on each node of the graph, with low complexities on local information exchange between neighboring nodes, local computation, and local storage. For an acyclic graph under the condition of diagonal dominance, the algorithm is shown to converge to the correct solution in a finite number of iterations, equaling the graph diameter. For a loopy graph, the algorithm is shown to converge to the correct solution asymptotically. Simulations verify that the proposed algorithm significantly outperforms the classical Jacobi method and an orthogonal projection-based method.