A distributed algorithm is proposed for solving a linear algebraic equation Ax = b over a multi-agent network, where the equation has a unique solution x* ∈ ℝⁿ. Each agent knows only a subset of the rows of [A b], controls a state vector xi(t) of size smaller than n and is able to receive information from its nearby neighbors. Neighbor relations are characterized by time-dependent directed graphs. It is shown that for a large class of time-varying networks, the proposed algorithm enables each agent to recursively update its own state by only using its neighbors' states such that all xi(t) converge exponentially fast to a specific part of x* of interest to agent i. Applications of the proposed algorithm include solving the least square solution problem and the network localization problem.