In this paper, we present a distributed algorithm to solve unconstrained distributed Least Squares optimization problems over networks where the communication links could stochastically break. The iterative algorithm allows each node with limited computation and communication capabilities to obtain the optimal solution of the centralized problem. We provide the convergence analysis of the algorithm based on an internal structure decomposition and an application of stochastic systems theory. The results suggests that agents with simple dynamics, one step memory and local gradient information could collectively solve complicated optimization problems in the presence of unreliable communications.