In recent years, there has been a wide range of applications of crowdsensing in mobile social networks and vehicle networks. As centralized learning methods lead to unreliabitlity of data collection, high cost of central server, and concern of privacy, one important problem is how to carry out an accurate distributed learning process to estimate parameters of an unknown model in crowdsensing. Motivated by this, we present the design, analysis, and evaluation of FINE, a distributed learning framework for incomplete-data and non-smooth estimation. Our design, devoted to develop a feasible framework that efficiently and accurately learns the parameters in crowdsensing networks, well generalizes the previous learning methods in which it supports heterogeneous dimensions of data records observed by different nodes, as well as minimization based on non-smooth error functions. In particular, FINE uses a novel distributed record completion algorithm that allows each node to obtain the global consensus by an efficient communication with neighbors, and a distributed dual average algorithm that achieves the efficiency of minimizing non-smooth error functions. Our analysis shows that all these algorithms converge, of which the convergence rates are also derived to confirm their efficiency. We evaluate the performance of our framework with experiments on synthetic and real-world networks.