As a major supplement to LTE-Advanced, D2D communications underlaying cellular networks have proven efficient in offloading network infrastructures and improving network performance. The scheduling mechanism plays a key role in providing better user experience in D2D communications. However, controlled by operators, D2D communications pose specific problems that do not exist in available wireless networks. Therefore, mature scheduling mechanisms devised for cellular networks or ad hoc networks are not directly applicable to D2D communications. In this article, we first review recent research on scheduling mechanisms for D2D communications, and discuss the design considerations and implementation challenges. Then we propose an on-demand scheduling mechanism, DO-Fast, which can provide QoS differentiation capabilities. Next, we provide performance evaluations based on simulations and an experimental testbed. Finally, we conclude this article and point out possible directions for future research.