This paper considers the position-based consensus in a network of agents with double-integrator dynamics and directed topology. Two types of distributed observer algorithms are proposed to solve the consensus problem by utilizing continuous and intermittent position measurements, respectively, where each observer does not interact with any other observers. For the case of continuous communication between network agents, some convergence conditions are derived for reaching consensus in the network with a single constant delay or multiple time-varying delays on the basis of the eigenvalue analysis and the descriptor method. When the network agents can only obtain intermittent position data from local neighbors at discrete time instants, the consensus in the network without time delay or with nonuniform delays is investigated by using the Wirtinger's inequality and the delayed-input approach. Numerical examples are given to illustrate the theoretical analysis.