In this paper, we consider the agreement problem of a group of agents with nonlinear stochastic dynamics and each agent applies a local (distributed) nonlinear consensus protocol over random networks. We relate the almost sure consensus of agents under nonlinear consensus protocols to the stochastic semistability of nonlinear stochastic systems. We have proposed the notion of almost sure semistability for nonlinear systems with a continuum of equilibrium solutions, and have derived a set of Lyapunov sufficient conditions for almost sure semistability of nonlinear stochastic systems. The almost sure semistability results are then applied to the multi-agent nonlinear consensus problems over random networks.