In this paper, the leader-following consensus problem of multi-agent is studied. A decentralized adaptive design method is presented for multi-agent systems with non-identical unknown nonlinear dynamics, and for a leader to be followed that is also nonlinear and unknown. By parameterizations of unknown nonlinear dynamics of all agents, a decentralized adaptive consensus algorithm is proposed for networks with switching topologies by using only relative position feedback of neighboring agents. Analysis of stability and parameter convergence of the proposed algorithm are conducted based on algebraic graph theory, Lyapunov theory and PE (Persistent Excitation) condition. Finally, an example is given to validate the theoretical results.