Localization is one of the most crucial and difficult problems for motion control of mobile robots despite of tremendous research efforts made for years. This paper presents a new vision-based controller for controlling a nonholonomic mobile robot to track a desired trajectory without directly measuring its position. A novel adaptive estimator is embedded into this new controller to estimate global position of the mobile robot online using natural visual features measured by a vision system, and its orientation and velocity measured by odometry/inertia/magnetic sensors. It is proved by Lyapunov theory that the proposed controller gives rise to asymptotic tracking of a desired trajectory and convergence of the position estimation to the actual position. The experiment is conducted to validate the proposed controller.