In this paper, we propose a visual path following control scheme for wheeled mobile robots based on the epipolar geometry. The control law only requires the position of the epipole computed between the current and target views along the sequence of a visual memory. The proposed approach has two main advantages: explicit pose parameters decomposition is not required and the rotational velocity is smooth or eventually piece-wise constant avoiding discontinuities that generally appear when the target image changes. The translational velocity is adapted as required for the path and the approach is independent of this velocity. Furthermore, our approach is valid for all cameras obeying the unified model, including conventional, central catadioptric and some fisheye cameras. Simulations as well as real-world experiments with a robot illustrate the validity of our approach.