In this paper, we present a method to efficiently manage visual memory for autonomous vehicle navigation in large scale environments. It relies on two crucial issues for real-time navigation: an efficient organisation of the memory and small computational cost. A software platform (SoViN) dedicated to visual memory management and navigation strategies (including vision-based memory building, localization and navigation) has been developed to fulfill these requirements. We show that using this software architecture makes possible real-time navigation in large-scale outdoor situation using a single camera and natural landmarks.