High-definition (HD) map is crucial for intelligent vehicles to perform high-level localization and navigation. To improve the availability and usability of HD map, it is meaningful to investigate crowd-sourced mapping solutions and low-cost map-aided localization schemes which don't rely on high-end sensors. In this letter, we propose a novel vision-based mapping and localization system, which could generate compact instance-level road maps automatically and provide high-availability map-aided localization. The spatial uncertainties of the map elements are taken into consideration by analyzing the inverse perspective mapping (IPM) model, which enables more flexible map usages in both mapping and localization phases of the system. Besides, a pose graph optimization framework is developed for accurate pose estimation by fusing global positioning (GNSS), local navigation (odometry) and map matching information together. Real-world experiments in urban environment were conducted to validate different phases of the system, including on-vehicle mapping, multi-source map merging and map-aided localization.