Vision and inertia/odometry sensors fusion strategy is popular in the recent years for the robot localization, due to its feasibility in GPS-denied environments. In this paper, a new adaptive estimation algorithm, inspired by the Slotine-Li adaptive control algorithm, is designed to fuse the monocular vision and inertia/odometry sensors for estimating the robot position. By the new method, the robot can be localized in GPS-free and map-free environments, and the localization results can be theoretically proved convergent to their real values and robust to the measurement noises. Comparing with other methods, our algorithm is simple to implement and suitable for parallel processing. To achieve the real-time performance, the algorithm is implemented in parallel using GPU, therefore it can be easily integrated into control tasks which need the real-time robot localization information.