Traditional visible light positioning (VLP) systems estimate receivers’ coordinates based on the assumption that light-emitting diode (LED) coordinates are known accurately, which is not always practical. Even with accurate laser range finders, the structural changes of the buildings due to temperature, humidity, and material aging might affect the LED coordinates unpredictably. Besides, when the number of LEDs increases, it is time-consuming to build a database of LED coordinates. In this paper, we propose a novel system exploiting two optical angle-of-arrival (AOA) estimators to localize the LEDs, and the AOA estimators have fixed relative positions. Each AOA estimator has four photodiodes (PDs) towards different orientations to estimate the incident light direction. Considering the additive noises imposed on the PDs, we derive a closed-form error expression of the LED coordinate estimator. Analytical and simulation results show that the two AOA estimators should be set symmetrically with respect to the room center to reduce the error of the LED coordinates database. Moreover, an experimental platform is built with two fixed AOA estimators and one moving LED in order to verify the approach. The measured average localization error is 4.46 cm and the time of one localization is about 23.4 ms.