Safe and accurate spacecraft trajectory tracking is challenging with position-only measurements when there exist unmodeled dynamics, external disturbances, and static/moving obstacles. In this article, a collision-free strategy is investigated for achieving the satisfactory tracking performance. First, a prescribed-time extended state observer is introduced to obtain the estimations of lumped uncertainties and velocity using position measurements. It is proven that the devised observer provides the possibility to gain fast and robust control against uncertainties and unavailable velocity signal. Then, a control barrier function-based quadratic programming is constructed, with which the constrained problem is formulated to generate an optimal guidance law. The fully autonomous operating space can be, therefore, pursued in obstacle-cluttered environments. The stability analysis of the control system is given using the Lyapunov method, and the simulation results are presented to illustrate the effectiveness of the proposed strategy.