Using three actuators, in this article, we construct a control system for rubber-tired gantry (RTG) cranes to track three actuated outputs and stabilize two unactuated outputs. The controller is designed by utilizing the fast-terminal sliding mode, fractional calculus, and backstepping. Considering that RTG cranes encounter with uncertainties and random winds, an adaptive mechanism is constituted using neural networks to estimate uncertain parameters and unknown winds. A neural observer that reduces the number of sensors by half is also proposed for approximating feedback velocities. By applying these advanced techniques, the control system is endowed with adaptive and robust features. The quality and effectiveness of the control system are investigated by simulation and experimentation.