The estimation of state variables plays a significant role in the control of ship-mounted elastic cranes. Elastic-ship mounted crane can show very complex dynamic vibrations during their operations. A vision sensor (i.e. camera) realizing a contactless measurement sensor can be used to measure the deformations. Unfortunately effects like limited accuracy and time delay occur, which are the main inherent problems of vision sensors. The effects and related compensation approaches are studied in this work. The main goal of the work is to develop an approach to combine suitable measurement devices easy to realize with improved reliability. The task to be solved is to combine the estimations of the variable gain observer (based on vision measurements) with those of the variable gain observer (based on potentiometers). Realizing a multi-model approach the variable gain controller uses the estimated states and the roll angle to generate the required damping to control the system. Simulation results show that the variable gain observers can estimate the states and the unknown disturbance acting on the payload very well and the variable gain controller can reduce effectively the payload pendulations.