The electrohydraulic elastic manipulator (EEM) includes an electrohydraulic servo system and a variable stiffness mechanism. It is a complex system with nonlinear elements, such as friction, parametric uncertainties, and modeling error as well as coupling torques in the elastic joint. This paper focuses on developing an equilibrium position controller for the EEM with the presence of the matched and unmatched uncertainties. The proposed controller is based on a backstepping scheme optimized using an adaptive sliding mode technique and an adaptive nonlinear PI structure. To estimate parameters of known functions in the system dynamics, the parametric learning laws are employed, while adaptive switching gain laws are used to adjust robust gains of discontinuous control terms. The robust switching gains increase or decrease as state control variables move away or close to the surfaces, respectively. Working together, these kinds of adaptation laws can effectively deal with the problem of matched and unmatched uncertainties. Next, the Lyapunov approach is used to prove the robustness and stability of the whole system. The proposed method is practically implemented and compared with other controllers to demonstrate the effectiveness of the proposed method with the variant stiffness and the uncertainties.