Reinforcement Learning (RL) is a machine learning technique that deals with linear and nonlinear systems without necessarily knowing their exact dynamic models. It can learn to bring the system states directly and quickly to any reachable point, but has no convergence guarantees. On the other hand, the Sliding Mode Control (SMC) approach is a robust control technique based on variable structure systems that can handle parametric uncertainties and external disturbances, provably driving system trajectories to the vicinity of the origin. However, tuning the control gains and the sliding surface parameters is not straightforward. This work presents a methodology for merging RL and SMC approaches in a unified intelligent control technique to ensure system stability and robustness to perturbations and modeling inaccuracies. In contrast to previous work, the control gains are tuned using the same technique as the RL actor. The newly proposed controller is applied to the swing up and stabilization of the inverted pendulum system for validation purposes. Its performance is then compared to its component parts: Twin Delayed DDPG (TD3) and first-order SMC. Simulation results are presented to demonstrate the effectiveness and feasibility of the proposed methodology.