The aim of this paper is to face one of the main problems in the control of wastewater treatment plants (WWTPs). It appears that the control system does not respond as it should because of changes on influent load or flow. In that case, it is required that a plant operator tunes up the parameters of the plant. The dissolved oxygen setpoint is one of those parameters. In this paper, we present a model-free reinforcement learning agent that autonomously learns to actively tune up the oxygen setpoint by itself. By active, we mean continuous, minute after minute, tuning up. By autonomous and adaptive, we mean that the agent learns just by itself from its direct interaction with the WWTP. This agent has been tested with data from the well-known public benchamark simulation model no. 1, and the results that are obtained allow us to conclude that it is possible to build agents that actively and autonomously adapt to each new scenario to control a WWTP.