Self-driving cars rely on a plethora of algorithms in order to perform safe driving manoeuvres. Training those models is expensive (e.g. hardware cost, storage, energy) and requires continuous updates. This paper proposes a cloud-based framework for continuous training of self-driving AI models. In addition to training standalone models, the framework is capable of leveraging pre-trained models in expediting the training on environment changes (e.g. new driver or new car model). As use-case, this paper focuses on a driver's behaviour while the vehicle's control is being transferred between the driver and the self-driving AI. A human driver can hand over the control of a vehicle's driving tasks to an automated system, when that system's confidence level is high enough. Reciprocally, there are situations where that control has to be handed back to the human driver. This paper proposes a novel real-time system for Driving Not-Driving (DND) detection, which is able to capture the ability of the driver to re-take control of a vehicle when the automated driving system transitions from a higher to a lower level of automation (e.g. L3 to L2 vehicle automation). We are using a computer vision-based Driver Monitoring System (DMS) that captures in real-time head and eye movements. These are captured in the car and transferred to the cloud where a DND model is trained for a specific driver. The DND classification model is deployed in the vehicle and predicts if the driver is ready or not to resume control at a given time. The cloud-based framework proposed in this paper shows an end-to-end cycle of collecting, training and deploying self-driving AI technology, with the additional features of continuous and transfer learning.