Packet networks are agnostic to applications, which have served to keep the Internet infrastructure simple and scalable over the past several decades. However, the best-effort model is now seen as an inhibitor to meeting user experience expectations for the diverse applications such as streaming video, gaming, browsing, and social media. Current methods for prioritization of certain application types are static, and do not react to changes in network conditions or user experience. We envisage a self-driving network that is able to continuously monitor user experience and intervenes to assist applications as and when needed. Our contributions are: (1) We propose a self-driving network architecture that directly measures, optimizes, and dynamically controls application performance. We develop a method to measure and model application state in real-time using network behavior data. (2) We apply our framework to two representative applications, video streaming and gaming, and show how the network can detect application deterioration in terms of playback buffers and ping latency respectively, and apply remedial action to improve application performance without requiring any explicit signaling.