We present a system that allows drivers and fleet managers to interact with their connected vehicles both by means of direct control and indirect goal-setting. The ability to move data from vehicles to a remote server is established by the flexible and secure open vehicle telematics platform “CloudThink.” Based on this platform, we present several prototypes of how people can be enabled to conveniently interact with connected vehicles: First, we demonstrate a system that allows users to select and interact with vehicles using object recognition methods and automatically generated user interfaces on smartphones or personal wearable devices. Second, we show how functional semantic metadata can be used to smooth the boundaries for interacting with vehicles in the physical and virtual worlds. Finally, we present a method for monitoring interactions between vehicles and remote services which increases safety and security by enhancing driver oversight and control over the data that leaves and enters their vehicle.