Millimeter wave (mmWave) technology provides abundant high-capacity channel resources for vehicular communications. However, the mobility of vehicles and the blocking effect of mmWave propagation brings new challenges to communication security. From the perspective of cooperative secure communication, this paper proposes a deep reinforcement learning (DRL)-based joint relay and jammer selection scheme in mmWave vehicular networks. The mmWave base station selects idle vehicles as relay transmission nodes to overcome the severe blocking attenuation of the multi-user downlink legitimate transmissions. Moreover, to ensure secure transmission, a cooperative vehicle is selected to transmit jamming signals to the eavesdropper while the users are not disturbed. We utilize the asynchronous advantage actor-critic (A3C) learning algorithm to optimize the cooperative vehicle selection with the objective of maximizing the total secrecy capacity. Besides, we set the secrecy rate punishment mechanism to guarantee the secrecy performance of each vehicle. We demonstrate that the proposed scheme can rapidly adapt to the highly dynamic vehicular networks and effectively improve secrecy performance.