This paper proposes a novel joint source-channel adaptive scheme that integrates the authentication into source and channel coding components to achieve 100% effective verification probability and an optimal end-to-end video quality. By jointly considering source coding and channel conditions with authentication, the proposed layered framework is able to minimize end-to-end quality degradation incurred by both wireless channel noise and authentication failure. In particular, the competing requirements of high verification probability and low authentication overhead are concurrently satisfied by elegant design of hash appending with efficient adaptation to H.264 source coding and channel conditions. A joint source-channel-authentication rate allocation scheme is then developed to achieve optimal end-to-end video quality. Experiment results on H.264 video sequences confirm the efficacy of this joint adaptive scheme and demonstrate that it indeed outperforms the state-of-the-art graph based authentication algorithms.