Authentication has become an emerging issue for video streaming over lossy networks. Although the advanced video coding standards, such as H.264/AVC, efficiently reduce the amount of data to be transmitted, the coding dependency brings new challenges in designing efficient stream authentication scheme. In this paper, we propose a novel joint-designed-layered source-channel adaptive scheme that integrates authentication into source and channel coding components to sufficiently use the related information to efficiently address the coding dependency and to design the optimal rate allocation scheme for the sake of end-to-end video quality. The proposed layered framework is able to minimize end-to-end quality degradation incurred by both the wireless channel noise and the authentication failure. In particular, the competing requirements of high verification probability and low authentication overhead are concurrently satisfied by the elegant design of layered hash appending with efficient adaptation to the H.264 source coding and channel conditions. A joint source-channel-authentication rate allocation scheme is then developed to achieve optimal end-to-end video quality. The experimental results on H.264 video sequences confirm the efficacy of this joint adaptive scheme and demonstrate that it indeed outperforms the state-of-the-art graph-based authentication algorithms.