Due to the high efficiency of discriminative correlation filter (DCF), it has attracted widespread attention in the field of UAV object tracking. To handle the problem of filter degradation, many trackers usually introduce temporal regularization to enhance the discriminative power of the filter. However, these temporal regularization methods only utilize the limited information between two consecutive frames, which are susceptible interference by previous corrupted information. Besides, regularization terms with predefined hyperparameters can not well adapt to the variations of the target across sequent frames, which may cause the model degradation or drift. We propose a tracker based on DCF framework to fully exploit the information hidden in the historical response map, namely adaptive weighted response consistency-based DCF tracking. Specifically, carefully selected historical response maps with fixed weight distribution are introduced in training phase to increase the robustness of the filter. Further, we present a unified loss for jointly learning the filter and the weight distribution, which can be solved by the alternate convex search method. The joint loss guarantees that reliable response maps contribute more to filter learning, leading to a more discriminative and adaptive filter for tracking the target. Extensive experiments show that the proposed method achieves state-of-the-art results on two datasets.