Visual object tracking is a crucial task across numerous computer vision applications. However, object tracking algorithms face significant challenges stemming from deformation and fast motion, which frequently incur dramatic changes to the target’s appearance. To address this problem, we propose a Siamese-network-based object tracking method that combines contrastive learning and adaptive template updating. First, we designed a contrastive branch based on a Siamese network, which constructs a contrastive network with a template branch. The objective of this network is to capture the invariance associated with different images of the same target. Subsequently, we implemented an adaptive template updating strategy for the timely capture of appearance changes in targets and adjusted the size and position of the bounding box. Finally, the α-complete intersection over union function was introduced to continuously optimize the generation of bounding boxes during training, guiding the model to produce more accurate tracking boxes and further improving the algorithm performance. The experimental results demonstrate that our algorithm achieves advanced performance on five datasets, namely GOT-10k, LaSOT, UAV123, OTB2015, and NFS.