Frequency-agile (FA) transmission strategy plays a crucial role in radar antijamming applications. This strategy is usually designed or trained offline, which would lose the advantage of adaptability and flexibility when facing diverse jamming patterns and nonstationary target echoes. Considering the scenarios where a radar detects a target with strong nonstationary scattering characteristics under fast-variant interference, the radar is required to immediately adjust the FA transmission strategy to react to the variation of both jamming signals and target echoes quickly. In order to enhance radar transmission strategy in both generalization and flexibility aspects, an online FA strategy, called Combinatorial Discounted Thompson Sampling, is developed for the antijamming by exploiting the target's scattering change with the multiarmed bandit model. With the advantages of both optimal exploration and fast convergence, the proposed algorithm can efficiently adapt to the scattering fluctuation under dynamic frequency jamming emissions. Experimental comparisons with conventional deep reinforcement learning demonstrate the proposed algorithm's superiority for FA transmission strategy learning to boost radar detection performance under frequency response dynamics when avoiding frequency jamming.