This letter proposes a waiting-time-dependent semi-Markov (WTDM) switching based multiple model multiple hypothesis (M³H) filter to track a target in the presence of outliers. Two models, namely, a normal noise model and an outlier model, are constructed in the semi-Markov system. The adaptive transition probabilities are derived as the functions of the waiting time of outliers (or the interval of outlier occurrences), and this waiting time is treated as a discrete random variable governed by an exponential pmf. Performance of the WTDM-based M³H filter is demonstrated through simulation experiments. The proposed WTDM-based M³H filter outperforms the existing interacting multiple model (IMM) filter, which was proposed for the same purpose.