This paper introduces the novel concept of using Set-Valued Observers (SVOs) in Fault Detection and Isolation (FDI), for discrete-time linear time-varying systems. The proposed method relies on SVO-based model invalidation to discard models that are not compatible with the input/output data. We argue that there are mainly three significant advantages of using SVOs for FDI, when compared to the most common strategies available in the literature: i) under suitable conditions, we can guarantee that there will be no false alarms; ii) unlike residual-based architectures, the proposed technique does not require the computation of a threshold to declare faults; iii) the SVOs can be used with a wide class of time-varying linear uncertain discrete-time systems. We further show, in simulation, that the proposed FDI algorithm in general requires a very small number of iterations to detect and identify a faulty behavior.