Present day distributed inference systems consist of sensors with different modalities working as a system to perform specific tasks. With multiple sensors sensing heterogeneous data over multiple time instants, diversity is an inherent aspect of such systems. In this work, we take the first step to characterize the diversity of a general heterogeneous sensing system performing inference tasks. We provide a unified definition for diversity which can be customized for the system in use. The use of the definition is illustrated by applying it to a specific detection system where the sensors collect data over heterogeneous sensing channels. We assume the data to be both temporally and spatially correlated and analyze the effect of dependence on the diversity of the detection system.