Nowadays, when the vast majority of biomedical research relies on machine learning methods, paying attention to the meaningfulness of the data we work with is crucial. This is especially true if the dataset is scarce and we are required to use various augmentation techniques to enlarge training sets. The additional data are, however, not guaranteed to have the same characteristics as the original data, and therefore, the augmented set may be inconsistent. This can subsequently lead to incorrect training of biomedical image analysis methods, which may result in biased classification, detection, segmentation, or tracking results.In this paper, we present an online tool called COMPYDA¹, that allows users to easily assess the similarity of a pair of datasets using well-founded, commonly used statistic methods. COMPYDA guides users through univariate and multivariate analyses and helps them understand and explain dataset differences to ascertain a compatible dataset for further training. The tool is available at: https://cbia.fi.muni.cz/compyda/