In this paper a fully Bayesian factor analysis model is developed that assumes a very general model for each factor, namely the Gaussian mixture. We discuss the cases where factors are both independent and dependent. In the statistical literature, factor analysis has been used principally as a dimension reduction technique, with little interest in a priori modelling of the factors, but here the application is source separation where the factors may have a direct interpretation and the usual Gaussian model for a factor may not be appropriate. That is the case for the application that illustrates our work, which is that of identifying different sources of extra-terrestrial microwaves from all-sky images taken at different frequencies. In particular there is interest in separating out the cosmic microwave background (CMB) signal from the other sources.