In this paper, we propose a novel semi-supervised methodology to detect spam or ham SMSs, using frequent item set mining algorithm Apriori, probabilistic model Naive Bayes and ensemble learning. This paper considers the unbalanced data set problem which means designing of two class SMS classifier using small number of ham and unlabeled dataset only. Using only a few labeled examples with Semi-supervised training is typically unreliable. However, by applying user-specified minimum support and minimum confidence on ham and unlabeled dataset, we gained significant accuracy on classifying SMSs, experimenting on UCI data Repository.