B-mode ultrasound (BUS) imaging is widely used for diagnosis of breast tumors. In recent years, another ultrasound modality, namely ultrasound elastography (UE), has also been successfully applied in clinical practice. Although the combination of bimodal ultrasound imaging can improve the diagnosis accuracy for breast tumors, the single-modal ultrasound-based diagnosis is more popular in clinical practice, especially in the rural areas. Since transfer learning (TL) can transfer knowledge from a source domain to a target domain to help train more effective model, we propose to develop a single-modal ultrasound-based computer-aided diagnosis (CAD) with the transferred information from an additional modality in this work. We propose a projective model (PM) based multilayer kernel extreme learning machine (ML-KELM-PM) algorithm, which performs the parameter transfer approach in classifier to perform TL in CAD. The experimental results on a bimodal ultrasound image dataset show that the proposed ML-KELM-PM algorithm outperforms all the compared algorithms for the single-modal ultrasound-based CAD for breast tumors.