In this paper, an automatic Computer Aided Diagnosis (CAD) system is completely designed for breast cancer diagnosis and it is verified on a publicly available mammogram dataset constructed during Image Retrieval in Medical Applications (IRMA) project. This database comprises three different patch types indicating the health status of a person. These types are normal, benign cancer, and malignant cancer and they are labeled by the radiologists for the IRMA project. In the realization of CAD system, all mammogram patches are firstly preprocessed performing a histogram equalization followed by Non-Local Means (NLM) filtering. Then, the Local Configuration Pattern (LCP) algorithm is performed for feature extraction. Besides, some statistical and frequency-domain features are concatenated to LCP-based feature vectors. The obtained new feature ensemble is used with four well-known classifiers which are Fisher's Linear Discriminant Analysis (FLDA), Support Vector Machines (SVM), Decision Tree, and k-Nearest Neighbors (k-NN). A maximum of 94.67% recognition accuracy is attained utilizing the new feature ensemble whereas 90.60% was found if only LCP-based feature vectors are used. This consequence obviously reveals that the new feature ensemble is more representative than an LCP-based feature vector by itself.