In this paper, a computer-aided diagnosis (CAD) system for early diagnosis of prostate cancer from diffusion-weighted magnetic resonance imaging (DWI) is proposed. The proposed system begins with defining a region of interest that contains the prostate across the various slices of the input volume. Then, the apparent diffusion coefficient (ADC) of the defined region is calculated, normalized and refined. Finally, the classification of prostate into either benign or malignant is performed through two stages. In the first stage, seven convolutional neural networks (CNNs) are utilized to get initial probabilities for each case. Then, a random forest (RF) classifier uses these probabilities a s input to decide the final diagnosis. The proposed system is a novel system in the sense that it has the ability to detect prostate cancer without any prior processing (e.g., the segmentation of the prostate region). Evaluation of the developed system is done using DWI datasets collected at seven different b-values from 32 patients (16 benign and 16 malignant). The acquisition of these DWI datasets is performed using two different scanners with different magnetic field strengths (1.5 Tesla and 3 Tesla). The resulting accuracy of the proposed system after the second stage of classification shows a good performance close to the performance of up-to-date systems.