To provide speech prostheses for individuals with severe communication impairments, we investigated a classification method for brain computer interfaces (BCIs) using silent speech. Event-related potentials (ERPs) obtained when four subjects imagined the vocalization of Japanese vowels, /a/, /i/, /u/, /e/, and /o/ in order and in random order while the subjects remained silent and immobilized were recorded using 111 scalp electrodes and 3 reference electrodes. Regarding detection of the imagined voice, some problems occurred by which the related brain geometries and suitable electrodes for classifications differed between subjects. To overcome these problems, we used an adaptive collection that divided ERP data into small elements, performed evaluation relative to the elements, and selected better elements for classification. In earlier reports of studies using the common spatial patterns (CSPs) filter and support vector machines (SVMs), the classification accuracies (CAs) were 56-72% for the pairwise classification /a/ vs. /u/ in the case of 63 channel EEG measurement. In this study, the CA was improved to 73-92% using the adaptive collection. According to the CA, 19 channel measurements were worse than 111 channel measurements, but 63 channel measurements were slightly worse that 111 channel measurements. Using 63 channel measurements, 73% of CA was achieved for all pairwise combinations of the five vowels. The average of the CAs was 85%. These results show that the proposed method exhibited great potential for use in classification of imagined voice for a speech prosthesis controller.