Various intangible cultural expressions in Indonesia such as oral traditions and literature are fragile and easily lost. Currently among 726 languages, 146 are endangered. Although several projects have been initiated for cultural preservation, the available technology that could support communication within indigenous communities, as well as with people outside the community, is still very rare in Indonesia. Speech-to-speech translation is a technology that enables communication among people speaking in different languages, and therefore it is significant for indigenous communities to preserve their cultural language and overcome language barriers. This paper presents the earlier step of long-term development of speech-to-speech translation system from Indonesian ethnic languages to other languages (i.e., English/Indonesian), which is a design and collection of graphemically balanced and parallel speech corpora of four Indonesian major ethnic languages: Javanese, Sundanese, Balinese and Bataks.