In this paper, we propose a master-slave SoC structure composed of an ARM7-TDMI and a co-processor for Mahalanobis distance calculation. The SoC was implemented on an Actel ProASIC series M7A3P1000 FPGA. Furthermore, we implement a HMM based speech recognition system based on this SoC. Compared with the conventional ASIC co-processor and slave SoC structure, the new master-slave structure reduces the number of SRAM access and improves the bus efficiency. Experiment results show that with 1.40s Chinese speech “feixi” and 24MHz clock, the processing time of the M-S SoC system is 1.85s, a 64.12% reduction compared with the software implement on ARM7-TDMI, and a 5.95% reduction compared with slave structure SoC.