AI-Assisted Translation and Speech Synthesis for Community Services Supporting Limited English Proficient Individuals

Abstract

When high school students volunteer at nursing homes, churches, and medical facilities, they often encounter individuals with limited English proficiency (LEP), highlighting a critical need for communication in their native languages. Our study aims to develop an AI system capable of translating English sentences into multiple foreign languages (Spanish, Japanese, Korean, Filipino, and Chinese) and generating corresponding synthetic speech. We utilized Google Translate for text translation and Microsoft Azure’s text-to-speech studio for synthetic speech production. To assess the AI-generated speech, we had five native speakers evaluate its word accuracy, translation quality, and fluency using a 5-point Likert scale: strongly agree (2), agree (1), neutral (0), disagree (−1), and strongly disagree (−2). The predominance of evaluation scores above 1 suggests that the AI system is generally effective for its intended purpose. However, we did note instances of unsatisfactory performance in certain languages, as indicated by Likert scores of −1. Leveraging machine translation and synthetic speech technologies significantly aids in bridging language gaps and enhancing multilingual communication. Nevertheless, human oversight is necessary to rectify occasional errors. Looking ahead, our goal is to launch a comprehensive online service dedicated to assisting LEP individuals in accessing community services.