Passenger comfort is a crucial aspect that influences humans’ acceptance of automated vehicles. The passenger comfort score (PCS) is closely related to the passengers’ psychological states, however, comfort quantification methods based on the passengers’ psychophysics mechanism are rare. This research pioneers a passenger comfort quantification model (PCQM) specifically designed for automated vehicles, demonstrating the model’s ability to accurately quantify subjective PCS under urban LCS. Three significant contributions form the basis of this study: 1) A dataset dedicated to comfort quantification is collected. A novel PCQM based on ensemble learning of psychophysics mechanism based sub-model and encoder-transformer based sub-model is proposed. The psychophysics mechanism model is derived from Stevens’ power law. 2) As a subjective indicator, the self-reported score (SRS), which is the indicator of PCS contains considerable noise. The PCQM addresses the issue of substantial noise prevalent in the subjective SRS by incorporating a semi-supervised learning strategy, which enhances data consistency and suppresses noise. 3) The efficacy of the proposed PCQM is corroborated via deployment on an automated vehicle, where the model’s real-time predictions strongly align with SRS from onboard passengers.