Shared autonomous vehicles (SAV) have the potential to have a transformative impact on future urban transportation. In this study, we analyze the qualitative and quantitative effects of the introduction of SAV to commuters traveling in a central Tokyo district with a well-developed rail network using the large-scale agent-based transport simulator MATSim. First, we conducted a scenario in which no SAV were introduced and reproduced the modal split error for Tokyo within 1.2%. We then conducted multiple scenarios, varying the number of SAV and fare of SAV, to examine the impact on the mode share for number of vehicles introduced and fare of SAV. The results demonstrated that approximately 14%–32% of the population would shift to SAV, and those who traveled 2.0–8.0 kilometers by rail or bicycle were likely to shift to SAV and fare of SAV do not affect mode share of SAV. The results also confirmed that if the total number of SAV is increased too much, it is possible that people who used to walk or bicycle, that is, those using modes of transportation that have a smaller environmental effect and improve health, will shift to SAV. This suggests that it is important to limit the total number of SAV to an appropriate amount from the perspectives of environmental impact and health impact.