This paper addresses the problem of the meeting-point-based electric demand responsive transport routing and charging scheduling under charging synchronization constraints. The problem considered exhibits a similar structure with the location-routing problem, which is more difficult to solve than many electric vehicle routing problems with capacitated charging stations. We propose to model the problem using a mixed integer linear programming approach based on a layered graph structure. A two-stage simulated annealing based algorithm is proposed to solve the problem efficiently. The algorithm is tested on 20 instances with up to 100 customers and 49 bus stops. The results show that the proposed algorithm outperforms the best solutions found by a commercial mixed-integer linear programming solver set with a 2-hour computational time limit for 12/20 test instances and with less than 1-minute computational time on average.