Modern video streaming applications apply adaptive bitrate (ABR) algorithms to enhance user quality of experience (QoE). The existing model-based ABR algorithms failed to generalize to diverse network conditions and personalized QoE objectives due to their fixed control rules. The learning-based ABR algorithms required significant tuning to learn a well-performed model which can cause a QoE degradation during the model testing phase. In this paper, we propose FedABR, a novel ABR algorithm based on personalized federated learning to address the above challenges. To enable clients' local model dealing with network environment changes, we introduce a federated learning approach to train a global model using all the clients' local model without gathering their data together to protect clients' privacy. We also introduced an adaptation phase to train a personalized model for each client to maximize their individual QoE. By jointly training multiple learning tasks with a global model, it has the ability to provide transferable knowledge to supervise bitrate selection, and can be efficiently adapted to a new task in unseen environment with much fewer data samples and training epochs. We implement the proposed FedABR based on an emulation platform which connects to the Linux network protocol stack through a virtual network interface to send real data packets for evaluation. Extensive experiments based on real-world traces show that FedABR achieves the best comprehensive QoE compared with the state-of-the-art ABR algorithms in a variety of network environments.