Federated learning, a distributed machine learning framework, can be used in many Internet of Vehicles (IoV) scenarios to enable privacy-preserving distributed intelligence. While federated learning avoids transmitting raw data in the learning process, it also requires to transmit learning models between clients and server, where the limited wireless resources is always the bottleneck for performance. In this paper, we propose a deep reinforcement learning (DRL) based approach for selecting the best wireless network in a multi-access environment to improve the performance of federated learning. The proposed approach can enhance the overall robustness of the network with efficient network switching based on network environment. We conduct realistic computer simulations to show that the proposed approach exhibits significant performance advantages over existing baselines.