This paper addresses a resource allocation strategy for network slices, where each network slice supports a different federated learning task. A slice is established when a new federated learning model needs to be trained and is released once the training is complete. The goal is to minimize the average network slice holding time while also providing fairness between slice tenants and improving network efficiency. We propose a reinforcement learning-based strategy to periodically reallocate resources according to the current state of each federated learning task. We offer two reinforcement learning models. The first model achieves more stable performance and considers correlations between tasks, while the second model utilizes fewer parameters and is more robust to varying number of tasks. Both approaches have better performance than baseline heuristic methods. We also propose a method to alleviate the effect of various resources scales to make the training stable.