Although federated learning (FL) has become a privacy-preserving machine learning paradigm that enables a collaborative form of tiny machine learning (TinyML) on edge devices, unfairness may arise when the performance of the global model varies due to heterogeneous devices and data. Existing works mainly focus on improving fairness in a single time slot, often ignoring the temporal coupling of FL in TinyML. To tackle this issue, we introduce a novel long-term fairness-aware model aggregation mechanism, named FedLV, which aims to reduce the accuracy distribution variance by considering the FL process as a whole. Specifically, we introduce the long-term fairness criterion as well as the long-term fairness problem into the design of FedLV for FL. To promote the global model’s performance, we quantify the long-term guarantee of clients’ contributions, and transform the long-term fairness problem into a queue stability problem via Lyapunov optimization. Since the global model’s accuracy distribution is unmeasurable before model aggregation, we further propose a prior estimation solver to derive an approximately optimal solution and provide theoretical proof of the solver’s convergence. Extensive experiments conducted on four real-world datasets demonstrate that the accuracy distribution variance of FedLV is at least 12% lower than that of both FedAvg and q-FFL.