We investigate a remote estimation system with communication cost for multiple Internet-of-Things sensors, in which the state of each sensor changes according to a Wiener process. Under sublinear communication cost structure, in which the per-transmission cost decreases with the number of simultaneous transmissions, we address an interesting unexplored trade-off under source dynamics between frequent updates of a smaller number of sensors at a higher cost and sporadic updates of a larger number of sensors at a lower cost. We first suggest two benchmark strategies, an all-at-once policy and a multi-threshold policy, and generalize them to a unified framework, called the MAX- $k$ policy. Furthermore, we address the problem of parameter optimization of the MAX- $k$ policy by developing online learning algorithms with stochastic feedback and a continuous search space. Through simulations, we demonstrate that the joint solution of the MAX- $k$ policy and particle swarm optimization-based online learning achieves a high performance, outperforming the well-known upper confidence bound-based competitor.