We study the convergence rate of consensus algorithms in a Network of Networks model. In this model, there is a collection of networks, and these individual networks are connected to one another using a small number of links to form a composite graph. We consider a setting where the links between networks are costly to use, and therefore, are used less frequently than links within each network. We model this setting using a stochastic system where, in each iteration, the inter-network links are active with some small probability. U sing spectral perturbation theory, we analyze the convergence rate of this system, up to first order in this activation probability. Our analysis shows that the convergence rate is independent of the topologies of the individual graphs; the rate depends only on the number of nodes in each graph and the topology of the connecting edges. We further highlight these theoretical results through numerical examples.