In real-world networks, we regularly face the effect of propagating failures over networks, for example, rumors spread over social networks, outages spread over power networks, viruses spread over communication and biological networks. Often, these failures spread over a network of agents with unknown and potentially diverse degrees of vulnerabilities to the propagating phenomenon. In this work, we consider a general network model subject to propagating failures and develop provably fast mechanisms for learning the unknown vulnerabilities of the network with minimal cost incurred in the process. We propose an extension to the classic Independent Cascade (IC) model where we incorporate both node and edge failures with non-uniform costs. From an online learning perspective, the goal is to find an optimal policy to control where to start failures and generate samples. Therefore, we formulate a cost minimization problem with Probably-Approximately-Correct (PAC) type guarantees. As a theoretical benchmark, we design a linear programming problem using a proposed joint Bernstein inequality. Then we characterize the performance of randomized policies that use a fixed budget distribution independent of sampling history. Finally, we propose a fast Lyapunov-based online learning policy, for which we give a formal theoretical analysis. The performance of the policy are validated under extensive numerical studies for both synthetic and real-world networks.