As the world becomes evermore connected and instrumented, decision-makers have ever more rapid access to ever changing and growing streams of data - but this makes the decision-maker's problems ever more complex as well, because it is impossible to learn everything in the time frame in which decisions must be made. What the decision-maker must do, therefore, is to discover in real time what is relevant in the enormous stream of data and use the relevant information to make good decisions. This talk presents a systematic framework and associated algorithms that enable a decision-maker to do this. The algorithms we propose yield strong performance guarantees for both the long run and the short run. The applications are numerous and include patient monitoring, online recommendation systems, social networks, targeted advertisement, surveillance, network security, finance etc.