Abstract
In a partial-monitoring problem in every round a learner chooses an action, simultaneously an opponent chooses an outcome, then the learner suffers some loss and receives some feedback. The goal of the learner is to minimize his (unobserved) cumulative loss. In this paper we explore a variant of this problem where in every round, before the learner makes his decision, he receives some side-information. We assume that the outcomes are generated randomly from a distribution that is influenced by the side-information. We present a “meta” algorithm scheme that reduces the problem to that of the construction of an algorithm that is able to estimate the distributions of observations while producing confidence bounds for these estimates. Two specific examples are shown for such estimators: One uses linear estimates, the other uses multinomial logistic regression. In both cases the resulting algorithm is shown to achieve \(\widetilde O(\sqrt{T})\) minimax regret for locally observable partial-monitoring games.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Abbasi-Yadkori, Y., Szepesvári, Cs.: Regret bounds for the adaptive control of linear quadratic systems. Journal of Machine Learning Research - Proceedings Track (COLT 2011) 19, 1–26 (2011)
Abbasi-Yadkori, Y., Pál, D., Szepesvári, Cs.: Improved algorithms for linear stochastic bandits (extended version). In: NIPS, pp. 2312–2320 (2011), http://www.ualberta.ca/~szepesva/papers/linear-bandits-NIPS2011.pdf
Auer, P.: Using confidence bounds for exploitation-exploration trade-offs. The Journal of Machine Learning Research 3, 422 (2003)
Bartók, G., Pál, D., Szepesvári, Cs.: Minimax regret of finite partial-monitoring games in stochastic environments. Journal of Machine Learning Research - Proceedings Track (COLT 2011) 19, 133–154 (2011)
Bartók, G., Zolghadr, N., Szepesvári, Cs.: An adaptive algorithm for finite stochastic partial monitoring. To appear in ICML (2012)
Cesa-Bianchi, N., Lugosi, G., Stoltz, G.: Regret minimization under partial monitoring. Math. Oper. Res. 31(3), 562–580 (2006)
Dudík, M., Hsu, D., Kale, S., Karampatziakis, N., Langford, J., Reyzin, L., Zhang, T.: Efficient optimal learning for contextual bandits. In: UAI, pp. 169–178 (2011)
Filippi, S., Cappé, O., Garivier, A., Szepesvári, Cs.: Parametric bandits: The generalized linear case. In: NIPS, pp. 586–594 (2010)
Foster, D.P., Rakhlin, A.: No internal regret via neighborhood watch. CoRR, abs/1108.6088 (2011)
Helmbold, D.P., Littlestone, N., Long, P.M.: Apple tasting. Information and Computation 161(2), 85–139 (2000)
Piccolboni, A., Schindelhauer, C.: Discrete Prediction Games with Arbitrary Feedback and Loss. In: Helmbold, D.P., Williamson, B. (eds.) COLT/EuroCOLT 2001. LNCS (LNAI), vol. 2111, pp. 208–223. Springer, Heidelberg (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Bartók, G., Szepesvári, C. (2012). Partial Monitoring with Side Information. In: Bshouty, N.H., Stoltz, G., Vayatis, N., Zeugmann, T. (eds) Algorithmic Learning Theory. ALT 2012. Lecture Notes in Computer Science(), vol 7568. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-34106-9_25
Download citation
DOI: https://doi.org/10.1007/978-3-642-34106-9_25
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-34105-2
Online ISBN: 978-3-642-34106-9
eBook Packages: Computer ScienceComputer Science (R0)