Gabriele Sottocornola
Markus Zanker
ABSTRACT
Post-harvest diseases of apple are one of the major issues in the economical sector of apple production, causing severe economical losses to producers. Thus, we developed DSSApple, a picture-based decision support system able to help users in the diagnosis of apple diseases. Specifically, this paper addresses the problem of sequentially optimizing for the best diagnosis, exploiting past interactions with the system and their contextual information (i.e., the evidence provided by the users), while exploring the set of candidate diseases. This online learning problem is commonly addressed in the literature through a stochastic active learning paradigm - i.e., Contextual Multi-Armed Bandit (CMAB). The methodology interactively updates the decision model considering the success of each past interaction with respect to the context provided in each round. However, contextual information is very often partial and inadequate to handle such a complex decision making problem. On the other hand, human-made decisions implicitly include unobserved factors (referred to as unobserved confounders) that significantly influence their choices. In this paper, we take advantage of the information embedded in the observed human decisions to marginalize confounders and improve the capability of the CMAB model to identify the correct diagnosis. Specifically, we propose a Counterfactual Thompson Sampling (CF-TS), a CMAB model based on the causal concept of counterfactual. The proposed model is validated with offline experiments based on data collected through a large user study on DSSApple application. The results prove that CF-TS is able to significantly outperform both traditional CMAB algorithms and observed user decisions, in the real-world task of predicting the correct apple disease.
- Alekh Agarwal, Daniel Hsu, Satyen Kale, John Langford, Lihong Li, and Robert Schapire. 2014. Taming the Monster: A Fast and Simple Algorithm for Contextual Bandits(Proceedings of Machine Learning Research, Vol. 32), Eric P. Xing and Tony Jebara (Eds.). PMLR, Bejing, China, 1638–1646. http://proceedings.mlr.press/v32/agarwalb14.htmlGoogle Scholar
- Shipra Agrawal and Navin Goyal. 2013. Thompson Sampling for Contextual Bandits with Linear Payoffs. In Proceedings of the 30th International Conference on International Conference on Machine Learning - Volume 28 (Atlanta, GA, USA) (ICML’13). JMLR.org, III–1220–III–1228.Google Scholar
- Peter Auer, Nicolò Cesa-Bianchi, and Paul Fischer. 2002. Finite-Time Analysis of the Multiarmed Bandit Problem. 47, 2–3 (May 2002), 235–256. https://doi.org/10.1023/A:1013689704352Google ScholarDigital Library
- Elias Bareinboim, Andrew Forney, and Judea Pearl. 2015. Bandits with Unobserved Confounders: A Causal Approach. In Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 1 (Montreal, Canada) (NIPS’15). MIT Press, Cambridge, MA, USA, 1342–1350.Google Scholar
- Léon Bottou, Jonas Peters, Joaquin Quiñonero Candela, Denis X. Charles, D. Max Chickering, Elon Portugaly, Dipankar Ray, Patrice Simard, and Ed Snelson. 2013. Counterfactual Reasoning and Learning Systems: The Example of Computational Advertising. J. Mach. Learn. Res. 14, 1 (Jan. 2013), 3207–3260.Google Scholar
- Olivier Chapelle and Lihong Li. 2011. An Empirical Evaluation of Thompson Sampling. In Proceedings of the 24th International Conference on Neural Information Processing Systems (Granada, Spain) (NIPS’11). Curran Associates Inc., Red Hook, NY, USA, 2249–2257.Google ScholarDigital Library
- Maria Dimakopoulou, Zhimei Ren, and Zhengyuan Zhou. 2021. Doubly-Adaptive Thompson Sampling for Multi-Armed and Contextual Bandits. arxiv:2102.13202 [cs.LG]Google Scholar
- Maria Dimakopoulou, Zhengyuan Zhou, Susan Athey, and Guido Imbens. 2019. Balanced Linear Contextual Bandits. In The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI 2019, The Thirty-First Innovative Applications of Artificial Intelligence Conference, IAAI 2019, The Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019, Honolulu, Hawaii, USA, January 27 - February 1, 2019. 3445–3453. https://doi.org/10.1609/aaai.v33i01.33013445Google ScholarDigital Library
- Miroslav Dudík, John Langford, and Lihong Li. 2011. Doubly Robust Policy Evaluation and Learning. In Proceedings of the 28th International Conference on International Conference on Machine Learning (Bellevue, Washington, USA) (ICML’11). Omnipress, Madison, WI, USA, 1097–1104.Google ScholarDigital Library
- Andrew Forney, Judea Pearl, and Elias Bareinboim. 2017. Counterfactual Data-Fusion for Online Reinforcement Learners. In Proceedings of the 34th International Conference on Machine Learning - Volume 70 (Sydney, NSW, Australia) (ICML’17). JMLR.org, 1156–1164.Google Scholar
- Sham M. Kakade, Shai Shalev-Shwartz, and Ambuj Tewari. 2008. Efficient Bandit Algorithms for Online Multiclass Prediction. In Proceedings of the 25th International Conference on Machine Learning (Helsinki, Finland) (ICML ’08). Association for Computing Machinery, New York, NY, USA, 440–447. https://doi.org/10.1145/1390156.1390212Google ScholarDigital Library
- John Langford and Tong Zhang. 2007. The Epoch-Greedy Algorithm for Contextual Multi-Armed Bandits. In Proceedings of the 20th International Conference on Neural Information Processing Systems(Vancouver, British Columbia, Canada) (NIPS’07). Curran Associates Inc., Red Hook, NY, USA, 817–824.Google ScholarDigital Library
- Sanghack Lee and Elias Bareinboim. 2018. Structural Causal Bandits: Where to Intervene?. In Proceedings of the 32nd International Conference on Neural Information Processing Systems (Montreal, Canada) (NIPS’18). Curran Associates Inc., Red Hook, NY, USA, 2573–2583.Google Scholar
- Sanghack Lee and Elias Bareinboim. 2019. Structural Causal Bandits with Non-Manipulable Variables. Proceedings of the AAAI Conference on Artificial Intelligence 33, 01 (Jul. 2019), 4164–4172. https://doi.org/10.1609/aaai.v33i01.33014164Google ScholarDigital Library
- Lihong Li, Wei Chu, John Langford, and Robert E. Schapire. 2010. A Contextual-Bandit Approach to Personalized News Article Recommendation. In Proceedings of the 19th International Conference on World Wide Web (Raleigh, North Carolina, USA) (WWW ’10). Association for Computing Machinery, New York, NY, USA, 661–670. https://doi.org/10.1145/1772690.1772758Google ScholarDigital Library
- Yangyi Lu, Amirhossein Meisami, Ambuj Tewari, and William Yan. 2020. Regret Analysis of Bandit Problems with Causal Background Knowledge. In Proceedings of the Thirty-Sixth Conference on Uncertainty in Artificial Intelligence, UAI 2020, virtual online, August 3-6, 2020. 77. http://www.auai.org/uai2020/proceedings/77_main_paper.pdfGoogle Scholar
- P. Maxin, M. Williams, and R. W. Weber. 2014. Control of Fungal Storage Rots of Apples by Hot-Water Treatments: A Northern European Perspective. Erwerbs-Obstbau 56(2014), 25–34.Google ScholarCross Ref
- Armin Niederkofler, Sanja Baric, Giancarlo Guizzardi, Gabriele Sottocornola, and Markus Zanker. 2019. Knowledge Models for Diagnosing Postharvest Diseases of Apples. In Proceedings of the Joint Ontology Workshops 2019 Episode V: The Styrian Autumn of Ontology, Graz, Austria, September 23-25, 2019(CEUR Workshop Proceedings, Vol. 2518). CEUR-WS.org. http://ceur-ws.org/Vol-2518/paper-ODLS6.pdfGoogle Scholar
- Maximilian Nocker, Gabriele Sottocornola, Markus Zanker, Sanja Baric, Greice Amaral Carneiro, and Fabio Stella. 2018. Picture-Based Navigation for Diagnosing Post-Harvest Diseases of Apple. In Proceedings of the 12th ACM Conference on Recommender Systems (Vancouver, British Columbia, Canada) (RecSys ’18). Association for Computing Machinery, New York, NY, USA, 506–507. https://doi.org/10.1145/3240323.3241616Google ScholarDigital Library
- Sandeep Pandey, Deepayan Chakrabarti, and Deepak Agarwal. 2007. Multi-Armed Bandit Problems with Dependent Arms. In Proceedings of the 24th International Conference on Machine Learning (Corvalis, Oregon, USA) (ICML ’07). Association for Computing Machinery, New York, NY, USA, 721–728. https://doi.org/10.1145/1273496.1273587Google ScholarDigital Library
- Judea Pearl. 2000. Causality: Models, Reasoning, and Inference. Cambridge University Press, USA.Google ScholarDigital Library
- Ilaria Pertot, Tsvi Kuflik, Igor Gordon, Stanley Freeman, and Yigal Elad. 2012. Identificator: A Web-Based Tool for Visual Plant Disease Identification, a Proof of Concept with a Case Study on Strawberry. Comput. Electron. Agric. 84 (June 2012), 144–154. https://doi.org/10.1016/j.compag.2012.02.014Google ScholarDigital Library
- Mahsa Shahbandeh. 2021. Global fruit production in 2019, by selected variety (in million metric tons). hhttps://www.statista.com/statistics/264001/worldwide-production-of-fruit-by-variety/. Accessed: 2021-05-10.Google Scholar
- Ruiyang Song, Stefano Rini, and Kuang Xu. 2021. Hierarchical Causal Bandit. arxiv:2103.04215 [stat.ML]Google Scholar
- Gabriele Sottocornola, Sanja Baric, Fabio Stella, and Markus Zanker. 2021. Case Study on the Development of a Recommender for Apple Disease Diagnosis with a Knowledge-based Bayesian Network (Long paper). In Joint Workshop Proceedings of the 3rd Edition of Knowledge-aware and Conversational Recommender Systems (KaRS) and the 5th Edition of Recommendation in Complex Environments (ComplexRec) co-located with 15th ACM Conference on Recommender Systems (RecSys 2021), Virtual Event, Amsterdam, The Netherlands, September 25, 2021(CEUR Workshop Proceedings, Vol. 2960), Vito Walter Anelli, Pierpaolo Basile, Tommaso Di Noia, Francesco Maria Donini, Cataldo Musto, Fedelucio Narducci, Markus Zanker, Himan Abdollahpouri, Toine Bogers, Bamshad Mobasher, Casper Petersen, and Maria Soledad Pera (Eds.). CEUR-WS.org. http://ceur-ws.org/Vol-2960/paper13.pdfGoogle Scholar
- Gabriele Sottocornola, Maximilian Nocker, Fabio Stella, and Markus Zanker. 2020. Contextual Multi-Armed Bandit Strategies for Diagnosing Post-Harvest Diseases of Apple. In Proceedings of the 25th International Conference on Intelligent User Interfaces (Cagliari, Italy) (IUI ’20). Association for Computing Machinery, New York, NY, USA, 83–87. https://doi.org/10.1145/3377325.3377531Google ScholarDigital Library
- Richard S. Sutton and Andrew G. Barto. 2018. Reinforcement Learning: An Introduction(second ed.). The MIT Press. http://incompleteideas.net/book/the-book-2nd.htmlGoogle ScholarDigital Library
- Turner B Sutton, Herb S Aldwinckle, Art Agnello, and James F Walgenbach (Eds.). 2014. Compendium of Apple and Pear Diseases and Pests (2 ed.). APS press.Google Scholar
- Adith Swaminathan and Thorsten Joachims. 2015. Counterfactual Risk Minimization: Learning from Logged Bandit Feedback. In Proceedings of the 32nd International Conference on International Conference on Machine Learning - Volume 37 (Lille, France) (ICML’15). JMLR.org, 814–823.Google Scholar
- Adith Swaminathan and Thorsten Joachims. 2015. The Self-Normalized Estimator for Counterfactual Learning. In Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 2 (Montreal, Canada) (NIPS’15). MIT Press, Cambridge, MA, USA, 3231–3239.Google Scholar
- Joannès Vermorel and Mehryar Mohri. 2005. Multi-Armed Bandit Algorithms and Empirical Evaluation. In Proceedings of the 16th European Conference on Machine Learning (Porto, Portugal) (ECML’05). Springer-Verlag, Berlin, Heidelberg, 437–448. https://doi.org/10.1007/11564096_42Google ScholarDigital Library
- Daniel Workman. 2020. Apples Exports by Country. http://www.worldstopexports.com/apples-exports-by-country/. Accessed: 2021-05-10.Google Scholar
- Junzhe Zhang and Elias Bareinboim. 2017. Transfer Learning in Multi-Armed Bandits: A Causal Approach. In Proceedings of the 26th International Joint Conference on Artificial Intelligence (Melbourne, Australia) (IJCAI’17). AAAI Press, 1340–1346.Google ScholarCross Ref
Recommendations
Contextual multi-armed bandit strategies for diagnosing post-harvest diseases of apple
IUI '20: Proceedings of the 25th International Conference on Intelligent User InterfacesThis paper describes a decision support system to fulfill the diagnosis of post-harvest diseases in apple fruit. The diagnostic system builds on user feedback elicitation via multiple conversational rounds. The interaction with the user is conducted by ...
Picture-based and conversational decision support to diagnose post-harvest apple diseases
AbstractThis article presents the development of an expert system to support the diagnosis of post-harvest diseases of stored apples. We propose a picture-based and conversational interaction with users, where sampled images depicting symptoms ...
Highlights- Picture-based expert system to support the diagnosis of apple diseases.
- ...
Global versus individual accuracy in contextual multi-armed bandit
SAC '19: Proceedings of the 34th ACM/SIGAPP Symposium on Applied ComputingMost works on Multi-Armed Bandits focus the evaluations of their methods on a global accuracy performance metric. In the case of Contextual Multi-Armed Bandit (CMAB), the existing algorithms claim to eventually provide full personalization. This ...
Comments