Severin Hacker
Luis von Ahn
ABSTRACT
Eliciting user preferences for large datasets and creating rankings based on these preferences has many practical applications in community-based sites. This paper gives a new method to elicit user preferences that does not ask users to tell what they prefer, but rather what a random person would prefer, and rewards them if their prediction is correct. We provide an implementation of our method as a two-player game in which each player is shown two images and asked to click on the image their partner would prefer. The game has proven to be enjoyable, has attracted tens of thousands of people and has already collected millions of judgments. We compare several algorithms for combining these relative judgments between pairs of images into a total ordering of all images and present a new algorithm to perform collaborative filtering on pair-wise relative judgments. In addition, we show how merely observing user preferences on a specially chosen set of images can predict a user's gender with high probability.
- von Ahn, L. and Dabbish, L. Labeling images with a computer game. In Proc. SIGCHI Conf. on Human Factors in Computing Systems (Vienna, April 24-29). ACM, New York, 2004, 319--326. Google Scholar
Digital Library
- Law, E. and von Ahn, L. Input-Agreement: A New Mechanism for Collecting Data using Human Computation Games. To appear in Proc. of the SIGCHI Conf. on Human Factors in Computing Systems (Boston, April 4-9). ACM, New York, 2009. Google ScholarDigital Library
- Lee, B. and von Ahn, L. Squigl: A Web game to generate datasets for object detection algorithms. In submission.Google Scholar
- von Ahn, L., Liu, R., and Blum, M. Peekaboom: a game for locating objects in images. In Proc. SIGCHI Conf. on Human Factors in Computing Systems (Montréal, April 22-27). ACM, New York, 2006, 55--64. Google ScholarDigital Library
- von Ahn, L., Kedia, M., and Blum, M. Verbosity: a game for collecting common-sense facts. In Proc. SIGCHI Conf. on Human Factors in Computing Systems (Montréal, April 22-27). ACM, New York, 2006, 75--78. Google ScholarDigital Library
- Flickr, a Web site for photo-sharing. http://www.flickr.comGoogle Scholar
- Butterfield; Daniel S.; et al. Interestingness ranking of media objects. U.S. Patent application 2006/0242139 A1, filed February 8, 2006.Google Scholar
- Digg, a Web site for social bookmarking. http://www.digg.comGoogle Scholar
- Youtube, a Web site for sharing videos. http://www.youtube.comGoogle Scholar
- Hot or Not, a Web site for rating pictures of people. http://www.hotornot.comGoogle Scholar
- Gajos, K. and Weld, D. S. Preference elicitation for interface optimization. In Proc. 18th Annual ACM Symp. on User Interface Software and Technology (Seattle, Oct. 23-26). ACM, New York, 2005, 173--182. Google ScholarDigital Library
- GWAP, a Web site for Games with a Purpose. http://www.gwap.comGoogle Scholar
- Elo, Arpad. The Rating of Chessplayers, Past and Present. Arco Publications, New York, 1978.Google Scholar
- Herbrich, R., Minka, T., and Graepel, T. TrueSkillTM: A Bayesian Skill Rating System. In Proc. Advances in Neural Information Processing Systems 19, MIT Press, Cambridge, MA, 2007, 569--576.Google Scholar
- G. Takács, I. Pilászy, B. Németh, and D. Tikk. On the Gravity Recommendation System. In Proc. 13th ACM SIGKDD Intl. Conf. on Knowledge Discovery and Data Mining (San Jose, August 12-15). ACM, New York, 2007, 22--30.Google Scholar
- Amazon Mechanical Turk. http://www.mturk.comGoogle Scholar
Index Terms
- Matchin: eliciting user preferences with an online game
Recommendations
Eliciting single-peaked preferences using comparison queries
AAMAS '07: Proceedings of the 6th international joint conference on Autonomous agents and multiagent systemsVoting is a general method for aggregating the preferences of multiple agents. Each agent ranks all the possible alternatives, and based on this, an aggregate ranking of the alternatives (or at least a winning alternative) is produced. However, when ...
Tutorial on preference handling
RecSys '13: Proceedings of the 7th ACM conference on Recommender systemsThis tutorial, addressed to researchers and practitioners in the area of recommendation systems, reviews the foundations of decision and utility theory, focusing on methods for representing, learning and reasoning with preferences. Recent advances on ...
Preferences in artificial intelligence
The paper presents a focused survey about the presence and the use of the concept of "preferences" in Artificial Intelligence. Preferences are a central concept for decision making and have extensively been studied in disciplines such as economy, ...
Comments