skip to main content
10.1145/2507157.2507225acmconferencesArticle/Chapter ViewAbstractPublication PagesrecsysConference Proceedingsconference-collections
poster

Evaluating top-n recommendations "when the best are gone"

Published: 12 October 2013 Publication History

Abstract

In a number of domains of interest for recommender systems, items are characterized by constrained and variable "capacity": the same product or service can be consumed by a limited number of users and the possibility of item consumption depends on contextual circumstances (e.g., time). Our work explores recommenders in the context of these "bounded" domains. We consider online hotel booking as a case study, and investigates if and how "missing" items (hotels that eventually becomes unavailable for users' consumption) affect the quality of recommendations. The paper proposes a technique for defining "missing" items as "best items", and presents an articulated empirical research in which recommendations for hotel online booking are evaluated in different experimental conditions with a user centric approach involving 142 participants.

References

[1]
Bell, R., Koren, Y., Scalable Collaborative Filtering with Jointly Derived Neighborhood Interpolation Weights. In Proc. ICDM '07. IEEE, pages 43--52, 2009.
[2]
Burke, R., Hybrid web recommender systems. In The adaptive web, Brusilovsky P., Kobsa A, Nejdl W. (Eds.). LNCS Vol. 432 Springer-Verlag, pages 377--408, 2007.
[3]
Cremonesi, P., Koren, Y. and Turrin, R., Performance of recommender algorithms on top-n recommendation tasks. In Proc. RecSys '10. ACM, pages 39--46, 2010.
[4]
Cremonesi P., Garzotto F., Smoothly Extending e-Tourism Services with Personalized Recommendations: A Case Study. In Proc. EC-Web 2013. Springer, 2013 (to appear)
[5]
Lops, P., De Gemmis, M., and Semeraro, G., 2011. Content-centric recommender systems: State of the art and trends. In Recommender Systems Handbook, F. Ricci, L. Rokach, B. Shapira, P. B. Kantor (Eds). Springer, pages 73--105, 2011.
[6]
Marlin, B.M. and Zemel, R.S., Collaborative prediction and ranking with non-random missing data. In Proc. RecSys '09. ACM, New York, NY, USA, 2009, 5--12.
[7]
Porter, S. R., and Whitcomb, M. E., The Impact of Lottery Incentives on Survey Response Rates. Research in Higher Education, 44(4), 389--407, 2003.
[8]
Pradel, B., Usunier, N. and Gallinari, P., Ranking with non-random missing ratings: influence of popularity and positivity on evaluation metrics. In Proc. RecSys '12. ACM, pages 147--154, 2012.

Cited By

View all
  • (2015)Smart booking without lookingProceedings of the 15th International Conference on Knowledge Technologies and Data-driven Business10.1145/2809563.2809616(1-4)Online publication date: 21-Oct-2015
  • (2015)What recommenders recommendUser Modeling and User-Adapted Interaction10.1007/s11257-015-9165-325:5(427-491)Online publication date: 1-Dec-2015

Index Terms

  1. Evaluating top-n recommendations "when the best are gone"

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      RecSys '13: Proceedings of the 7th ACM conference on Recommender systems
      October 2013
      516 pages
      ISBN:9781450324090
      DOI:10.1145/2507157
      • General Chairs:
      • Qiang Yang,
      • Irwin King,
      • Qing Li,
      • Program Chairs:
      • Pearl Pu,
      • George Karypis
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      In-Cooperation

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 12 October 2013

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. e-tourism
      2. missing items
      3. perceived accuracy
      4. perceived quality
      5. top-n recommendation task
      6. user-centric evaluation

      Qualifiers

      • Poster

      Conference

      RecSys '13
      Sponsor:

      Acceptance Rates

      RecSys '13 Paper Acceptance Rate 32 of 136 submissions, 24%;
      Overall Acceptance Rate 254 of 1,295 submissions, 20%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)4
      • Downloads (Last 6 weeks)0
      Reflects downloads up to 16 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2015)Smart booking without lookingProceedings of the 15th International Conference on Knowledge Technologies and Data-driven Business10.1145/2809563.2809616(1-4)Online publication date: 21-Oct-2015
      • (2015)What recommenders recommendUser Modeling and User-Adapted Interaction10.1007/s11257-015-9165-325:5(427-491)Online publication date: 1-Dec-2015

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media