skip to main content
10.1145/3132847.3133059acmconferencesArticle/Chapter ViewAbstractPublication PagescikmConference Proceedingsconference-collections
short-paper

Ranking Rich Mobile Verticals based on Clicks and Abandonment

Published: 06 November 2017 Publication History

Abstract

We consider the problem of ranking rich verticals, which we call "cards," for a given mobile search query. Examples of card types include "SHOP" (showing access and contact information of a shop), "WEATHER" (showing a weather forecast for a particular location), and "TV" (showing information about a TV programme). These cards can be highly visual and/or concise, and may often satisfy the user's information need without making her click on them. While this "good abandonment" of the search engine result page is ideal especially for mobile environments where the interaction between the user and the search engine should be minimal, it poses a challenge for search engine companies whose ranking algorithms rely heavily on click data. In order to provide the right card types to the user for a given query, we propose a graph-based approach which extends a click-based automatic relevance estimation algorithm of Agrawal et al., by incorporating an abandonment-based preference rule. Using a real mobile query log from a commercial search engine, we constructed a data set containing 2,472 pairwise card type preferences covering 992 distinct queries, by hiring three independent assessors. Our proposed method outperforms a click-only baseline by 53-68% in terms of card type preference accuracy. The improvement is also statistically highly significant, with p ≈ 0.0000 according to the paired randomisation test.

References

[1]
Rakesh Agrawal, Alan Halverson, Krishnaram Kenthapadi, Nina Mishra, and Panayiotis Tsaparas. 2009. Generating labels from clicks. In Proceedings of ACM WSDM 2009. 172--181.
[2]
Aleksandr Chuklin and Pavel Serdyukov. 2012. Potential Good Abandonment Prediction. In WWW 2012 Companion. 485--486.
[3]
Qi Guo and Yang Song. 2016. Large-Scale Analysis of Viewing Behavior: Towards Measuring Satisfaction with Mobile Proactive Systems. In CIKM 2016.
[4]
Thorsten Joachims, Laura Granka, Bing Pan, Helene Hembrooke, Filip Radlinski, and Geri Gay. 2007. Evaluating the accuracy of implicit feedback from clicks and query reformulations in web search. ACM TOIS 25, 2 (2007).
[5]
Yuta Kadotami, Yasuaki Yoshida, Sumio Fujita, and Tetsuya Sakai. 2017. Mobile Vertical Ranking based on Preference Graphs. In Proceedings of ACM ICTIR 2017.
[6]
Dmitry Lagun, Chih-Hung Hsieh, and Dale Webster Vidhya Navalpakkam. 2016. Towards better measurement of attention and satisfaction in mobile search. In Proceedings of SIGIR 2014. 113--122.
[7]
Jane Li, Scott Huffman, and Akihito Tokuda. 2009. Good abandonment in mobile and PC internet search. In Proceedings of ACM SIGIR 2009. 43--50.
[8]
Eldar Sadikov, Jayant Madhavan, Lu Wang, and Alon Halevy. 2010. Clustering Query Refinements by User Intent. In Proceedings of WWW 2010. 841--850.
[9]
Tetsuya Sakai. 2012. Evaluation with informational and navigational intents. In Proceedings of WWW 2012. 499--508.
[10]
Yang Song, Xiaolin Shi, Ryen White, and Ahmed Hassan Awadallah. 2014. Context-aware web search abandonment prediction. In Proceedings of ACM SI- GIR 2014. 93--102.
[11]
Kyle Williams, Julia Kiseleva, Aidan C. Crook, Imed Zitouni, Ahmed Hassan Awadallah, and Madian Khabsa. 2016. Detecting Good Abandonment in Mobile Search. In Proceedings of WWW 2016. 495--505

Cited By

View all
  • (2020)Good Evaluation Measures based on Document PreferencesProceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3397271.3401115(359-368)Online publication date: 25-Jul-2020

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CIKM '17: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management
November 2017
2604 pages
ISBN:9781450349185
DOI:10.1145/3132847
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 November 2017

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. click data
  2. good abandonment
  3. mobile search
  4. vertical ranking

Qualifiers

  • Short-paper

Conference

CIKM '17
Sponsor:

Acceptance Rates

CIKM '17 Paper Acceptance Rate 171 of 855 submissions, 20%;
Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)4
  • Downloads (Last 6 weeks)0
Reflects downloads up to 10 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2020)Good Evaluation Measures based on Document PreferencesProceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3397271.3401115(359-368)Online publication date: 25-Jul-2020

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media