1 Introduction

Search has reached a level at which a good understanding of user interactions may significantly impact its quality. Among all kinds of user interactions, click-through behavior on search results is an important one that attracted much attention. Clicking a certain result (or advertisement, or query suggestions, etc.) is usually regarded as an implicit feedback signal for its relevance, which is, however, very noisy. To understand if and how much a user click on a result document implies true relevance, one has to take into account different factors (usually named behavior biases), in addition to the factor of relevance, that may affect user click behaviors. Joachims et al. (2005) worked on extracting reliable implicit feedback from user behaviors, and concluded that click logs are informative yet biased. Previous studies revealed several bias aspects such as “position” (Craswell et al. 2008), “trust” (O’Brien and Keane 2006) and “presentation” (Wang et al. 2013) factors. Recently, we have also witnessed the rising of ranking models which rely on click-through data as a biased noisy information source for training purposes (Wang et al. 2016; Joachims et al. 2017).

Several click models (e.g. Dupret and Piwowarski 2008; Chapelle and Zhang 2009; Guo et al. 2009) have been proposed, which usually involve additional events (e.g. examination) and different assumptions. These models are designed to eliminate the effects of various behavior biases (e.g. position bias, presentation bias, trust bias, etc.) to provide a better estimation of result relevance. Many of these efforts have been adopted to generate useful ranking signals for production rankers of commercial search engines.

Although many existing works show that click models help improve performance of search engines, a lot more research questions remain to be investigated. For example: How are clicks influenced by user’s background and preferences? by multimedia verticals in a result page? by the underlying search task and the search session? How can we derive a reliable evaluation measure of search quality from user’s click-through? How can we gain information on the quality of documents using user clicks? How should the user interface designed to facilitate the collection of reliable click-through?… The purpose of this special issue of Information Retrieval Journal is to highlight researches that have the potential to answer some of these questions and promote information retrieval research.

2 Overview of papers

We received five submissions for the special issue, of which three were accepted. Each paper has been reviewed by at least three professional reviewers. Both the reviewers and the authors have worked hard to improve these works to make them both technically-solid and inspiring. They provide different perspectives about click behavior modeling and outline the directions for future research.

In Modeling Users’ Search Sessions for High Utility Query Recommendation, the authors propose a joint Query Utility Model to model search users’ query reformulation and click-through behavior simultaneously. The motivation is to design a query recommendation method that not only provides attractive relevant queries, but also helps users to finish their ultimate search goals. They start by defining the utility of a query as a product of both perceived utility (before clicking search results) and posterior utility (after reading the landing pages). After that, they design a joint model based on the framework of dynamic Bayesian network and learn query utility based on users’ sequential behaviors in sessions. The effectiveness is tested on a publicly available dataset named UFindIt and the proposed model outperforms some strong query recommendation baselines.

In Decoding Multi-Click Search Behavior Based on Marginal Utility, the authors focus on the context factors on search engine result pages that may influence users’ click-through behaviors. They name the kind of factor as “novelty bias” and try to explain users’ multi-click behavior based on both relevance and content similarity of result document. Although there are some existing works which try to take results’ content into consideration while modeling clicks. The proposed framework features the adoption of maximum marginal utility model, which is usually adopted in diversified search researches. The authors evaluate the performance of the proposed model on both AOL (English) and Sogou (Chinese) log data sets. They use both click prediction and relevance estimation results to show effectiveness of their method compared with UBM.

In Enhancing click models with mouse movement information, the authors try to predict users’ examination behavior with mouse movement behavior and incorporate the prediction results into click models. Examination behavior is one of the key factors in users’ search interaction process and most click models try to model this implicit user behavior with certain assumptions (e.g. cascade assumption). Different from these traditional efforts, the proposed model investigates the correlation of examination and mouse movement features with a lab-based eye-tracking study. The extracted features are then adopted to predict users’ examination from practical behavior logs collected from a commercial search engine. Results show that the incorporation of mouse movement behavior is useful in both click prediction and relevance estimation.