Automatic new topic identification using multiple linear regression

Abstract

The purpose of this study is to provide automatic new topic identification of search engine query logs, and estimate the effect of statistical characteristics of search engine queries on new topic identification. By applying multiple linear regression and multi-factor ANOVA on a sample data log from the Excite search engine, we demonstrated that the statistical characteristics of Web search queries, such as time interval, search pattern and position of a query in a user session, are effective on shifting to a new topic. Multiple linear regression is also a successful tool for estimating topic shifts and continuations. The findings of this study provide statistical proof for the relationship between the non-semantic characteristics of Web search queries and the occurrence of topic shifts and continuations.

Introduction

Search engines are becoming a major tool to access information over the Web for many people. It is important, for this reason, to study the behavior of search engine users. One dimension of search engine user profile is content-based behavior. Currently, search engines are not designed to differentiate according to the user’s profile and the content that the user is interested in, and personalization and context of search engine use have been recognized as major research challenges in a number of workshops (Liu, Croft, Oh, & Hart, 2004). However, exploiting the user’s interest in various topics has the potential to improve Web retrieval systems (Goker, 1997, Talja et al., 1999). The capability of understanding or at least estimating the user interests could be a significant step towards the development of intelligent search engines.

One of the main elements in developing an intelligent search engine is new topic identification. New topic identification is discovering when the user has switched from one topic to another during a single search session. If the search engine is aware that the user’s new query is on the same topic as the previous query, the search engine could provide the results from the document cluster relevant to the previous query, or alternatively, if the user is on a new topic, the search engine could resort to searching other document clusters. Consequently, search engines can decrease the time and effort required to process the query. In addition, custom-tailored graphical user interfaces can be offered to the Web search engine user, if topic changes were estimated correctly by the search engine. Ozmutlu, Ozmutlu, and Spink (2003a) mention that users interested in different topics could benefit more from such IR systems designed according to their searching needs. Some features of custom-tailored IR systems, which are more sensitive to users’ various information needs are mapped out by Ozmutlu et al. (2003a). Had topic identification been successfully performed, sophisticated graphical user interfaces could be offered by search engines that can help users (a) enable the reformulation of multiple queries on different or related topics, and facilitate task switching, i.e. allowing the tracking, storing and manipulating of retrieved results and printouts related to different topics over multiple searches, (b) provide the ability to create multiple sets of working notes related to different or related search topics, i.e., sketching and note creation tools, (c) enable Web users to submit and track multiple queries concurrently on different or related topics, (d) allow for searching multiple search engines or collections concurrently on multiple topics, (e) review search histories from various searches and topics, and provide the ability to create clusters of retrieved information related to different or related topics.

There are few studies on query clustering and new topic identification, presented in more detail in the related research section. The studies generally analyzed the queries semantically. Semantic analysis of queries is a promising line of research, but is a complicated task, hence its current success is ambiguous. In our previous studies, we applied content-ignorant methodologies for automatic new topic identification, such as Dempster–Shafer Theory and genetic algorithms (Ozmutlu and Cavdur, 2005a, Ozmutlu et al., submitted for publication) and neural networks (Ozmutlu and Cavdur, 2005b, Ozmutlu et al., 2004a). These methodologies rely on the statistical characteristics of the queries, such as the time between query submissions and the reformulation of the subsequent queries, instead of the meaning of the queries. The initial indications of the relation between statistical characteristics of queries and topic change were shown in Spink et al., 2002b, Goker and He, 2000 and He and Goker (2000). However, none of these studies demonstrate the statistical significance of the relationship between the non-semantic characteristics of queries and the timing of topic shifts and continuations.

In this study, we aim to estimate topic shifts in search engine query logs using multiple linear regression and demonstrate the statistical significance of the relationship between non-semantic characteristics of query logs and topic shifts/continuations. Using the characteristics of the search queries as independent factors and the existence of topic shifts as the dependent factor, multiple linear regression is applied to investigate the relationship between statistical characteristics and topic shifts. We also apply ANOVA to examine the structure of the variance of the topic shifts with respect to the statistical characteristics of the search queries. These studies will be helpful in identifying whether there is a relationship between statistical characteristics of the search queries and topic shifts/continuations. If such a relationship exists, content-ignorant methodologies can be expected to be successful.

We initially present the literature review related to topic identification, followed by the description of the methodology, results and the conclusion.