Using a Delphi method and the analytic hierarchy process to evaluate Chinese search engines: A case study on Chinese search engines

The purpose of this study is to develop an evaluation criteria system for search engines using the Delphi method and the analytic hierarchy process, and to conduct an empirical study on Chinese search engines to verify it. This is because the evaluation criteria in current research rarely considers expert opinions and experience, and seldom calculates the weight of each criterion by quantitative methods.

Expert investigation of the web‐based Delphi method was employed to develop the evaluation criteria, and the analytic hierarchy process based on an exponential scale was adopted to calculate the weight of each criterion. An empirical study was designed to test the evaluation criteria system. A total of 12 study participants were invited to evaluate six popular Chinese search engines by means of e‐mail questionnaires. All the calculation processes were executed automatically by Java programming.

The indexing structure and search method were the core criteria for evaluating the search engines. Google (using simplified Chinese characters) achieved the best performance among the six typical Chinese search engines.

The criteria for search engine evaluation were determined and weighted by the web‐based Delphi method and the analytic hierarchy process based on an exponential scale.